en Uncategorized 4,699 palabras

Client Side Development with OpenERP

Frederic van der Essen on Hacking & Design

Posted on June 7, 2013

So you want to understand how OpenERP’s web client works ? Then welcome, you’re at the right place. Altough I have to warn you, this is not an exhaustive documentation, nor even a tutorial. This is just an overview that covers the basics. It’s what you need to get started, but you’ll have to dig deeper and follow the links and pointers before you can start being productive.

To make this overview useful to JavaScript and HTML5 newcomers, I’ll dedicate the first chapters to a general introduction of what is web client development and the technologies involved. If you are already familiar with HTML5 you can skip those chapters.

And because OpenERP client side development cannot be isolated from the broader field of web application development, this overview will sometimes stray way from OpenERP to examine other popular technologies used today.

And please keep in mind that OpenERP’s web client is in constant evolution, which partly explains the bad state of its documentation, but also means that this document will become quickly obsolete. This document is thus only relevant to version 7.0.

What is a Web application ?

A web application is simply an application that is delivered and used in a web browser, but the term has recently taken a more specific meaning.

The old way to make a web application, and the way OpenERP worked until version 6.0 is to make the server send to the user complete HTML documents representing the state of the application’s GUI. This means the server has to compute and send a new HTML document for each interaction; buttons clicks, searches, history navigation all require the server to resend a document.

This puts a large load on the server and thus severely impact the number of concurrent users that can be served. It also creates a large latency in the application that makes the implementation of many features impossible, and limits what can be achieved in terms of usability.

The solution is to create a complete and standalone application in JavaScript and HTML that runs on the user’s web browser. So the Web-Client is very similar in architecture to a traditional GTk Client. The only difference, besides the programming language and libraries used, is that the Web Client is downloaded and run by the user’s browser each time he visits the OpenERP website.

The downside to this approach is that it makes content indexing by search engines very difficult since they now have to download and run an entire application instead of simply reading text documents. But this is not a problem for most web applications since their content are transient and not meant to be indexed.

JavaScript

JavaScript is the language understood by web browsers and thus a de-facto language for web applications. If you want to develop for OpenERP’s Web client you’ll need to know JavaScript.

JavaScript is a poor programing language (largely because it was a rushed design) and despite its name it has absolutely nothing in common with Java (not implying it would be a good thing). Actually they did try to make the syntax look like Java, for marketing purposes, and they named after it, again just for marketing purposes. But the language is in fact closer to lisp, even if it tries hard to conceal this fact.

But JavaScript’s shortcomings are compensated by the community having identified, documented and popularized the good patterns and produced excellent libraries such as Underscore or JQuery. JavaScript is also very fast. Fast to start, fast to run and fast to deploy. JavaScript also gives you access to HTML, CSS, powerful multimedia API and other invaluable browser features to build excellent interactive applications.

The advantages far outweighs the inconveniences and makes JavaScript and the browser one of the best environment to develop applications for the general public.

There is also a variety of new programming languages aiming to solve JavaScript’s design issues. But they must be compiled to JavaScript before deployment making debugging more complicated. CoffeeScript is clearly the most popular. Dart is another interesting project by Google. At OpenERP we only use JavaScript. We tried CoffeeScript for some modules but we do not use it anymore.

If you want to learn or improve your JavaScript, you can try these:

For information on the various JavaScript browser API, go there:

We also use jQuery and Underscore abundantly.

We make heavy use of jQuery’s Deferred. It is not an easy subject and you’ll need to get familiar with them:

We use Backbone for the Point of Sale, and in a few other places. It also served as an inspiration for parts of our framework design choices.

For JavaScript development, most developers at OpenERP use either vim or Sublime Text 2 and use Google Chrome for testing and debugging. Some developers use huge-ass IDE likeEclipse or Aptana but we do not like these people. And nobody uses emacs because it is rubbish.

HTML & CSS

HTML and CSS are essential technologies for web applications since they are the means to display content in a web browser. Traditional applications can have direct access to the screen and use any technology to render their content. Web applications must be rendered in HTML and CSS. (You can also use <canvas> but still…)

Those technologies are very well known so I will only review things directly relevant to OpenERP’s Web client.

One big problem with HTML/CSS is the poor cross browser compatibility. To make this less painful we decided to support only the latest version and the previous for each browser vendor. This is a reasonable compromise that is made by many developers including Google.

We also make good use of SASS, a stylesheet language that compiles to CSS and provides a lot of syntax sugar and useful features. This is all done on the server side, the client only sees CSS files.

Here are some good ressources on HTML/CSS

Web Client Architecture

As I told you before, the web client is really a stand alone application developed in JavaScript. So let’s first have a look at its architecture.

Web Modules

As you probably know OpenERP is organized with modules. Those can either provide new functionalities, or alter and extend the behaviour of existing modules. The WebClient uses the same module framework and organisation, there is no technical distinction in OpenERP between ‘server’ modules and ‘web’ modules, as modules can provide functionality and extend both the server and the web client.

The Web-Client consists of several modules; there is a core webmodule that provide the core functionality of the web client. Then there is a serie of related «web modules», which happen to have aweb_ prefix to their name like web_kanban or web_calendar that provide additional functionalities for the web client. This prefix is only there for historical reasons. Now almost all modules provide web client functionality. Those «web modules» are located in alaunchpad repository separate from the other modules. This separation of repositories is not well justified and we’d like to merge these two repositories in the future. (Probably at the same time we switch from bzr to git, as we are considering that as well)

To modify the Web Client, a module must provide some JavaScript, HTML or CSS resources. Since those files are not executed by the server they are put in the static module’s subdirectory :

  • static/src/js : the JavaScript files
  • static/src/css : the CSS files
  • static/src/xml : the HTML template files
  • static/img : pictures used in templates or CSS
  • static/libs : JS libraries needed by the module

The JS, CSS and XML files may not be executed by the server but they are still processed in various ways before they are delivered to the client, and thus they must be referenced in the module manifest, the

__openerp__.py file:

# A snippet of a __openerp__.py file
{
    # ...
    'js':  [ 'static/src/js/file.js' ],
    'xml': [ 'static/src/xml/template.xml' ],
    'css': [ 'static/src/css/style.css' ],
}

Since those files are not executed, you don’t need to restart the server to see changes made to those files; just refresh your browser tab.

Delivering the Web Client

When a user loads the web client, the server must collect all the JavaScript, HTML and CSS files provided by the installed modules, package them and send them to the client. The principal challenge here is to make it fast and thus to minimise the application size and number of requests needed. This means reducing the number of files and compressing them.

All the CSS files are concatenated into one, and the same thing is done with the JavaScript files. The concatenation is ordered by the module dependencies.

The JavaScript files are then minified, a process that reduces the length of the code source by removing white space and renaming local variables to shorter aliases.

A mostly empty html page that loads the single CSS and JavaScript file is then generated and sent to the client.

Those files are served gzip compressed to further reduce the size.

The concatenation and minification process makes the application harder to debug as it isn’t possible anymore to read the JavaScript source code or map CSS declarations to their original files. We thus also provide a debug version of the application that is available by appending ?debug=1 to the URL, where the files are not concatenated or minified.

What’s in the web module

Of all the modules involved in the web client, the web module is the most important as it contains the core JavaScript and HTML resources that the other modules build upon to produce the Web Client.

Let’s have a look at some important files and what they do.

In the module base directory we have http.py which gives the server the ability to respond to HTTP requests, and thus serve the JavaScript and HTML ressources to the client, but also to communicate data from and to the Web-Client via JSON encoded requests.

The controllers/main.py contains the server code for handling specific requests. It contains for example the bootstrap HTML file that will load the JavaScript and CSS files on the browser. Sending this bootstrap page is the starting point of delivering the client to the browser.

The static/src/js directory contains all the JavaScript file that creates the core of our Web-Client.

  • boot.js : the entry point for the JavaScript on the web client; sets-up and loads the modules
  • corelib.js : the definitions of our custom Class and Mixin system, as well as the core classes of the Widget framework that we use to build our GUI.
  • data.js : the API used by the Web-Client to communicate with the server.
  • chrome.js : the basic building blocks of the GUI.
  • view_*.js : the implementation of the GUI for the basic OpenERP’s model views.

The static/src/xml contains the HTML templates associated with the abovementioned GUI elements. And static/src/css contains the SASS files that generates the CSS file associated with the abovementioned HTML.

JavaScript Module Architecture

Using a module based architecture means that we must somehow have a way in JavaScript to namespace, reference and load modules. Unfortunately, unlike Python, Javascript doesn’t provide a native and standard way to do so. But the community has popularised several design patterns that can provide a modular structure to an application.

  • The Module Pattern is very simple and straigthforward but not very standardised and requires the usage of globals possibly leading to namespace clashes.
  • CommonJS used by
  • Node.JS is a synchronous, requires server processing of the javascript code, and is thus more suitable for server-side JavaScript execution.
  • AMD used by almost everything else, is more complicated but more suited for browser applications.

I invite you to read this article that provides a good comparison of CommonJS and AMD.

When the Web-Client was designed, AMD wasn’t popularised yet, and a variation of the module pattern was chosen.

OpenERP’s client side module system uses a single global variable named openerp. This global contains a reference to every web module’s function, this is how you can define such a module function :

openerp.my_module = function(instance){
    // module code goes here
    console.log('My module has been initialized');
};

The module function must have the same name as the module.

The instance object that is passed to the initalisation function will contain all the module’s public data and APIs that are relevant to a currently running instance of the web client. It is possible to have several client instances running in parallel in the same browser window, this is mainly used when OpenERP is embedded in other webpages. In this case, the modules are loaded several times with different a instance object.

Since the module code runs inside a function, it is isolated from other modules and data not made available trough the instanceobject stays private to the module.

To avoid conflicts within modules, the module should not store its data directly into instance but instead declare a single namespace object in instance and put all its data here. It is a common practice to reference this namespace object by a modulelocal variable;

openerp.my_module = function(instance){
    var module = instance.my_module = {};

    module.CustomWidget = instance.web.Widget.exend({
        //widget code
    });

    module.data = 42;
};

A module’s code is executed after its dependencies and it is thus «safe» to directly access the dependencies and modify the code to alter its behavior in the desired way. In the following method we override the foo method of the FooWidget class of another_module.

openerp.my_module = function(instance){
    instance.other_module.FooWidget.include({
        foo: function(){
            console.log('hello!');
            return this._super();
        },
    });
};

This system has a few drawbacks:

  • It is completely custom to openerp
  • It is not possible to have multiple JavaScript files in a single module in a clean manner.
  • All the module’s JavaScript files have to be run in one go for the modules to load correctly.

The last point means we must provide from the start the code for installed but seldomly used modules and libraries, and this increases the size and thus loading time of the application. We have considered moving to AMD which would allow us to do lazy loading of libraries and modules where it makes sense. But for now those benefits don’t seem to be worth the added complexity.

Object Oriented Framework

Javascript not only lacks proper module constructs, its object system also lacks important features; the syntax to extend classes is complicated and there is no super keyword or functionality. To solve these problems We have chosen to use John Resig’s Simple JavaScript Inheritance, and it is used as the basis for all our classes. The root class is available in as instance.web.Class. Here is how you would use it.

var Foo = instance.web.Class.extend({
    init: function(a){
        // init is the constructor, and called automatically at the
        // object creation
        this.a = a;
    },
    foo: function(x){
        // a simple method
        console.log(this.a + x);
    },
});

var Bar = Foo.extend({
    init: function(a,b){
        // we have overridden the constructor.
        this._super(a); // we call the parent constructor
        this.b = b;
    },
    foo: function(x){
        this._super(this.b+x); // _super works here as well
    },
});

We can not only extend classes, we can modify them with theinclude method. All objects and classes inheriting of the modified class will see those modifications as well. This is not part of John Resig’s code, but is very useful if you want to modify some part of the web client in a new module.

Foo.include({
    foo: function(x){
        // This codes replaces the foo() method of Foo, and the
        // _super() used in Bar's foo() method.
        console.log(this.a + x * 2);
    },
});

This code is located inweb/addons/web/static/src/js/corelib.js.

Client-Server Communication

As I’ve said before the Web-Client is a true standalone application that can run on its own. It doesn’t need to communicate with the server to run. However, all the business data is on a database on a distant server.

The Web-Client only needs to communicate with the server to read the business data that it has to display, and to send back the data that the user has modified.

Reading data is very simple. You just need to specify what kind of business object you want and which fields you want to read. The server will send you back the result as a list of dictionaries containing the current values of the requested fields for each business object that matched your query parameters. Here is how you do it:

// fetching the name and emails of 15 active users
new Model('res.users').query(['name','user_email']).limit(15).all().then(function(users){
    // All interactions with the server are asynchronous, we get
    // the results in a callback
    console.log(users);
});

If the fields points to one or multiple OpenERP objects you will get a list of those object’s id. If you need to get those objects, you’ll have to do another query with an id filter.

new Model('res.company')
    .query(['name'])
.filter([['id','=',user.company_id[0]]])
.all().then(function(companies)){
    console.log(company[0].name);
});

Sending data is quite different as the server cannot trust the validity and accuracy of the data sent by the client. Server-side data modification is done by calling OpenERP object methods on the server via RPC.

// changing the password of the current user
new Model('res.users').call('change_password',['oldPassword','newPassword']);

Presuming that the user logged into the Web-Client has the required permissions, you can call any OpenERP object server method by this mechanism.

Building the screen

One essential part of the web client is to build the web page that is displayed and react to the user interactions. The nice thing about doing this on the client is that we do not need to communicate with the server to do this, and that we don’t have to re-render the entire web page at each modification.

The application starts as an empty page and the html is generated entirely from JavaScript. When part of the page changes, we rebuilt those changes in place. The application thus stays on the same page at all times.

Widgets

This is done with what we call Widgets. A Widget is a JavaScript object that is represented on the user screen by some HTML. The screen is built by composing widgets in a hierachical manner, each widget potentially having sub Widgets, and the whole screen being contained in a single big root Widget.

In addition of representing some part of the displayed HTML, a Widgets is responsible of many things :

  • the generation and destruction of its associated HTML.
  • the modification of its associated HTML resulting from user interaction with the widget
  • to fetch the business data displayed by the widget.
  • to send to the server the modifications of business data resulting from user interaction with the widget.
  • to create, position and destroy its sub-widgets

The usage of widgets are thus very varied, from Widgets representing the whole application screen, to Widgets representing some field in a form.

As a demonstration, here is a (very) simple widget that can be used to perform some action when the user clicks on it.

var ButtonWidget = instance.web.Widget.extend({
    template: 'ButtonWidget', //the name of the HTML template that will be used to display the widget
    init: function(parent, label, action){
        // the constructor. Widgets always have the parent widget as
        // first constructor parameter
    this._super(parent);
        this.label = label || 'Button';
        this.action = action || function(){};
    },
    start: function(){
        // start() is called when the DOM has been rendered. This
        // is where we can register callbacks. And we do just that
        // by registering the provided action to the click event on
        // the root DOM element of the widget (this.$el)
        this.$el.click(this.action);
    },
});

We have implemented two of the Widget’s method:

  • init(parent,args) : is the Widget’s constructor. A Widget’s constructor always takes the parent widget as the first argument, and we should not forget to call the default constructor with this._super(parent);
  • start(): is called when the widget is first shown on screen and when all his subwidgets have been started as well. It is the place to do anything that requires the widget’s HTML to actually be part of the screen’s HTML, such as binding callbacks to it’s HTML elements.

Rendering Widgets with templates

The process by which the widget creates its displayed HTML has been so far kept a mistery. Let it be no more.

The HTML is generated entierely inside the browser in JavaScript. Nothing is fetched on the server. Generating HTML by hand in JavaScript is tedious and complicated, so we delegate this to a template engine.

A template engine compiles an HTML-like definition into a JavaScript function that can generate HTML. As these JavaScript functions will only be used client-side ( no HTML rendering is done on the server ), they are all bundled with the Web-Client. The compilation process is done server-side, the Web-Client only sees the compiled functions.

We use the term template to design both the server-side template definitions, and the client-side template functions.

The template definition language used by OpenERP is called QWeb. It is a strict XML subset of HTML, adding a few tags and attributes to standard HTML. As an example template, let me show you the one associated with the abovementioned ButtonWidget.

<templates>
  <t t-name='ButtonWidget'>
    <div class='button'>
       <t t-esc='widget.label' />
    </div>
  </t>
</templates>

Each Widget has one associated template function that will generate displayed HTML. The association between the two is done with the template field of the widget corresponding to the template’s name. By convention the same name is used for both.

When the template function generates the Widget’s HTML, it can access all the widget’s fields and methods, and this is very useful to generate dynamic content;
in our example the button’s label is not hardcoded in the template, it is read during the generation from the widget’s label field.

This access to the widget’s field and methods is made with the t-esc attribute. This attribute will evaluate the JavaScript expression provided and insert the escaped evaluated string into the HTML in place of the tag containing the attribute.

In the provided expression, widget always refers to the Widget instance that rendered the template, and we have used it to insert the Button label.

As far as I know, QWeb is not publicly used outside the OpenERP project, where simpler templating languages such as Handlebarsand Jade enjoy more popularity.

Still, QWeb is a powerful templating language. Among its many feature is the ability to extend and modify other templates so that you can build your widgets in a true object oriented fashion. Doing a complete overview of the QWeb templating language is outside the scope of this document. I thus invite you to check out theofficial QWeb documentation

Using Widgets

Widgets can be used either as part of a larger widget, in which case it is instantiated and placed by the larger widget, or it represents an entire application page in which case it is instantiated via an action referred by an URL. Let’s have a look at the first case.

One good place to instantiate and place a widget is in the start()method of the larger widget, because at that point the larger widget DOM is ready.

var LargeWidget = instance.web.Widget.extend({
    start: function(){
        // we instanciate the widget with 'this' as parent.
        var button = new ButtonWidget(this,'Button!');

        // we place our button inside the HTML element of the
        // LargeWidget with a 'button-bar' class
        button.appendTo(this.$('.button-bar'));
    },
});

Setting the parent correctly is important to insure good garbage collection when the LargeWidget is destroyed.

The other way to instantiate a widget is to associate to a client action. This is quite well explained in the doc.

Independent Widgets

We put much importance into making our widgets completely independent of each other and of the context they are placed in as this allows us to embed an OpenERP widget in any web page, allowing a user to put order forms in is public website with ease.

This requires to properly namespace the widgets, especially the CSS. All CSS rules must be prefixed by a .openerp and optionally a.widget-name where widget-name is an unique identifier for the widget the CSS rules applies to.

This is enough about Widgets for today. If you want to know more you can read the specifications in the JavaScript file inweb/addons/web/static/src/js/corelib.js, or check this blog regularly as I plan to make another post on that subject.

Managing the URL

We would like the browser’s history to work within the application and we would also like to have the ability to directly jump to a specific openerp screen with a specific URL.

But Web-Applications always stay on the same page (they just modify the page). So we have to manipulate the URL and browser history via JavaScript. We use the fact that anything after the # in the URL is ignored by the server to append a serialised version of the application state to the URL. The client must also intercept URL changes to parse the state and change and render the corresponding user interface.

The way we do this is still hackish and incomplete as we face the problem of having an application state that can simply be too big to store in 2K characters, the maximum cross browser URL length.

Developping for the Web Client with Google Chrome

Almost all OpenERP developpers use Google Chrome for client side development and it is not a coincidence. Google Chrome has all the tools needed built in. Exploring the JavaScript Code, setting-up breakpoints, pausing the execution, changing the JavaScript code on the fly, examining the application log, introspection of the data, inspection of the network traffic, performance profiling; all of this is available right inside Google Chrome just by pressingCTRL+SHIFT+J on your keyboard.

I won’t write a complete tutorial on JavaScript development with Google Chrome as it simply already exists. Go check it out!

Conclusion

This is the end of this overview. I hope it gave you a clearer view of the web client. If you want to know more, you can always go to theofficial documentation . While it is still a work in progress, the topics covered are explained in much more depth than I went here. There is also help.openerp.com, a stackoverflow-like board where a good deal of questions have already been answered.

On these words, good night and happy hacking.

Deja un comentario