Looking for the "Sacred Grail of Programming" rebooting a website - Part 2/3     [Nov 24, 2018]

In the first of this three parts article I described the genesis of the “site” project. V1, the first version, accomplished to have my personal website giandisa.it hosting another my pet project: the Rilassicelle app for iphone and ipad. V1 website was responsive thanks to Bootstrap and distribuited in two datacenter across the ocean thanks to CouchDB NoSql database. I mentioned that I developed a software component based on CouchDB to distribute/update the webapp in the two datacenter.

But now, I reveal that in the latest releases of V1, I implemented some http services to provide authentication and score management for another mobile app that I was developing. The users and score database was built on CouchDB. The idea was to let the users of the mobile app game communicate with the nearest server and to let CouchDB distributing the data between the nodes. If one node should have been down the other would have been active, eventually storing the data. When the missing node would have become active again, CouchDB would have realigned the data between nodes while a software component of mine would have managed some conflicts resolution. This system should have been able to handle the so called split-brain situations when two nodes are temporaly isolated for networking problems.

So I was working a lot on the foundations of V1. But new winds were blowing in the skies of internet so I was tempted to reboot the “site” project.

V2 Blocking vs Non-Blocking: being “reactive” and the “new” event-loop paradigm

In that time, Node.js was gaining popularity. Node.js is an asynchronous event driven JavaScript runtime, designed to build scalable network applications. An advantage was to be able to use JavaScript in the client, the browser, and in the server. But the real interesting part was the rediscovery of a concurrency model based on not blocking I/O operations where callbacks are invocated by specific events. Instead, in the previous two decades the popular concurrency model was based on multiple threads with blocking I/O operations. Threads give some relatively simple abstractions to achieve concurrency and synchronization to access shared resources without conflicts. See the monitor pattern implemented natively in the Java language. The price is a lot of memory usage, high CPU usage for context switching and the risk of deadlocks. Concurrency based on events reduce a lot the resource toll at the risk of the so called callback hell, when multiple callbacks are nested one inside an other, and that a long running routine could block the execution of other routines, without some tricks.

I also wanted my site and its services to respond to billions of requests as promised by asynchronous programming techniques!!! I know, I know that the traffic of my site is ten request/hour (sob), but let me dream!!!

Even if JavaScript has proved its utility to give dynamic content to web pages, I prefer to use a language with static type checking. Static type checking IMHO is very useful because:

• a code analyzer, the compiler in case of compiled languages, can detect a lot of errors before execution
• the editor/IDE can provide some useful hints like contextual code completion, suggestions and warning on the code

Furthermore, node.js provided out–of-the-box only one event-loop for process, and needed some work, clustering etc..., to avoid a long running routine could block the system.

After some search I ended up to Vert.x by RedHat, now under Eclipse umbrella. Vert.x is a tool-kit for building reactive applications on the JVM. It supports many languages: Java, JavaScript, Groovy, Ruby, Ceylon, Scala and Kotlin with idiomatic APIs for every language. It’s general purpose and unopinionated. An other important thing: Vert.x was very well documented!!! I choosed to use the vanilla Java flavour, so I could reuse my experience on this language and some code of V1 project.

For V2 I would have kept Bootstrap for responsive web UI and CouchDB as persistence layer. Web pages would be server dinamically rendered. I could not ever use JSP and the Stripes Framework. In the past I had used some hand-made template engine and I had evaluated some template engine library. I choosed the flexible Chunk Templates that I was using in another project of mine. I appreciated the plus to have many sections/templates per file beyond the minimal common directives to support inline conditionals and iterations. To interact with CouchDB, I had to abandone the blocking library LightCouch, and I developed a client that access CouchDB via http REST calls. Using Vert.x, I had to master the new asyncronuous paradigm. It was challenging but I published my brand new web site with an asyncronous core. In subsequent releases I also added an articles menu to access articles written in markdown. All the other features of V1 were ported to V2, even the system to distribuite the application binary between datacenter using CouchDB and much part of the experimental services for a my mobile game.

The problem that I felt mainly was managing the various callback to have a code full asyncronous. I now understand the phrase callback hell!!! It’s very challenging managing asynchronuos programming even using Promises/Futures that help to compose parts of event handling.

After some months I considered other possibilities offered by Vert.x to relieve from direct event handling, like the support for RxJava or Vert.x Sync.

Vert.x Sync was really interesting because I could keep using a syncronous style of programming even if the execution would be asyncrounous thanks to “fibers”. Fibers are a sort of very lightweight threads that can be blocked without blocking a kernel thread. But Vert.x Sync, at that time, didn’t seem to me very mature and It needed more tooling as Quasar Agent/Ahead Of Time instrumentation, so I leaved up.

I really need a much more simple way to develope new features in my site project. Asynchronous/Event programming was very interesting, but required too much effort. So my search for new solutions continued.

V3 Back to the future: a minimal traditional approach

Looking for a more simple approach to extend the functionalities of “site” project, I decided, sadly, to turn back to the origins, merging V1 and V2 projects using the lessons that I had learned.

Tomcat, servlets, no JSP, but a template engine, no Spring but a class mimic an ApplicationContext for wiring the components as prescribed by IoC pattern. All synchronous and multithreaded.

I was not convinced and V3 never came to conclusion and to a public release.

I was looking for something different and I’ll describe the next step of my quest in the third and final part of this article.

Gianni Di Sanzo