Blog 2.0

After a good 5 years on this blog, I have moved my blog over to a newer blog engine. Please visit blog.smsohan.com

Thank you for your interest.

Deploying on TV screens

Off late, I am working on a project to render real time business data with interesting visualizations, so people can feel the pulse of the business. For the last couple of months, I have been planning to write a detailed post about it. But after a few false starts, I am finally settling on smaller posts, telling a small part of the story each time.

So, have you ever worked on a web application that is primarily viewed through 55"+ 1080p TV screens?

Photo credits to  5mal5

We are showing real time business data, aggregated from multiple data sources as they are happening. The screens are gonna be mounted on the wall, like as you'd see in the airports. And it needs to be running 24x74

This introduces an interesting deployment challenge:

How would you reload the screen every time you re-deploy the app?

A regular web app is interactive. So, when we re-deploy the app, the users typically get the latest version as soon as they reload the page or navigate through the app. However, in an airport like setting, where information is displayed across many screens, and typically no-one is clicking it, the app needs to be aware of updates and reload itself to achieve the same. This is essential, for example, if there's an API change on the server side, the HTML/Javascript/CSS must be in sync to be able render it.

The app itself uses JSON API calls to render the live data. Each screen is somewhat like a single page app, using multiple AJAX calls to render different parts of the screen, showing different data. The API calls are all funneled through a single Javascript module. The module looks like the following (showing a simplified version for brevity): If you have noticed here, there's an extra check inside the success callback. To begin with, the page remembers the server token on reload. So, whenever there's a new token, it refreshes the page. Since all API calls are funneled through this module, this become a no-brainer to support new screens/API calls.

Our API's respond with a server token, which is guaranteed to:

1. remain same for each server deployment, and
2. change whenever there's a new deployment.

However, we still need to make sure the server token indeed ensures these two essential properties. With a little trick, this becomes trivial. For our app, we are using Capistrano to deploy our Ruby on Rails project. For those new to Capistrano, it uses a timestamped directory for each deploy, symlinking it to the current. So, it looks somewhat like the following: Every Ruby on Rails app also comes with a little method, Rails.root that returns the full path to the directory of the current deployment. So, in this example scenario, we get the following:

Rails.root #==> /app/realtime/releases/20120729083021

Since every deployment will be a new timestamp, this method ensures a unique token for each deployment. That's all we need for the api module to be aware of new deployments and auto refreshes. Here's an example controller/action (again, simplified for brevity): I liked the organic nature of this technique. It is harvesting on the available tools. Although the examples in this post show Ruby/Rails as an example, I am sure the same techniques can be applied to other technologies with the same simplicity.

Before I conclude, I would share one limitation of this technique here. Since the page reload happens on a shared api module, the reload needs to be generic, without requiring any special knowledge about the pages to reload. This pretty much means, a page needs to be able to reconstruct itself entirely from it's URL. Requiring any Javascript state beyond the URL, would probably require API specific handling to reload, killing the advantage of this technique. But the good news is, its always a good practice to rely solely on the URL to construct a page.

Thanks a ton if you've followed all the way. Stay tuned for the upcoming posts, where I will be telling the story of handling multiple API calls on a page, highlighting data changes and some other interesting bits about a real time dashboard.

My Take on Client-Side MVC

I just rolled off my last project this Wednesday. It was a great team, worked on some cutting edge technologies, mobile web, HTML5, mongo and some other fun stuff. Now that I got two "beach" days before jumping on to the next gig, I spent some time trying out Backbonejs to knockout one item from the long list of to-dos. This post is about my initial take on client side MVC, solely based on these two days with Backbonejs.

Summary

Client-side MVC is in its infancy, but promises a bright future

The Good

Writing "clean code" while doing Javascript can be quite a challenge. Doing MVC gets you a step closer to that by splitting responsibility among models, views and controllers. So, your Javascript code will be domain driven, instead of just a bunch of monolithic methods.

Your client-side MVC application can be very responsive, or even a "single page app". This is an advantage if your app needs some data from the server, but can leverage Javascript for most of its visual needs. For example, a real-time analytics dashboard can use client-side MVC to do most of the processing on the client side after fetching the data from the server.

Backbonejs offers event wiring API between your models and views. For example, say you have a model called Vehicle and a view called VehicleView, you can put the following code to wire up the "change" event on the model so that the view renders whenever the model is updated.

Using such events, it becomes a trivial process to keep all the views in sync with the models as they change across different actions.

Another advantage of client-side MVC is, it forces "eat your own dog food" as you are essentially creating an API on your server and using it first hand on the client side. So, if you are building multiple clients, for example, native mobile clients and browser web apps, you can leverage the same API.

The Bad

Ever since Ruby on Rails came into the game, we all became keen on having conventions. Can't speak for the rest of the client-side MVC projects, but Backbonejs definitely lacks in this regard. It doesn't default to any view template. Also, it doesn't have a default project structure. And even worse, no in-built testing recipe either. This means, you are left on your own to choose and debate about each of these, and more that just these, to find a standard set of tools for your project.

Having no in-built test framework reduces the emphasis on testing, this is a Big Big lacking.

I find the backbonejs MVC to be a little weird, they have Model and View, but no Controller. Instead they have a Router and a confused concept of Collection. So, for example, if you wanted to work on a resource called Vehicle, you will probably end up with the following "classes":

Vehicle model, Vehicles collection, Vehicle view, Vehicles view and Vehicle router

But as you can see here, there is no Vehicle controller. I know you can always write a controller on your own, but thats the whole point about having strong opinionated conventions.

The relation between a Collection and its Model can be confusing, specially when wiring up events. For example, a change in a model can trigger the view for the model and/or the view for its collection. The logic for handling/bubbling such events can get quite messy at times.

The Future

It took several years for server side MVC to find a good common set of conventions and tools. Then, it took a few more years for people to get used to the goodness of it. I think client-side MVC frameworks will hit a similar sweet spot, but only after we give it some time to settle on the conventions and build intelligent tools around those.

Are You Still Parsing HTML Element Ids?

To render a list, often times we use repeated HTML elements with a similar template. For example, the search results in ebay.com is a list of divs, each containing different data in the same template. In a server side, this rendering code is somewhat similar to this:

If you noticed, a product with it's database id = 123 will be rendered inside a div with id="product_123", and similarly div id="product_456" will contain the product with id = 456.

With such ids, we can write JavaScript code that can extract the database id of a selected product to do interesting things, for example:


This is a useful and a common technique. However, in this era of HTML5, we can make use of its custom attributes feature to write a clean JavaScript event handler without the help of parsing. Here's an example:


I will never parse my HTML element Id again. What about you?

Object Versioning is an Open Design Problem

This unsolvable maze is a local food from Bangladesh, known as Jilapi
Photo credits to udvranto pothik

Object Versioning is often required by a business rule, for example, to maintain an audit trail or to be able to revert to a previous version, etc. This is the 3rd time in my career where this Object Versioning requirement made me think like -

There's gotta be an easier solution! 

But, I am yet to find one. So, I am thinking it's one of those open design problems, may be.

To clarify the requirement with an example, let's consider the following scenario:

A lawyer is preparing a document for one of her clients using a software. On January 17th, she needs to take a look at the version of the same document from May last year so that she can backtrace some changes that took place during these months.

Lets assume the lawyer is using a software that stores the documents in a relational database with the following schema.

A Document has many Evidences, each provided by an EvidenceProvider

Document (id, client_id, case_id, name)
Evidence (id, document_id, evidence_provider_id, details)
EvidenceProvider(id, name)

Now, given the versioning requirement how would you design your data model?

Here's a couple of points that your design should address at a minimum:

• Going back to a version means a complete snapshot of the old version of the document. So, the version of May 1st should only bring the evidences that were there on that very day.
• As a new version is created, it should inherit all previous evidences.

As I have mentioned earlier, I am yet to find a good data model that can take care of these concerns without over-complicating everything. Let me know if you got a beautiful solution to this problem.

However, in my latest project, the requirement is even harder. It's somewhat like this:

The lawyer may have some documents in the "work in progress version". This means, if she needs to print a document for the court, she only wants to print the last "good version", skipping the "work in progress version".

Also, when there is such a "work in progress version", she needs to attach any new Evidence to both the last "good version" as well as to the "work in progress version".

Well, now you see the design of a data model for Object Versioning becomes really messy and unintuitive.

So, here's my question to you - how would you design a solution for this?