Blog 2.0

After a good 5 years on this blog, I have moved my blog over to a newer blog engine. Please visit blog.smsohan.com

Thank you for your interest.

Deploying on TV screens

Off late, I am working on a project to render real time business data with interesting visualizations, so people can feel the pulse of the business. For the last couple of months, I have been planning to write a detailed post about it. But after a few false starts, I am finally settling on smaller posts, telling a small part of the story each time.

So, have you ever worked on a web application that is primarily viewed through 55"+ 1080p TV screens?

Photo credits to  5mal5

We are showing real time business data, aggregated from multiple data sources as they are happening. The screens are gonna be mounted on the wall, like as you'd see in the airports. And it needs to be running 24x74

This introduces an interesting deployment challenge:

How would you reload the screen every time you re-deploy the app?

A regular web app is interactive. So, when we re-deploy the app, the users typically get the latest version as soon as they reload the page or navigate through the app. However, in an airport like setting, where information is displayed across many screens, and typically no-one is clicking it, the app needs to be aware of updates and reload itself to achieve the same. This is essential, for example, if there's an API change on the server side, the HTML/Javascript/CSS must be in sync to be able render it.

The app itself uses JSON API calls to render the live data. Each screen is somewhat like a single page app, using multiple AJAX calls to render different parts of the screen, showing different data. The API calls are all funneled through a single Javascript module. The module looks like the following (showing a simplified version for brevity): If you have noticed here, there's an extra check inside the success callback. To begin with, the page remembers the server token on reload. So, whenever there's a new token, it refreshes the page. Since all API calls are funneled through this module, this become a no-brainer to support new screens/API calls.

Our API's respond with a server token, which is guaranteed to:

1. remain same for each server deployment, and
2. change whenever there's a new deployment.

However, we still need to make sure the server token indeed ensures these two essential properties. With a little trick, this becomes trivial. For our app, we are using Capistrano to deploy our Ruby on Rails project. For those new to Capistrano, it uses a timestamped directory for each deploy, symlinking it to the current. So, it looks somewhat like the following: Every Ruby on Rails app also comes with a little method, Rails.root that returns the full path to the directory of the current deployment. So, in this example scenario, we get the following:

Rails.root #==> /app/realtime/releases/20120729083021

Since every deployment will be a new timestamp, this method ensures a unique token for each deployment. That's all we need for the api module to be aware of new deployments and auto refreshes. Here's an example controller/action (again, simplified for brevity): I liked the organic nature of this technique. It is harvesting on the available tools. Although the examples in this post show Ruby/Rails as an example, I am sure the same techniques can be applied to other technologies with the same simplicity.

Before I conclude, I would share one limitation of this technique here. Since the page reload happens on a shared api module, the reload needs to be generic, without requiring any special knowledge about the pages to reload. This pretty much means, a page needs to be able to reconstruct itself entirely from it's URL. Requiring any Javascript state beyond the URL, would probably require API specific handling to reload, killing the advantage of this technique. But the good news is, its always a good practice to rely solely on the URL to construct a page.

Thanks a ton if you've followed all the way. Stay tuned for the upcoming posts, where I will be telling the story of handling multiple API calls on a page, highlighting data changes and some other interesting bits about a real time dashboard.

My Take on Client-Side MVC

I just rolled off my last project this Wednesday. It was a great team, worked on some cutting edge technologies, mobile web, HTML5, mongo and some other fun stuff. Now that I got two "beach" days before jumping on to the next gig, I spent some time trying out Backbonejs to knockout one item from the long list of to-dos. This post is about my initial take on client side MVC, solely based on these two days with Backbonejs.

Summary

Client-side MVC is in its infancy, but promises a bright future

The Good

Writing "clean code" while doing Javascript can be quite a challenge. Doing MVC gets you a step closer to that by splitting responsibility among models, views and controllers. So, your Javascript code will be domain driven, instead of just a bunch of monolithic methods.

Your client-side MVC application can be very responsive, or even a "single page app". This is an advantage if your app needs some data from the server, but can leverage Javascript for most of its visual needs. For example, a real-time analytics dashboard can use client-side MVC to do most of the processing on the client side after fetching the data from the server.

Backbonejs offers event wiring API between your models and views. For example, say you have a model called Vehicle and a view called VehicleView, you can put the following code to wire up the "change" event on the model so that the view renders whenever the model is updated.

Using such events, it becomes a trivial process to keep all the views in sync with the models as they change across different actions.

Another advantage of client-side MVC is, it forces "eat your own dog food" as you are essentially creating an API on your server and using it first hand on the client side. So, if you are building multiple clients, for example, native mobile clients and browser web apps, you can leverage the same API.

The Bad

Ever since Ruby on Rails came into the game, we all became keen on having conventions. Can't speak for the rest of the client-side MVC projects, but Backbonejs definitely lacks in this regard. It doesn't default to any view template. Also, it doesn't have a default project structure. And even worse, no in-built testing recipe either. This means, you are left on your own to choose and debate about each of these, and more that just these, to find a standard set of tools for your project.

Having no in-built test framework reduces the emphasis on testing, this is a Big Big lacking.

I find the backbonejs MVC to be a little weird, they have Model and View, but no Controller. Instead they have a Router and a confused concept of Collection. So, for example, if you wanted to work on a resource called Vehicle, you will probably end up with the following "classes":

Vehicle model, Vehicles collection, Vehicle view, Vehicles view and Vehicle router

But as you can see here, there is no Vehicle controller. I know you can always write a controller on your own, but thats the whole point about having strong opinionated conventions.

The relation between a Collection and its Model can be confusing, specially when wiring up events. For example, a change in a model can trigger the view for the model and/or the view for its collection. The logic for handling/bubbling such events can get quite messy at times.

The Future

It took several years for server side MVC to find a good common set of conventions and tools. Then, it took a few more years for people to get used to the goodness of it. I think client-side MVC frameworks will hit a similar sweet spot, but only after we give it some time to settle on the conventions and build intelligent tools around those.

Are You Still Parsing HTML Element Ids?

To render a list, often times we use repeated HTML elements with a similar template. For example, the search results in ebay.com is a list of divs, each containing different data in the same template. In a server side, this rendering code is somewhat similar to this:

If you noticed, a product with it's database id = 123 will be rendered inside a div with id="product_123", and similarly div id="product_456" will contain the product with id = 456.

With such ids, we can write JavaScript code that can extract the database id of a selected product to do interesting things, for example:


This is a useful and a common technique. However, in this era of HTML5, we can make use of its custom attributes feature to write a clean JavaScript event handler without the help of parsing. Here's an example:


I will never parse my HTML element Id again. What about you?

Object Versioning is an Open Design Problem

This unsolvable maze is a local food from Bangladesh, known as Jilapi
Photo credits to udvranto pothik

Object Versioning is often required by a business rule, for example, to maintain an audit trail or to be able to revert to a previous version, etc. This is the 3rd time in my career where this Object Versioning requirement made me think like -

There's gotta be an easier solution! 

But, I am yet to find one. So, I am thinking it's one of those open design problems, may be.

To clarify the requirement with an example, let's consider the following scenario:

A lawyer is preparing a document for one of her clients using a software. On January 17th, she needs to take a look at the version of the same document from May last year so that she can backtrace some changes that took place during these months.

Lets assume the lawyer is using a software that stores the documents in a relational database with the following schema.

A Document has many Evidences, each provided by an EvidenceProvider

Document (id, client_id, case_id, name)
Evidence (id, document_id, evidence_provider_id, details)
EvidenceProvider(id, name)

Now, given the versioning requirement how would you design your data model?

Here's a couple of points that your design should address at a minimum:

• Going back to a version means a complete snapshot of the old version of the document. So, the version of May 1st should only bring the evidences that were there on that very day.
• As a new version is created, it should inherit all previous evidences.

As I have mentioned earlier, I am yet to find a good data model that can take care of these concerns without over-complicating everything. Let me know if you got a beautiful solution to this problem.

However, in my latest project, the requirement is even harder. It's somewhat like this:

The lawyer may have some documents in the "work in progress version". This means, if she needs to print a document for the court, she only wants to print the last "good version", skipping the "work in progress version".

Also, when there is such a "work in progress version", she needs to attach any new Evidence to both the last "good version" as well as to the "work in progress version".

Well, now you see the design of a data model for Object Versioning becomes really messy and unintuitive.

So, here's my question to you - how would you design a solution for this?

Care Driven Development: Javascript

There is * Driven Development, where they listed "all possible thing" driven development and dedicated a whole website to it! Well, I am adding one more to the list, "Care Driven Development", with an emphasis to Javascript coding.

Javascript coding, do you care enough?

CSS is the most hairy spagetti piece of almost any web project. And its not leading by a far distance to it's first cousin; Javascript. But, the good thing is, it just takes a little care to clean the bush out of Javascript and make it pretty.

Do you care enough not to use the following ever again?

• $('#deep_under_the_ocean').parent().parent().hide()?
• $.ajax(url: 'bank_accounts/transfer_money', amount: 500...)
• $('#eiffel_tower').height($('#paris').height() +  $('#eiffel_tower base').height() + $('#eiffel_tower tower').height() + ...)
• populateNewYork('east', 'north', 50, 100, 23, ...)
• $('.shark').click(function(){$('.small_fish').attacked(function(){...})})
• parseInt($('.selected').id().split('-').last())

I am compiling a list of such careless JS coding examples and this is just a start. If you have a few to add to this list, please keep sending!

Enterprise Software Projects: oh, yeah!

I just rolled off my first ever enterprise software project couple of weeks ago. After staying six months on the project and while the memory is still fresh, I thought I would write a retrospect on the project and on my role.

The program is rather big, I guess it had ~300 people for the last couple or so years. I joined late for the program. So, by the time I went there, a massive amount of work has already been done, meaning a huge learning curve for me. On retrospect, I think I did well :)

My team was small, 4-5 people including 3 devs. But we were responsible for really high value enterprise integration work, especially, linking the two most important applications in the program. This was a great place to learn some of the core business domain specific knowledge and I loved it!

However, enterprise integration stuff is full of XML messaging and translation. While XSLT works when it works, in our situation we had too much logic and stateful data involved in the translation. So, we had to write a hell lot of custom code for all the XML messaging... and it sucks! Why?

Service APIs change all the time! 

I was mostly writing Java at work in this project for the first time in my professional career. This was a big change as well as a good exposure, considering my recent work mostly being in C# and Ruby.  In retrospect, I would say, I would rather not use Java, its a great platform but offers your an aging language.

Being in the enterprise world means dealing with enterprise service buses, queues, oracle database, weblogic, load balancers, risk and compliance and you name it... sometimes the complexity of these things seemed to be unnecessarily troublesome, but then again, at times they made sense as well.

People factors matter way greater than any other factor in such a big project. In retrospect, I think a hierarchical system often leads to longer meetings, useless communication and less ownership for most people.

I will continue this post to a second episode where I will talk about development practices, challenges with multi-project communication, politics and power relations etc. Stay tuned! 

Aligning an HTML DIV Inside Another One

Across different projects, I have found people taking CSS shortcuts for translating the Photoshopped design templates into HTML. Back in the days, layout used to be all Table based and it was kind of straight forward to fit everything into grid cells. However, with CSS styling the extra flexibility to put elements in any arbitrary layout came extra responsibility - to make sure things still align nicely while being fluid to accommodate different screen resolutions and form factors.

Aligning the child div in red inside a parent div in green using CSS

Here's a quick and dead simple example to create CSS alignment for a div inside another one!

Pro Tip: Use position:relative for the parent div and position:absolute for the child div.

This will make sure the parent is positioned relatively to other elements in the page. However, the absolutely positioned child can be positioned anywhere inside the parent without making the whole layout fixed.

Using instanceof is mostly a Code Smell

When using static programming languages such as Java, often time I have seen people writing methods that accept Object as a parameter. These methods typically don't really work on any Object, but works with multiple types of classes that don't have any common base class. Here's an example of such a method:

As you can see in this example, the process method actually expects one of CleanFloor or LaunchRocket instances. However, since the two don't have a common subclass, it falls back to an Object type. And that results in the smelly code as you see in the example.

An ideal solution would be to change the design of the classes so that you can either use a common base class or a generic method. But if you can't just do that

Get rid of the instanceof anyway!

However, this doesn't need to be smelly. Turning to basics of OO programming concepts, you will recall that there's this thing called method overloading that is specifically there to solve this very problem.

It seems so obvious in this example that you might question, why someone would use the former code? Well, I have seen quite a few of them and if you search for instanceof in your Java project, I won't be surprised if you see a few code fragments that match the former example.

Nightly Build is a Warning (or Horror)

Photo credits: insidethemagic

I think, Nightly build is an anti-pattern for Continuous Integration. If you are integrating continuously, then you shouldn't need a nightly build, that would be too discrete :(

If you already have a per-commit continuous integration setup, you shouldn't need the so called nightly build unless there's something bad about the code.

Often times, builds taking >10 minutes are put in the bucket of nightly builds.
Some people put batch processing related tests under such nightly builds. A colleague told me they used nightly build to clean up resources (disk space, recreate the database etc.) on a nightly build - to ensure the apps worked fine on a fresh state.

Under most circumstances, a nighly build means something long running. That in itself can be an indication of poor performance.

But what I have seen is, when such nightly builds are broken - they are rarely looked after or cared for. Because, first thing in the morning, who the hell wants to look at that broken nightly build? Even if someone takes a look, since its long running, it might well eat up all morning to actually try/fix/commit changes and see it go green again! Since it takes longer to make the build green, people keep delaying it for later... meaning

it becomes generally acceptable to let the nightly build stay in RED. 

And once people get used the redness of the nightly build, they start ignoring it, inevitably.

So, here's what I would do:

Avoid nightly builds altogether.
Avoid nightly builds altogether.
Avoid nightly builds altogether.
Otherwise, make one person from the team responsible for the build (rotate people).

The Perils of Soft Delete

Often times, applications cannot get rid of the database records for business rules. For example, if you are a cable TV provider, you might have a customer calling you to stop the National Geographic Channel subscription from next month. Ideally you would like to delete this record, but you can't do it until the effective date:( If you delete it, the next invoice will not be able to charge for this channel, although it's still being used.

In such scenarios, its common to fall back to a soft delete model. As an example, here's a little code:


This code looks simple and harmless at a first sight. But, as you develop your app, you'll run into a lot of issues from this. Here's a short list:

1. You need to take care of cascading soft delete, for example, cancelling a customer's subscription needs to cascade down to all channels and other objects under it.
2. Whenever, you are listing the current subscriptions of a customer, you need to filter out the ended ones.
3. You need to figure out a strategy to periodically clean the ended subscriptions so that your database is not filled with outdated data.
4. If you have a business key used as a primary key as well, you will be in trouble if an ended subscription is restarted.
5. Your code will eventually have a lot of if-else blocks to apply changes to only on active objects.

So, what is the solution? Its best to avoid them as much as possible. Richard Dingwall has a detailed blog post on some alternative techniques to avoid soft-delete. But if you have to have soft delete, as shown in the example, its worth remembering the aforementioned points.

Hardcoded URLs in Javascript are too Slippery

Photo credits to Esteban Cavrico

I was working on a web project that lets you delete some records using Ajax. The following is just an example, but you might be familiar with a similar code already:


This code used to work when the DELETE endpoint was indeed under '/blog/posts'. However, at some point a developer wants to remove the '/blog' from all the endpoints to put them directly under '/posts'! You can see, in such cases it is very hard to catch the bug that would be caused by the faulty JS code.

Here's a quick recommendation:

Don't hardcode the URLs (or anything that is actually a server configuration) inside your JS. 

This is a code smell, as its duplicating code. But its also dangerous, because its hard to catch/fix these defects since they can be buried deep under an untested Javascript file. Instead feed this data from your server side code and you'll mitigate risk that way.

Excess of Private Methods is a Code Smell

Private methods, when used meaningfully, are a great tool for writing beautiful object oriented code. But as many other things in life, excess of private methods is bad, too!

In my opinion, we use private methods to:

1.  isolate a block of code to be reused inside the class.
2.  extract code from another method for code readability.

Now, taking these two use cases in mind, here's an easy conclusion:

The lower the ratio of public to private methods, the harder it is to write unit tests since the "units" are potentially larger.

I don't know if there is any rule of thumb, but you will smell it when you see your unit tests require a lot of setup and assertions. Here's a code example from the Play! framework, an MVC franework for Java developers.

ActionInvoker.java

You will see there are public methods with 100+ lines. I hope you agree with me:

"The ActionInvoker.java code is not readable"

For the sake of readability, introducing private methods with good names would help. However, that doesn't eliminate any of the possible code paths from the public methods. So, if you are lucky, you will see really long unit tests with complex setup conditions and mock expectations. Otherwise, there will be no tests at all! Without any tests for such long and complex methods, use them at your own risk. I won't :(

Disclaimer: I like the play! framework a lot. However, if you take a look at their code and if you think unit testing is important, you'll see they have a lot of rooms for improvement with simple extract method refactoring.

Quiz: C# Async and Await. What is the Output of this Code?

C# 5.0 (CTP version at this point) has support for async and await to allow clean implementation of non-blocking IO using a task-continuation model. I was just playing with it and here's the sample code to tease your brain. Tell me what is the output of this program?

How to Extend a Ruby Module?

Modules in Ruby are like classes with a few important differences. One of the differences is, a module cannot inherit from another module using the same syntax as class inheritance. However, its pretty easy to inherit a module's method into another module, by just using the include ModuleToInclude syntax. Here's an example if you are looking for one:

Should You Unit Test Interaction with Static Methods?

No. You should not. Here's an example:

If you are somehow mocking this static method, you are potentially making it dangerous for other places where this static method is used. Eventually, this will create red test results that are hard to find and make your tests dependent on each other.

How to list all classes that include a module?

Here's the code:

Should you unit test methods with a lot of mocking?

Well, anytime you have a lot of mocking, it is probably a smell. Particularly, a first smell is violation of the law of demeter. However, we were having discussion about whether or not we should write unit test for something like the following example:



As you can see, this method actually does a quite important task. It invokes the search method with specific parameters that will determine your search results. Now, we were debating whether or not the example unit test actually brings any value.

Here's what we have discussed:
Pros:
It confirms that you are actually calling the third party method with the right parameter.
Cons:
The test is too tightly coupled with the implementation, so it will break if something changes inside the method irrespective of its actual output.

But we failed to come to an unanimous decision on its usefulness. We all agreed on the point that it definitely needed some integration tests to make sure the search had actually worked.

Here are two questions for you:

1. Would you skip the unit test altogether for this method and rely completely on an integration test? Why/Why not?
2. If you are to rewrite this code, how would you write this using TDD?

Looking forward to see your input on this!

Ruby present? Method

Ruby has a nice little keyword called unless, that checks the opposite of if. So, you are probably used to a code like this:

return customer.first_name unless customer.nil?

If you haven't used present? before, you can in fact turn the above unless into a more familiar and easy to understand if statement:

return customer.first_name if customer.present?

So, in most cases when you are using unless with a negative condition, you can use present? and if instead. I find it way easier to read.

present? does the opposite of blank?. So, you will get the following:

nil.present? #=> false
[].present? #=> false
"hello".present? => true
["a"].present? #=> true

Hope it helps!

Constructors Are Methods, too!

Yes, if your constructor does something meaningful. This is just like another method and should be treated equally for unit testsing purpose. Here's an example that should need unit test:

It looks simple, so testing it should be simple, too. I would only skip the default constuctors, as long as they don't do anything interesting.