The Future is Cloudy

According to Google and Amazon, cloud computing is the next killer app of the web landscape. Could computing refers to running web applications in a vendor's computing cloud, free from the worries and expenses of maintaining and scaling your own servers and applications. Sounds amazing? It is. Most applications do not fully take advantage of their available hardware. Setting up and maintaining hardware and services is time consuming and expensive. Lastly, scaling applications is non-trivial and often requires the application to be architected from the ground up according to scalability tenets. Cloud computing can potentially make all these headaches disappear. But assuming that all applications will end up in a cloud is, as of today, unrealistic. Cloud computing restricts developers to working in a virtual sandbox. Accessing file system files is not permitted. Background threads are not permitted. The application works within the strict confines of the cloud's API. While this is probably fine for most simple web apps, it simply is too restrictive for most highly customized, full featured web applications. These restrictions will likely ease with time. When they do, cloud computing may very well become the prominent model for application development and hosting. Until then, the future is indeed cloudy.

Drag & Drop Using the Yahoo UI Library

Drag and drop is not only ubiquitous in "fat client" (desktop/PC based) applications, it is also widely understood by end users and often expected. Recent advances and a number of capable third-party utilities have brought this capability within reach of most web applications. One such utility is the Drag & Drop from Yahoo! UI Library. It allows developers to declare draggable objects and destinations. The API provides javascript call-back event-handlers to implement presentational and business behaviours (such as making an Ajax call when a hypothetical draggable object schedulingEvent is dragged and dropped into a calendarDayCell object).

The following code hosted on the Betterdot website demonstrates how easy it is to implement Drag & Drop on a page using the Yahoo UI Library.

Client Side Charting

Move over server-side charting, there is a new sheriff in town. His name is flash-based client-side charting. While a server-side charting library like Cewolf/JfreeChart produce charts as images server-side, a new (not so new but gaining ground) breed of charting utilities leverages the superior dynamic capabilities of Flash to build charts on the client side. The chart is typically populated by an XML or JSON payload, either on the page or sourced from an Ajax call. Users love the polished look and dynamic behaviours of these charts. Developers love the ease of implementation and natural separation of the model (data layer) and the view (presentation). FusionCharts is an excellent charting library that offers many types of common and complex charts. It is a commercial product however. Yahoo has one such product too although the product is currently listed as experimental (so deploy at your own risk...).
Client side charting offers clear advantages over server-side charting in terms of reduced complexity, shortened development cycles and encapsulation. Provided serving a flash widget is permissible in the client's specific environment, client side charting is a very compelling option and one that should gain even more ground in the months and years ahead.

Business Rule Engine

Much of the development in server-side programming in recent years has focused on declarative, loosely coupled components. This essentially amounts to building components (aka beans) and using XML descriptors to assemble business processes around these components. There are definite advantages to doing this, shortened development cycles and more flexibility around change management being two of the most compelling. The flag bearer of this movement is probably the wonderful Spring framework. While Spring is multi-talented, general purpose framework, there is another core technology most businesses need: a Business Rule Engine. A Business Rule Management System (BRMS), as these products are formally known, allows developers to declare and implement business rules, outside the realm of compiled code. The very capable, full-featured, open-source (and unfortunately named...) Drools from JBoss labs, is an BRMS tool that allows rules to be described using business semantics close to natural language. Since business rules is the portion of an application most likely to change over time, decoupling the rules from the core product makes sense. A BRMS is particularly well suited to data transformation, data validation and the implementation of any rule-based, conditional business process.

GZIP Compression for Improved Page Loads

All modern browsers support GZIP compression (by sending a header to the server, Accept-Encoding: gzip). Enabling gzip server-side for HTML and XML significantly reduces the footprint of the server response which in turns improves the page load-time (latency). It also saves bandwidth costs. The cycles required to zip the response server-side and unzip it client side, are more than offset by the savings in bytes transmitted. In Tomcat 5.x, this is simply a matter of declaring gzip compression for the connector node in server.xml.

Compress your Javascript and CSS files using Yahoo's YUI Compressor

The YUI Compressor from the Yahoo UI Library is an effective and easy to use compressor for js and css files. By eliminating white spaces and shortening local variables, I have found the tool compresses files by as much as 50%. This is particularly compelling for large web properties. Smaller downloads translate into bandwidth savings and lower page latency. Translation: costs less and makes users happy! The compressor comes as an executable jar file whose commands are easily included in an Ant deployment script. This seems preferable anyways since while the compressed files have smaller footprints, they are not particularly legible. Compressing at deployment time allows developers to continue working with a original, human-readable files.

JSON

JSON (JavaScript Object Notation) is a fast-rising data interchange format. As such it is an alternative to the XML format for data interchange. Unlike XML, JSON is a subset of the Javascript language. While JSON has a smaller footprint than XML, its most compelling advantage is that JSON sets are Javascript arrays. This is particularly advantageous to Ajax-rich pages where the page controller is typically a Javascript event. A basic but functional implementation of JSON in Java is available on the json.org site.

RSS/Atom Publishing Made Easy

Rome is an easy to use and configure Java library for creating RSS and Atom feeds. As with all such libraries, Rome lets developers write feeds using a simple object model. Rome is responsible for producing the feed compliant XML. The product supports several flavours of RSS and Atom.
While Rome works well as a servlet, large web properties are probably better off producing feeds as a scheduled batch or background thread and saving them to the local file system for a web server to handle. This allows the application server to focus on what it does best (serve dynamic requests) while delegating the serving of static content to web servers.

It's Good to Have Principles!

In software development as in life, it's good to have principles. Design principles are essential to discuss and document in the early days of a software development project. Principles support and protect the vision of the architecture; that which will allow it to be stable, flexible and/or scaleable. Without them (and this is particularly true in larger teams, new teams and teams with high turnover), it is easy to make day to day decisions that will weaken the application, introduce bugs or won't scale. Design principles are guardian angels for software. Discuss them, review them often and go back to them when alternate implementations are being discussed.

While I have long relied on design principles for application and database development, the rise of Ajax and Javascript libraries places a bigger emphasis on design principles for the front-end. Conflicts in Javascript libraries and styles are more frequent with Javascript and scriplet-friendly Web 2 applications. And with Ajax bridging the gap between front-end and back-end technologies (and in the process bridging the gap between front-end and back-end developers), establishing design principles for the front-end is a worthwhile exercise for a development team as a whole.

Choosing an AJAX library

The ubiquitous use of Ajax on Web 2.0 sites such as Google, Yahoo and Facebook has propelled Ajax into the spotlight. And for good reasons; used wisely, AJAX often improves the user experience by making it easier to find relevant and contextual information without leaving the page. The problem is that as of today, implementing AJAX is a mixed bag. Some features are trivial to implement (eg: auto-complete), others more complex and time consuming. It helps having a good AJAX library (or two) as a starting point. Just like designers and developers often use higher level tools to speed up development, AJAX libraries are a speedy alternative to coding AJAX by hand. The choice of a library hinges on a number of factors such as the template technology used (JSP, velocity templates, etc...), the type and number of AJAX functions and the degree of customization required. So unfortunately one size does not fit all. Also note that this is a relatively new market still. As the market mature so will the tools.

Today, one of the most useful AJAX tags library I have found for JSP development is Ajax Tags which is available as a Source Forge project.

Google and YouTube Scalability

Google's recent conference on scalability held in Seattle in June 2007 offered an interesting insight into how two of the world's largest properties have and continue to scale their systems.

Google scalability presentation

YouTube scalability presentation

Scalability Principles

It often comes as a surprise to business professionals but building scalable systems is a deliberate, purposeful and complex undertaking. Architected correctly, following certain design principles, a system will scale (handle a high level of concurrency and throughput), albeit with some effort. On the other hand, systems which were not designed to scale often end up being expensive and unreliable when stressed. So unless you are building a small departmental application, it's often best to sow the seed of scalability right into the core technologies, the base architecture of your application.

So how do you scale?

1. Identify a few design principles and guidelines before you start architecting. Google made the assumption that no single piece of the application could be deemed 100% reliable. What you think is bullet-proof will eventually fail. Knowing this forces the architect to handle failures gracefully and build fall-back systems. This principles applies to software, hardware and network components.

2. Avoid single points of failures. Application serves are load-balanced. Load balancers and switches are paired. Databases are setup using replication/pooling and/or partioning. Should any given machine fail, the traffic is automatically rerouted to other components.

3. Build your system as a set of discreet components. Each components hides its complexity (and error handling!) to the rest of the system. A component can run on all or some machines. For example, background jobs can be handled by a thread pool server(s) instead of bogging down an application server used to serve user requests. Please note that using property and descriptor files, a single application can be instantiated in a number of ways declaratively so the developer does not have to build several applications.

4. Automate your unit and load testing.

5. Build introspection capabilities into your production system. The worst thing about a bug is not being able to understand it. Threshold based introspection capabilities must be built into systems so a developer can understand what is happening, who is using the system, what resources are being used, etc...