Yaogang Lian

Muffin Series 2: The Evolution of Web Development

In this post I would like to take a step back and talk about web development in general. I believe understanding the many facets of web development in the past and present can help us build a better Web for the future.

The Timeline

Since its inception in the early 1990s, the Web has been constantly reinventing itself. As a result, there is a wide variety of websites coexisting in today’s web, ranging from static web pages, to dynamic websites, to highly interactive webapps.

The wide spectrum of websites is only trumped by an even greater number of web development tools. Below I sketch an overly simplistic timeline for the history of web development.

  • 1990 - 1995: Static web pages were built with simple text editors.
  • 1995 - 2005: Static web pages were built with WYSIWYG programs such as FrontPage and DreamWeaver. Dynamic websites were created with PHP, Java Servlet, ASP.NET, etc.
  • 2005 - 2010: Developing dynamic websites became much simpler with the advent of server-side web frameworks such as Rails and Django.
  • 2010 - present: Highly interactive webapps were built with client-side web frameworks such as Backbone.js, Ember.js and AngularJS.

Of course the actual history of web development was a lot messier —- most of the time many technologies coexisted and served different purposes. In today’s Web we are seeing static web pages reminiscent of those from early 1990s, as well as highly interactive webapps that behave like native apps.

An important omission from the above is the CMS platforms such as WordPress. It is a bit challenging to place them into the timeline because of their unique features and workflows. However, there are valuable lessons we can learn from WordPress’s enormous popularity. According to W3Tech, WordPress is used by over 19% of the top 10 million websites as of August 2013. I will come back to this in a bit.

Client-side vs. Server-side

There has been a paradigm shift in web development in the last few years. More and more webapps are now built with client-side web frameworks such as Backbone.js, Ember.js and AngularJS. This new generation of webapps boasts an impressive list of features —- highly interactive, low bandwidth usage, offline mode, simple backends, easy to scale and much more. How can client-side webapps achieve all these amazing features?

In a client-side webapp, the application UI is rendered entirely on the client side using JavaScript. The server provides a simple RESTful web service that can talk to any type of clients, such as an iOS app, an Android app, or in this case a JavaScript webapp. JSON data, instead of HTML fragments, are sent over the wire. Since all the view rendering and view state tracking are done on the client side, this design significantly reduces the complexity on the server side. Another benefit of this design is that all the client code can be packaged into static files that can be served on CDN, cached in the browser, or even bundled as offline webapps.

Let’s take a closer look at several key advantages of this approach.

  1. Backend as a service

    Building the backend as a web service can cut your server-side work substantially. First, since all the view rendering and view state tracking are done on the client side, the server only needs to set up data models, handle database persistence, and implement RESTful APIs for some data models. Creating APIs for simple data models can even be automated. Second, a backend as a web service can work with any type of client —- an iOS app, an Android app, or a webapp —- since they all consume the same API. Last, you can use a subset of the API as your public API for third-party integration.

  2. Snappy UI

    Since the application UI is rendered entirely on the client side, there is no page refresh at all when the user navigates the app. This makes the app feel much snappier, especially on slow Internet connections. In addition, since the client code is packaged into static files served on CDN, the app can load very quickly and cache the files for future sessions, which can be a great time saver on mobile devices. Finally, JSON data on the wire means less network traffic thus requests can return more quickly, which further improves the user experience.

  3. Low operating costs

    Serving client code as compressed files on CDN can significantly reduce the amount of traffic hitting the application server thus saving costs. Additionally, sending JSON data over the wire saves both CPU time and bandwidth usage on the application server, compared to sending full HTML pages or HTML snippets.

  4. Scalable

    Since the client code is served as static files on CDN, the frontend can scale incredibly well. The backend can also scale well if a decent server stack is chosen. We will cover Muffin’s preferred backend stacks in a later post.

  5. Less code to write

    When you develop an ajax-heavy webapp in a server-side web framework, you often end up writing view rendering code in both the server language and JavaScript. The initial page is rendered on the server side, but subsequent view updates are usually done by dynamically loading HTML snippets, which requires writing code for both sides. Taking the client-side webapp approach means you only need to write the view rendering code in one place. The server code, in turn, is also much easier to write. Less code to write means less code to maintain, and fewer bugs to hunt down.

Static, Dynamic and Interactive

Before moving onto other web development trends, we need to clarify some confusing terminology commonly used in web development.

A “static” website usually refers to a collection of web pages whose content is the same for all visitors. A “dynamic” website, on the other hand, usually displays different content for different users and contexts. However, sometimes “static” and “dynamic” are used to refer to the page creation process instead of page content. For example, sites built with WordPress usually shows the same content to all visitors, but they are often considered as “dynamic” sites because their content is generated at runtime.

Ironically, if the WordPress sites were built with a static site generator such as Jekyll, they would be considered as “static” sites. However, the end user won’t notice any difference since the pages are essentially the same. The only real difference between a WordPress site and a Jekyll site is when the pages are generated —- WordPress generates pages at runtime, while Jekyll generates pages during the build process.

I think there is a deep insight hidden behind all this confusion —- page content is separate from page creation, and page creation can happen at different points in time. Thus we can make web development more efficient by changing how and when pages are generated. The evolution of web development largely followed this course: page generation was moved from the editing phase, to the build phase, to the runtime on the server side, and lately to the runtime on the client side. Each change marked a new generation of web development tools with more flexibility and higher efficiency.

There is also a lot of confusion around “dynamic” and “interactive”. Many people use these two words interchangeably when referring to websites. In my opinion the two are distinctly different. As we discussed, “dynamic” refers to the page content or the page creation process, while “interactive” means the web page can respond to the user’s input. An interactive app doesn’t have to be dynamic —- for example, an image editing app can run entirely on the client side while interacting with the user constantly.

The Convergence of Web Development

Over the years web developers have self-organized into subgroups based on the tools they specialize in. Some build static content sites with WordPress. Some build database-driven Rails apps. Some use Backbone to create client-side webapps. Developers in different groups are often disconnected from each other, although their final products are made of essentially the same ingredients —- HTML, CSS and JavaScript.

But I think this disconnection is coming to an end. Just as the Web itself has embraced standardization in recent years, disparate web development practices are starting to converge. (I think this trend will become more visible soon.) This convergence is largely driven by the Web itself —- as the Web reinvents itself as an application platform instead of the document model from early 1990s, differences between various web development practices are becoming exceedingly small. For example, a site built with WordPress often uses large JavaScript libraries like jQuery to provide interactive features such as an image gallery, a Rails app may have half of its code written in JavaScript, and a highly interactive webapp might be merely a few static files served on CDN.

I think this convergence can be facilitated by a new generation of client-side web development tools, including Backbone.js, CoffeeScript, and the subject of this series —- Muffin. As more interactive features are added to WordPress sites and Rails sites, writing modular JavaScript is no longer just for client-side webapps. The gradual adoption of modular JavaScript in these sites will in turn speed up the convergence. I think this presents a unique opportunity for the tool developers —- a unified process can be utilized to build a large variety of websites, previously done with many different web development tools, and web development can achieve a whole new level of efficiency as a result.

Building a Better Web

I believe understanding the big picture of web development is important for building better tools for a better Web. Knowing where the Web is heading helps us abandon complicated and outdated practices, but instead focus on simple and powerful ones for the future. When we remove the clutter and concentrate on the gems, new insights can be formed and new ideas can be hatched. That’s what Muffin aims to achieve.

There are a few other trends I didn’t cover in this post, most notably, using JavaScript as a full-stack language and the real-time Web. I will cover those topics in later posts.

Yaogang Lian

An iOS, Mac and web developer. Focusing on building productivity and educational apps.