Re: Why Today Web Development Sucks

Stumbled upon a post by Harry Brundage about how web development sucks today. His main argument stems from the fact that Single Page Applications (SPA) and web stacks tend to suffer from violations of the DRY principle. Harry makes some good points, and proposes a single language framework that would bring something like Ruby On Rails to the JavaScript domain. While this may be good, I think the problem is in the way applications are conceived of and architected. A properly architected solution can avoid most of the DRY violations Harry speaks of, and will help you to do web development that doesn't suck, today.


Web Development is Hard
The crux of the issue is that developing a web application is hard. Developing a SPA is harder. At first glance, building a web application seems simple enough: write some HTML and CSS, put in a few JavaScript DOM event hooks, create a few web services in your language of choice, expose those web services via HTTP, and voila, you have a web application.


However, that simple description isn't simple at all, is it? I mean, we have a markup language and a styling language to learn, a framework (DOM) to learn and understand, an interpreted execution language to learn and master, a server-side language to learn and master, and a stateless protocol to allow data transport between the client and server. Then we've got security, persistence, browser differences, performance, download times, SEO, caching, maintainability and scalability concerns to add on top of the prerequisite knowledge stated previously. Then we have software architecture to learn. That's a lot of knowledge for one person to learn and use well.


Then we have the users and the client/customer to worry about. Oh, and application GUI design, because nobody wants a web application to look like some old Windows Forms application brought to the web. Animation, effects, transitions – there is more than enough to learn here for someone to specialize in their entire career and not be out of work. That's often what happens. However, acceptance and understanding of a few key concepts can help you get a handle on all of these things to become a good, general web application developer.


Learn To Think in Layers of Responsibility
In order to make sense out of this mess, I find it easier to lay out an application in layers of responsibility. These layer patterns are often recursive, and repeated within themselves. In its simplest terms, every layer should abide by the Unix Principles that a layer should "do one thing and do it well" and that a layer should be "written to work with other layers." I find this to be true for nearly all applications, but it especially applies to web applications and even more to SPAs.


Nearly every web application has at least three basic layers. The first is a persistence layer. This layer is where all your data is stored between requests from the client to the server. It stores the state of your entire application. Second, is the service layer. This layer provides communication and data formatting for requests from the client to the persistence layer. Finally, the only part of the application that your users will see, comes the client layer, which transforms the data into a human usable format using HTML, CSS, and JavaScript.


Notice that I did not use the traditional Model, View, Controller (MVC) pattern to describe the persistence layer as the Model, the service layer as the Controller, and the client as the View. Modern web applications, and SPAs in particular, have moved beyond this basic concept.


In reality, each layer will probably have its own MVC pattern, in which the persistence layer's model is the physical data structure, its controller the mechanism for reading and writing to that structure (typically a database engine with SQL, though XML, file-system, and NoSQL map-reduce schemes are popular as well), and its view the format of the data return (often handled by your language of choice's database connection library).


The service layer will likely have these features as well, with sections of code containing a business or domain model that is mapped to the persistence view, a controller for translating client models to and from the business model for manipulation, processing, and storage to the persistence layer, and a view section that formats data coming into and out of the client layer.


Likewise, a client will have an in-memory (typically, disregarding cookies and local storage) model for communicating with the business layer, a controller for processing communications from the client's view and from the service layer to the client, and a view (HTML, CSS, DOM, JavaScript GUI toolkit of choice) to present the data to the client.


The key thing is that each layer presents a view to the other layer, which processes it, and in turn presents the view to the next layer, and so on. The layers communicate with each other in ways that each layer can understand. The persistence layer has no knowledge of the client layer. The service layer need not have knowledge of how the client layer will present the data to the user, so long as the service layer provides a view that the client can interpret. The client layer need not have any explicit knowledge of the persistence layer, other than that the data must be retrieved and stored. If you learn to think in layers like this, then you can begin to separate the features and responsibilities of the layers involved, thus simplifying the responsibilities of each layer.


DRY Web Apps
The key thing to avoid repeating yourself, and thus save development headaches down the road when maintaining and upgrading an application, is to separate the responsibilities of the business logic between the layers. A good way I've found to do that is to model the whole stack as a series of Objects as if the entire web stack were to be coded as a single application. Treat each layer as a Facade to its inner workings. Then you only have three interfaces to work with! A persistence interface, a service interface, and a client interface. Strongly type the parameters and returns from methods on each layer. Treat communication objects and returns from each layer as Data Transfer Objects (DTO). This separates the models on each layer, preventing interference.


Validate only when and to the degree necessary. If you want to alert users that they aren't typing in their MasterCard in the correct format, you can validate the format using regular expressions on the client side. Yes, you'll have to duplicate the regex on the server side, but could you possibly persist the regular expression somewhere in your application for use by both layers? That would prevent you from having to maintain it in two layers. Also, simply validating the format of data isn't truly validating the credit card information is it? You have to combine the Name and Billing Address as well. If you are not storing the card, there isn't any need to strip out things for SQL injection is there? And since all of this business logic has to do with validation, perhaps a single validation Facade for all your exposed services would be a good place to store all that logic?


By using good object oriented design principles while modeling your application, you can avoid repeating yourself everywhere, and keep the business necessary to the appropriate layer in that layer, and keep the layer transparent to the other layers. The other advantage of modeling your application in a unified model is that it keeps you out of the weeds of the particular programming languages involved in the implementation of the model. Which brings me to my next point regarding transferring data between layers.


Be Language Agnostic
When you format your views from each layer to the next, it is important to keep the format of the data to be transferred language agnostic. Don't use binary data formats that only communicate with one client side language type to transfer data into and out of the service layer. If the browsers implement Objective-C as a client-side runtime, you may want to be able to use that. JSON and XML work particularly well for transferring information because they are string based and have strict syntactical rules. HTML does not work so well for transferring information because it doesn't have strict syntactical rules. There should be NO HTML in your returns or accepted data formats. You never know what language of a public service API that developers will use to connect to your services, so keep the information in some form of a string-based implementation.


Be Consistent
Be consistent with your data format returns. If you choose to use Comma-Separated-Values(CSV) for one service, do it for all of them. If you can't, use JSON instead. If you need references to other nodes in your data return, use XML. The key thing is to use a consistent format. You don't want to write serializers for a multitude of different possible data formats on the client side or the server side. It will lead to repeated code and a lot of difficult corner cases.


Be General and Concise
Try to be general with your services and returns. Don't write a thousand services for every possible request. Write as few general services as you can to relate any data to any other related data and generally expose more information rather than less. The most expensive part of a web application is requesting information from another layer, so try to limit those up front with general services. If you run into trouble later on down the road or your returns begin to take too long and you've exhausted every other avenue of optimization, then create a new service.


If Language Is a Barrier...
Use JavaScript for your server-side and client-side language of choice. If you are building a web application, JavaScript is the one language that can be used at runtime on the client-side and on the server-side. Node.js is a great place to start with server-side JavaScript. There are a ton of tutorials on getting started. CommonJS is an organization working to create a standardized API for writing JavaScript outside the browser. I suggest you check them out if you don't have time/feel like learning one language for the client side and one for the server side.


Writing web applications may suck today, but it doesn't have to tomorrow. If you use good object oriented design patterns, separate your application design into layers of responsibility, and manage your layers as if they were objects on the same system communicating with each other rather than separate systems over the wire, you can make it suck less. You can avoid some of the language syntax issues as well if you use Node.js and JavaScript on the server side. All of this is easy enough to do with enough planning, and you'll have a much happier day playing around with your application. Happy Coding!