Tag Archives: REST

Why REST Frameworks?

Many people are simply slapping an interface on to every web application they build and don’t give it much thought. Tim is right in asking why try to coerce your website into a REST paradigm since the browser is a bad (or rather not a general) REST client. We shouldn’t.

However, this doesn’t dispose of the need for a truly REST framework (that is one which uses real hypertext for is representations).

Lets say I have a compelling web application for which I publish a separate API. Assuming that I have a business model associated with that application, then someone is going to take my API and use it in some way I hadn’t anticipated. Now I’ve added value to my application without doing any work. That ability has a multiplicative effect as there are no limits to how many people can use my API. That applies to any API but REST is more likely to succeed since it is intended to promote long term (on the scale of decades) serendipitous evolution.

So there is value in publishing APIs for my applications (independent of the user interface), the potential for added value in the future.

But I think that there is a far greater benefit for our own development processes. Two principals can be maximized using REST APIs: Loose Coupling and High Cohesion.

What I am suggesting is that you should architect your systems so that you produce one (internal) application for each concern in your system. That is to say, if I was building an on-line storefront, I’d create three REST applications: one application to handle user management, one to handle product Management and one to handle order fulfillment. These all would be completely internal to my business.

I would then create a publicly accessible application which would consume the other three applications. This would be a User Interface so it would be designed around User Centered Design principals.

So what are the advantages to the approach? Here are but a few.

  • Scalability — Let’s say that the sale of widgets has gone through the roof and I’m seeing 10,000 people every hour come to my site. I’m going to have to look at load balancing. I now need to distribute my Web UI onto 4 machines, two of which I’ve put in Ottawa, and two are here in Vancouver. What else do I need to change? Well it turns out that once you’ve logged in (session mgmnt is handled by the Web UI), all you need is the User’s profile information. Since that is seldom changed in a session, the Web UI will be able to use its cached version; so long as the user management can respond with 304 NOT MODIFIED efficiently, one server can handle its load handily. Far more people browse than order, lets say 1 in 10 people place an order. That means that we’re expecting about 1000 orders per hour which our order application can handle nicely so we won’t scale it for now. But we are going to have to do something about the product information, that needs to scale with the Web UI… or does it? From the point of view of the Web UI, the product info is static and most of it changes infrequently. I don’t need to put up multiple instances of my product management app, I simply need a good web cache at each of my data centers to sit between the web UI and the product management system.
  • Maintainability — With four independent systems making up or storefront, I can now push changes to any one without affecting the others. This has some great operational benefits as any change to a system affects the whole system so when you’re only changing small systems that reduces the risk. Its also more testable. It becomes very easy to write mock applications to consume the resources of any of your internal systems or to produce mock APIs to test your web interface (this makes it very easy to write resilient selenium tests since the data won’t change).
  • Low(er) Risk Evolution — Next week the VP of Business Development is going to announce that you have a strategic partnership with Acme DooDads. They’re going to sell your widgets. Now we need to expose our product and order systems to them. We also want to make sure that only they have access to these services so we’re not going to put them outside the firewall. Instead we’re going to write a new public application that will interact with our product and order systems. Since ACME isn’t as modern as we are, we have to build a SOAP interface for them. Since they’re sending all the shipping particulars in each order request, we’ll create a new “AcmeShippingDestination” application to hold the representations that would normally be in our User management application. Here’s the value: We’ve added a significant number of features without editing any of the existing code. It also applies internally; I may need to create an application that consumes one or more other REST applications to expose other REST resources to internal applications adding some value along the way.
  • Fully Aware Deputies — Maybe its because I spent 7 years developing online banking applications, but I always expect the worst. If you look at your average open source CMS (like Word Press or Radiant) you’ll notice that one application handles both the distribution and creation of your content. This is an instance of “The Confused Deputy Problem”http://en.wikipedia.org/wiki/Confused_deputy_problem. The reason is that all the database access is through one user who generally has universal read/write privileges and any attacker who can exploit flaws in the system (and they always exist) has full reign over the database. A better architectural approach is a publicly accessible application that has only the permissions it needs and a private application that has full access. With REST, I would argue that our “Read Only” product product resources that are used by the Web UI are different than the “Read/Write” product resources that our business users are going to interact with. That is if we have separate applications for both the read only and read/write resources, I avoid the confused deputy and I can use normal network access controls to keep everything safe and secure. This is especially true in companies where information is classified in security terms as “Public”, “Private”, “Confidential”, etc. There you usually have strict controls on who can see the different types of information. Further any single entity may have information that can be disclosed at each security level; it would be most appropriate to have different resources for each security level served from each from their own application (again so that network level access controls can be used).

Some of these things are not unique to REST, to be sure. It also puts a little bit of strain on certain parts of our application; because we don’t know what is at the end of a URI, or if I even have permission to access it, we have to be a lot more flexible in what we expect. Actually flexibility is one of the hardest things here. Think of a web browser (which is a model for how we develop applications that consume REST resources); all it knows is the starting URI you provide, HTTP and how to render a handful of media types. This is all that your REST client application may know as well.

Where are the REST frameworks?

I like REST. I’ll admit that I didn’t understand all of Dr. Roy Fielding’s Dissertation (he is, after all writing to experts in the field of API design) but I think that I’ve managed to understand enough of it that Rails’ implementation of REST leaves a lot to be desired. Rails is not alone, many services are releasing APIs that they are calling REST. Dr. Fielding recently addressed this in a post on his site. Like his thesis, he is talking to API designers, not to Joe and Jane Developer who needs to get the latest web site off the ground. Perhaps more practically, Subbu Allamaraju gave the issue of REST and hyperlinking a thorough exfoliation over at InfoQ.

If you read Subbu’s article and compare it to the standard Rails 2.x web application you will quickly see that difference. In a rails application, if you GET a JSON representation of our quintessential blog all you are going to be told is that the owner_id is 1232; your code has to know that Owners are Users and that they can be retrieved from http://api.example.com/users/{user_id}. This is like putting an <img img_id=123> in your html and expect the browser to know to retrieve the file from http://www.example/resources/images/123.

There is the comparison that most people who think they are writing REST APIs are forgetting; HTTP. If you are writing a REST API then you are writing something VERY much like HTTP. None of us are likely to need to create a REST API. I work for a financial institution and we might have a need to create a REST API like HTTP that is geared towards exchanging financial transactions between internal systems, our partners and our clients, but I suspect that would be unnecessary.

No most of us have no need for creating a new REST API. What we should be talking about is publishing RESTful media types. If I’m going to build the archetypical Blog application, then I need media types for a Blog, a Post, a Comment, a Tag, an Author and what ever else.

A Note: I’m making a distinction that I don’t think Dr. Fielding would make. Dr. Fielding makes it clear that the linking mechanism is central to a REST API, and since HTTP leaves much of the linking mechanism (but certainly not all) up to the user agent, a true REST API requires a media types. However, the vast majority of the work that we do is over HTTP so we have no need, in general, to worry about that half of a REST API. I think it greatly clarifies things if we simply talk about creating REST Media Types rather than APIs. I just noticed that I’m not the first one to realize this (after Dr. Fielding of course).

What about the Frameworks

I’ve been trying to find a Ruby framework that can handle REST Media Types right out of the gate. Waves, which its resource oriented architecture seems like a good place to start. They are right at the start of putting together what they call foundations for REST services, I have high hopes for what we’ll see from that work.

REST ≠ MVC

One of the biggest problems is that people are trying to bolt REST onto MVC frameworks. The problem is it is probably the wrong programming model to use. The controller for a REST service is absolutely trivial; you can only call a maximum of 4 methods on any given resource, and they are always handled the same, so the controller is completely anemic and should fall into the framework itself. Views are also the wrong paradigm since they only deal with out going representations; you really need some mechanism for specifying the transformation of a resources into a representation (that is its media type), and back. All that you really have left are the resources (assuming they’re like models) and the transformations.

REST and Your Web Application

I’m probably going to annoy a lot of people with this. A web browser (as they are currently implemented) is NOT a RESTful client. You can use XHR object supplied by a browser’s JavaScript engine as a REST client, but the browser itself interacting with plain old HTML cannot act RESTfully. In part, its because your browser won’t make PUT or DELETE requests (you have to fake it with extra parameters on a POST). Mostly its a matter of usability.

For a non-trivial use case, you are going to want to something wizard like. That means you are going to have to create a number of intermediate, transient representations to maintain that state. Its an affectation in this case. The transient representations serve no purpose but to maintain this fiction that its REST.

I honestly think that you should really simply leave the your REST API as an API and build a separate application to act as your User Interface. You’ll make it easier on yourself in the end

REST and MVC

Like I said earlier, REST is not MVC. But that doesn’t mean that its not related to MVC. MVC is very well suited to building a User Interface. As I said earlier, you should build a separate application for your User Interface. Well, there is no reason why you can’t use a Resource as the Model in your MVC application. Rails already does this; ActiveResource allows you to use a REST Resource instead of a database table for your model.

This is where things get really interesting.

If you are already using resource representations for the models in your MVC user interface, why limit yourself to a single domain? There is no reason why you couldn’t use multiple domains. This allows for what I call DMVC: Distributed Model View Controller. By distributing your models over a number of applications (via REST media types) you can get get some great long term efficiencies by the serendipitous, and unplanned, reuse of REST APIs.

For example, if you have a non-trivial content management system, you are going to want to move your content changes from the author to an editor to perhaps a legal reviewer and maybe finally to a marketing manager before those changes are “live”. At the same time, you may have defect tracking system where defects move from reported, through any number of stages to completed. Traditionally, you would include something like acts_as_statemachine in each of your rails apps to get this functionality.

But if you apply the REST architectural style, you can create 3 independent applications: Content Management, Defect Management, Workflow. The Workflow applciation would have something like a State resource which would be associated with some other resource without knowing anything about that resource. If the business logic was sufficiently complex (for instance, a change in the State resource would change the associated resource) you could create another application that combined a State resource with a (let’s say) Defect resource to expose StatefulDefect resources.

Also, because you have decoupled your the workflow from both your CMS and your defect tracking, you can reuse both to create a project management application where the content is not work flowed but managed wiki style and defects are not workflowed rather raised and closed since you have hourly builds to the project server (or what ever).

We already know how to optimize and scale web servers, and the same can be done here. Or, if this is a purely internal set of services (or even largely) you could create a more efficient connection mechanisms/schemes than HTTP (I’m not sure what those would be). Or you might try more efficient media types (for example using JSON or even a binary format).

REST, Associations & ORM

One of the more valuable parts of Rails is the ActiveRecord ORM. It allows you to say that a Blog belongs_to a User and a User has_many blogs. Associations between these objects are by ID as they refer to other records in the database.

But REST blows that apart. Your Blog and your User are no longer stored in the same database. We refer to every unique resource by a URL so it may live somewhere totally unaccessible to us. If I have blogs and you have users, I can say that my blog belongs to a user, but I cannot expect your users will know anything about my blogs. Now perhaps the representation of the user that you give me includes a list of blogs that the user owns, in which case I can PUT back a representation that includes the URI of my Blog, but my code cannot rely on that information being known.

You also get some interesting behaviours.

All you have is a URI. You have NO idea what that resource is. As a result, its conceivable that I might try to create a blog that is “owned” by an image. Or the URI point to a resource that is a legitimate owner of the blog but its media types are not those that the blog application understands. For the most part, that’s OK because the blog application is really just an API, its not our User Interface. The User Interface application will need to have coping mechanisms for this but I’m not sure that’s a problem. If all you’re producing is an API, then its not even your problem.

More importantly, you get all sorts of strangely compelling polymorphism. Its liberating not to know what you’re associated with since it guarantees that you’ll become associated with all sorts of unexpected things.