RestSharp

One of my goals for this year was to start an open-source project. I had an idea earlier this summer while working on Managed Assembly that I wanted a way to reverse shortened URLs. Some providers (like bit.ly) provide an API for this so that requests to reverse it don’t count in the stats they track and also to pass along additional information about the link. This type of library would be useful for blog engines, link sites, twitter clients, etc.

I set out to write a library that would let you plug in providers for URL shorteners to properly expand URLs. Most of them use REST APIs so I started looking for a .NET library for accessing REST APIs. It should be simple I thought. All this library needed to do was basic HTTP and JSON or XML deserialization and I’d be on my way churning out URL expanders. I ended up getting lazy and just doing a simple GET request against a shortened URL and grabbing the redirect header and using that. It works well enough for what I needed at the time.

Fast forward a few months. My fascination with REST lingered and I had a lot of projects using REST APIs using either API-specific libraries (like the incredible TweetSharp) or example libs provided by vendors which are usually pretty terrible. When I decided to give a talk at my local .NET user group on integrating REST APIs into your .NET applications I started creating a little library for my own use so I could focus the talk on the ideas and not so much the implementation. Turns out this little library was extremely useful. I had my OSS project; RestSharp was born.

First Attempt: There be faeries!

I wanted to make calling an API as simple as this:

var bitly = new BitlyApi(account, secretKey);
var response = bitly.Shorten("http://foo.com");
// response.Results["http://foo.com"].shortUrl == "http://bit.ly/abc";

I thought that everything you would need to know to generate the proper HTTP calls would be included in the class definitions (pseudocode):

class BitlyApi {
    [GET]
    shortenResponse Shorten(string longUrl) { ... }
}

class shortenResponse {
    int errorCode;
    string errorMessage;
    Dictionary<string, shortenResult> Results;
}

class shortenResult {
    string shortUrl;
    string hash;
}

That pretty well describes the API for that action. To make a long story short, I tried writing a RestBase class and then proxying the method calls through that, turning it into the right HTTP calls and deserializing the result into the correct type. I encountered a few problems that prevented this from working. First, I ran into some limitations of reflection that I wasn’t smart enough to get around. I tried getting my Func<T> on and using other stuff (like Castle’s DynamicProxy) to get around the issues but it was way over my head. Second, how do you describe a URL like this: /accounts/{accountId}/users/{userId}? I could add another attribute for URL format, but I wasn’t interested in a stack of attributes on every method. It wasn’t going to work the way I wanted. I had to be less clever.

Second Attempt: When in doubt, make it simpler.

I stuck with the base class idea but instead of trying to do the work with magical reflection faeries, I would pass a new RestRequest class from each method to an Execute<X>() method in the base class.

public shortenResponse shorten(string longUrl) {
    var request = new RestRequest {
        Verb = Verb.GET,
        Action = "shorten",
        ResponseType = ResponseType.Json
    }

    request.AddParameter("longUrl", longUrl);

    return Execute<shortenResponse>(request); // Execute is in RestBase
}

This supported a lot more scenarios. I added things like base URL, authentication handling, etc. to RestBase and it was working pretty well for most of the (relatively) simple APIs I was testing against.

So at this point I had an API that made it easy to write APIs which is what I wanted, but you couldn’t use it to make standalone requests which would also be useful for one-off requests. Goodbye RestBase and hello RestClient.

Third Attempt: No catchy subtitle.

I moved all my HTTP and deserializaition out of RestBase and into a new RestClient class. My bit.ly proxy was updated to include it’s own Execute method that would handle all the common items for every request:

private X Execute<X>(RestRequest request) {
    var client = new RestClient();
    client.BaseUrl = "http://api.bit.ly";
    client.Authenticator = new ParameterAuthenticator(account, secretKey);

    request.AddParameter("version", "2.0.1");
    return client.Execute<X>(request);
}

The nice thing about this structure is you could use RestClient completely on it’s own. At this point, I had a structure I was satisfied with. There was still a lot of work under the covers to actually do the real work of making the HTTP request and deserializing the result though.

HttpWebRequest is your friend.

Being lazy and not wanting to write my own HTTP library I started out trying to use the HttpClient from the WCF REST Toolkit to do all the dirty work underneath the covers. It has a lot of deserialization options (XLinq, System.Xml, JsonDataContract, etc.) and does an alright job handling the HTTP requests. But there are problems. My biggest issue is my disdain for attributes and if you know anything about the deserializers included in .NET they sure love their attributes and I didn’t want to muck up my proxy classes with all that ugly.

Another problem I ran into came while trying to implement some of the Amazon S3 API. S3 uses a fairly complicated authentication scheme and I couldn’t get the HttpClient to allow me to set the header necessary to authenticate properly (I’ve since been told this is possible, but I’m past the point where it matters anymore anyway). And the last problem is that the WCF REST Toolkit includes a lot of functionality for producing REST APIs as well as consuming them and I wanted my library to be as small as possible. I also didn’t want to be tied to a closed-source library on an unknown release schedule from a company with a propensity of cancelling side projects like this without notice.

I was really hesitant to write my own HTTP handling, but there’s really not that much to it. It didn’t take me long to get a HTTP class with support for the "big 6" REST verbs and I have total control over the requests. It may not be as robust as some other libraries out there like the HttpClient but it does what it needs to do for now.

“Do your HTTP GET and parse the result like a man!” – Miguel de Icaza

I plugged in JSON.NET to handle JSON deserializing (which it handles brilliantly) and started writing my own XML deserializer. I know what you’re thinking, but you’ll have to trust me that this was necessary to allow for some of the convenience features I wanted to build later on.

The RestSharp XML deserializer takes any .NET class, loops through its public properties, then goes looking for the associated XML element (using LINQ to XML) and converts the XML value into the property type. It handles properties of type List<T> the way would hope and you can nest properties as needed to match the XML schema. This ended up not being as much work as I thought it would because there’s only so many CLR types you can represent in XML.

I wanted to add some "smart" features like automatic name matching as well. For instance, when pulling the value out of the XML it will search for an element with the right name and if it doesn’t find anything, look for an attribute with a matching name and use that value if it exists. If it still doesn’t find a value it tries to find a matching element or attribute with a name that might be in a different format. For example if your property is named AccountId and the XML element is named account_id it will find it and use that value. Try doing THAT with XmlSerializer.

After working up a pretty decent XML deserializer, I went back and updated the JSON deserializer to basically do the same thing but utilizing the LINQ to JSON features in JSON.NET.

Putting it all together

With all of this in place, calls to REST APIs are as simple as:

var client = new RestClient();
client.Authenticator = new HttpBasicAuthenticator("user", "pass");

var request = new RestRequest();
request.BaseUrl = "http://twitter.com";
request.Action = "statuses/friends_timeline.xml";

This could easily be wrapped in an API wrapper class, which I will demonstrate in a future blog post.

Early and often

I have a laundry list of features I want to implement but I figured it was better to get this out there in front of people to start getting feedback. I’ve posted the code up at the GitHub project which you can take a look at and start playing around with. I would love to hear any ideas you have. I’m working on the docs and some usage examples and those will be posted over the next few days. Most updates will be posted on the RestSharp site and the @RestSharp Twitter account, so be sure to subscribe to those if you want to keep up to date on RestSharp as it grows up.