John Sheehan : Blog

First Impressions of DynamoDB

I wanted to try out Amazon’s new DynamoDB service so yesterday while watching 14 hours of football and football games I wrote a script to track my Twitter followers and unfollowers. Here’s my first impressions.

Old habits die hard
I’ve played with various NoSQL stuff like Redis and Mongo in passing, but never really built anything beyond a hello world. I have a ton of experience with relational databases and it’s very hard for me to shift my mindset and break old habits. For instance, my data model for this script was ridiculously simple: a user ID in a table. I thought I would just use the scan functionality to return the whole table but table scans are not recommended. Instead I just used a common hash key and put the user ID into the range key. Queries over hash keys are fast and easy so it worked out fine.

boto is very nice
I wrote this script in Python which has a great third-party wrapper for Amazon Web Services called boto. It’s very complete and basic support for DynamoDB was added just a few days after it launched. It still has some feature gaps (like scan) but overall I was able to get what I needed out of it.

I still like Python a lot
Python syntax is pretty awesome, except for some of the self passing. I do miss LINQ from C# though. That would have made some of the list checking a lot easier. Between Google and Stack Overflow though I was able to find good short snippets to do what I needed to do when I got stuck. And requests for Python is my favorite HTTP client I’ve used on any platform.

SendGrid rocks
Dead simple REST API to send email. Since I plan on running this as a Heroku worker process I didn’t want to mess with SMTP. I have it set up to email the results when it’s done processing.

You can find the script on GitHub.

My .NET Open Source Project Management Nightmare

I’m still doing occasional maintenance on RestSharp because it’s my baby, I want to see it continue to thrive (preferably under the community’s tutelage) and the company I work for has a dependency on it.

RestSharp depends on JSON.NET and uses NuGet for distribution, listing JSON.NET as a dependency that gets automatically installed when you install RestSharp. I don’t specify a specific version of JSON.NET to depend on because they’re good about not breaking their API and it works from version to version just fine. Within the RestSharp projects themselves the reference to JSON.NET is managed via NuGet.

Or I should say, it did work. At some point recently I started getting innundated with support requests for both RestSharp and twilio-csharp that JSON.NET was throwing exceptions. Turns out when you add a reference via NuGet to a strongly-named assembly (a recent change for JSON.NET I can’t track down anywhere), your reference is tied to that specific version so if JSON.NET is updated to a newer version you start getting runtime errors.

As long as you install RestSharp when it was built against the current version of JSON.NET, no other projects depend on a different version of JSON.NET and never update your JSON.NET version out-of-band with RestSharp you won’t see this error. I could set a hard version dependency in my NuGet package, however JSON.NET is so popular (thanks Microsoft for including first-class JSON support in .NET! Not.) that’s almost assuredly going to cause issues if you have other packages installed that depend on it.

To keep the project working, every time JSON.NET releases an update (thankfully it’s not too often) I have to rebuild and distribute an updated RestSharp, which twilio-csharp depends on so that has to get updated and distributed too. twilio-csharp also depends on a JWT library I wrote that also uses JSON.NET so I have to fix and distribute that too before I can fix twilio-csharp. At any point JSON.NET can release an update that people manually update to and break everything that depends on it. It’s like Chaos Monkey for the project. At some unknown point any number of users could break, resulting in more support emails for me and another set of changes just to appease the dependency gods.

This is why I’m not able to work on new features for RestSharp. I have to spend all of the very little free time I have for the project dealing with this crap.

Sour Grapes

(These are my observations and opinions based on interactions with ASP.NET MVPs and ASP Insiders. I can’t speak for other MVP or Insiders programs.)

In response to Rob Eisenberg’s post about his travails with the Microsoft MVP program, I kicked up a dust storm on Twitter tonight with this series of tweets:

This take down of the MS MVP program by @EisenbergEffect should be required reading for anyone looking to achieve that award.

Don’t make the same mistake I did. Trying to become an MVP is a waste of time. Call it sour grapes because I didn’t get in if you want.

The same politics Rob talks about is how I became an ASP Insider, which is a disservice to real ASP.NET contributors.

Re-initiating my attempts to get out of ASP Insiders. If you’re a member, consider doing the same. Participation is an endorsement.

If you don’t want to support a corrupt system, vote with your feet.

I lucked out that in attempting to become an MVP I found something better. Not everyone else will be so lucky.

As predicted, this was interpreted as sour grapes, despite well documented proof that it turned out better for me not getting in the program. I’m genuinely not bitter that I didn’t get in. Doubt it if you want, I don’t really care.

What I am bitter about is that the MVP program disrespects the very developers it sets out to honor. There are a few well-documented problems the program has, most notably around selection criteria. I don’t want to rehash those, go read Rob’s post. But I do want to talk about some other aspects of the program that are damaging to the overall .NET community.

Club mentality segments the community

Before I was an ASP Insider (which I’ll cover below) I would go to user groups and conferences and talk to other developers about what they thought of the latest upcoming ASP.NET releases. I’d make my predictions or state opinions and they’d casually nod along. What I didn’t yet realize is that the MVP and Insiders of the group already knew the answers and weren’t able to discuss things openly because they were bound by an NDA. While this is a standard practice (the company I work for also gives early access under NDA to select customers), it turns out that this can create a rift between those who are in the know and those who are not. After I was granted access to the privileged group, I found myself stuck during conversations, not able to fully engage with those not under NDA. It was very uncomfortable to me to be hampered talking to other community members who are just looking to learn and network and become better developers. Eventually I couldn’t remember what was NDA and what wasn’t, who I could and couldn’t talk to openly, etc. It was exhausting to keep track of so I stopped trying to remember stuff. I broke my NDA a lot. Since I’m trying to give it back anyway, they can happily take it if this bothers them.

Wrong incentives lead to bad behavior

The MVP program also incentivizes bad behavior. In the yearly reviews, quantity is given priority over quality. Or at least it looks that way since there’s no clear acceptance guidelines to know for sure. This leads to a high volume of low quality content to boost numbers that look impressive at a glance. Or you’ll see one guy give the same lifeless user group talk over and over again to juice up the total instead of giving fewer, better talks. If the plural of anecdote is data I have a lot of data from people I’ve come across who have specifically mentioned reusing content in this manor specifically for their MVPs reviews, and not for, you know, helping better the community. The incentives are all wrong.

So the MVP program has outlived it’s usefulness and now does more harm than good but then we have the ASP Insiders program that just complicates matters further. Not that ASP Insiders is a bad program in isolation, it’s actually quite good. You get really direct access to product team, it’s a small, tight-knit group and it’s a lifetime appointment unless you forget to re-sign the NDA every few years. You also get voted in by other members of the group, so there’s no black box for acceptance. Just make 10 or so friends already in the group, do a little ASP.NET community work and don’t make enemies with Microsoft and you’re set. If you can deal with the few people in the group that would rather post the problems they encounter privately instead of on Stack Overflow where they would be embarrassed about publicly admitting there’s something about ASP.NET they don’t know, the mailing list is decent and you get to go to the MVP Summit (on your dime) and party with a bunch of really cool people which is insanely fun.

The problem with the ASP Insiders group is that most people trying to become ASP.NET MVPs believe what they’re after is what ASP Insiders offers. That’s how it’s pitched. It’s not how it works. And that’s unfair to people trying to become MVPs. You either work hard or game the system, get your MVP and then find out that you could have just become an Insider with all the same perks (and more) without as much hassle. I hope you read this and find it out before you go through all that.

I’ve asked to be removed from the Insiders. I got in through a back door (longer story) and don’t really do ASP.NET any more and it’s unfair to the people who really want to be there to have a non-participating zero (as far as the group goes) taking up space. Yeah, there’s no limit to how big the group can get (making it effectively easier to get in over time since the number of votes needed to get in is currently fixed at 10) but it’s still not fair that I’m in there and people who work on this stuff every day and would benefit from access aren’t. I’ll miss hanging out with friends at the Summit but I’ll find ways to meet up with them.

So why did I write all this? Mostly because I think it can be instructive for those working to build developer communities. When you institute classes, no matter how well-intentioned, it can have unintended negative side effects. You never, ever want to have a member of your community say this:

Looking over the last three years, I’ve definitely been a less happy, more frustrated developer. I’m sure it’s linked to the sort of fake “value” being a Silverlight MVP gave me. Basically, they give you a couple of cheep gifts, then they pretend to listen to your feedback, while not actually doing anything. Year after year of that and you get really unhappy. It’s demoralizing. You start to realize that you really are a commodity.

Is that how you want to treat people?

(Comments disabled for this post. Tweet at me or blog.)