The other side of AppEngine

I really do love Google AppEngine. That didn’t come out so well in a recent post. I’ve had a bad taste in my mouth ever since then and now that I’ve eliminated the dandelion wine as a potential culprit, I’m forced to conclude that I need to balance my hillbilly feng shui. So in no particular order, here is what I like about AppEngine:

Granular pricing

I complained about the confusing pricing in the last post but since then, a funny thing happened. BookedIN got some traffic (thanks in no small part to the Chrome Web Store.) We were getting feedback and adding features and fixing bugs and hey, howdy, hey, wouldja take a look at what’s happened to our AppEngine budget!

And for all the bitching I did on the different ways your application is charged, this jump in cost has led to some much-delayed but necessary shifts in how we code.

As an example, we are now hitting our datastore read quota regularly. Which leads to the question: “Why so many reads?” Which leads to a number of other questions most of which can be boiled down to “Why am I treating the datastore like a SQL database?”

Similar discussions have led to us taking much greater advantage of AppEngine features we’ve all but ignored, including task queues and the channel API.

This doesn’t mean we’re on a refactoring binge, mind you. Our costs have jumped to the point where we want to waste thousands of dollars in developer hours to save ourselves a few extra bucks every month. But it has affected our coding practices going forward. When discussing new features, there are new questions: Can we denormalize this object? Do we *really* need to send this email synchronously? Can we send a message back to the user via an open channel? That sort of thing. All of these questions we should have been asking but didn’t because we “didn’t have time”. Now that cash dollars are being spent and now that we’ve seen how little time it takes to tweak our way of thinking, we are now working towards a better performing and more scalable application.

Out-of-the-box services

I looked briefly at AWS. It’s very different from AppEngine. From what I can tell, it seems you get a server. What you do with it is up to you. With AppEngine, I have an environment for caching, task queues, logging, mapping, channels, etc.

The price you pay for all these services is that if you ever decide to move away from AppEngine, you’re rewriting. For the most part, this ain’t no generic framework you can swap in your own implementation fer.

Multiple simultaneous versions

You can deploy up to ten versions of an application to the same account. For us, that makes actual deployments very smooth (in theory). We deploy on Saturday nights because statistically, that’s when the app is least used. But any time the week prior, we can deploy to a new version in production and just not make it active. That allows us to do smoke tests (if possible) and make last minute tweaks (or, more likely, add new features). The night of, it’s mostly a matter of switching the default version from the old one to the new one. I say “mostly” because sometimes, we have mappers to run to transform the data in order to handle some new feature. If you structure your model properly, this is an optional feature, but I like to do it anyway. Even with our massive increase in traffic, this still takes less than 15 minutes.

There’s also now a new experimental feature that lets you redirect a percentage of your traffic to a different version. This is designed for A/B testing but something tells me it won’t be used as such. In any case, I haven’t tried it.

Backup and Restore

I’m including this one because my original post said it wasn’t there. Since writing it, an experimental backup/restore feature has been added to the Datastore Admin console. I’ve tried it and it works though the lack of blob support means that it’s usefulness for us is almost non-existent. I suspect it’s not far away from being ready though. In the meantime, there are other solutions. One is DatastoreBackup which looks promising. Another is Google App Engine Backup And Restore, which looks very out of date. I mention it only to serve as an example for a long-standing PSA I have: Acronyms are not always the best name for a software project.

(NOTE: I *still* have to use Firefox to access the Datastore Admin console because it usually doesn’t appear in Chrome.)


Final note: I lied about this list being in no particular order. For all my initial confusion, I can’t understate how much I now love the granular pricing of AppEngine. If we want to pay less for our hosting, there are any number of things we can do to restructure things. And the nice thing is: almost every one of them will result in a better performing application. Before this, I’ve always had a fuzzy feeling of “yes, I know we could optimize but how much is that *really* going to save us?” Now, it’s right there in front of me. “We’re using too many channel API calls.” That’s because there’s an error in our logic when we rejoin a channel. “Our front-end instance hours are going up.” Maybe stuff can be refactored into a task queue or a backend. “Our data reads have spiked.” Perhaps we should cache the list of countries instead of querying the datastore every time the user brings up a certain dialog.

Kyle the Discounted

This entry was posted in Uncategorized and tagged , . Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • dave

    Great read, now I have a question, not sure if you could guide me and others about “not thinking with relations” for NoSql, it pains me to say that my application quickly hit 50k free reads of the datastore after like 4-6 hours of releasing the experimental deployment. 

    I do however plan to cache a lot of things that aren’t necessary to have at least every 1-2 hours and keep the data that matters 30 mins to 1 hour refreshed

    Sadly, it seems that I will be forced to have 2-3k reads from the datastore before inserting the data collected to keep it “unique” 

    It kinda worries me in a way, I’ll take the risk and release the app but getting charged and not getting a cent back is well yea… a bit hard to take 

  • Jee

    Datastore-admin thing doesn’t load properly in chrome if you are logged in to more than one account. It tries to redirect you to the screen where you choose which ones, and that seems to be problematic inside the iframe. If you inspect the source of the page, you can get the URL that the iframe is attempting to load, then open that in a new tab. Once you’ve then chosen an account, this will be remembered (Not sure for how long) and then it will work in the console. Or bookmark the URL (it’s quite long and not particularly memorable…) and just use it in a tab on it’s own. If you’re not logged-in to more than one account, well then I don’t know…but maybe this will help someone else.

  • Tomáš Rampas

    You got it. From other APIs perspective you are absolutely right…

  • http://kyle.baley.org Kyle Baley

    That’s (sort of) fine from the datastore perspective. But I also meant all the other features, like the Channel API, task queue, Memcache, Image API, etc, etc. If you want to take full advantage of what AppEngine has to offer, it means a whole lot of wrapper APIs. Which is something we’ve generally done anyway but more for testing purposes than to allow us to switching hosting environments.

  • Tomáš Rampas

    The price you pay for all these services is that if you ever decide to move away from AppEngine, you’re rewriting.
    As I’m fan of Django project I’m using djangononrel and djangoappengine by All Button Pressed in my GAE project. I’ve decided to use that framework after I read following article: http://code.google.com/appengine/articles/django-nonrel.html. So I believe I’m not stuck in GAE. But frankly speaking I’ve never tried to run my GAE-Django project having run elsewhere…