CI and Google App Engine, or “How to force your way in”

If you’re the type that skips titles, we’re using Google App Engine in our project, mostly because it’s there.

I jest. We’re using it because it’s scalable and high performance and reliable and whatever other reason you can think of that will keep me from having a boring debate on the difference between Google and Amazon and Microsoft.

Along with Google App Engine, we’re also using TeamCity for our continuous integration server. And generally speaking, these are two very disparate topics. I.e. I’m not finding App Engine to be very CI-friendly.

Deploying to App Engine from Eclipse or IntelliJ is dead simple. There are plug-ins that manage it and all you do is provide credentials. Doing it from the command line is not too bad. There’s a console utility, appcfg, that lets you upload to App Engine.

So on the surface, it appears CI and App Engine play nicely. But there are a couple of quirks that don’t sit well. We’ll start with the Ant tasks I’m using from the CI server:

      <target name="setAppEngineVersionToDayOfWeek">
              <format property="day.of.week" pattern="E" />
            <stringutil string="${day.of.week}" property="day.of.week">
          <replaceregexp file="war/web-inf/appengine-web.xml"
              byline="true" />
        <target name="setAppEngineVersionToCurrentDate">
              <format property="" pattern="yyyy-MM-dd" />
          <replaceregexp file="war/web-inf/appengine-web.xml"
              byline="true" />
        <target name="update"
            description="Uploads the application to App Engine.">
          <java classname=""
                  <sysproperty key="appengine.sdk.root" 
  value="${appengine.folder}" />
                          <fileset dir="${appengine.folder}/lib" 
  includes="**/*.jar" />
                  <arg value="" />
                  <arg value="--passin" />
                  <arg value="update" />
                  <arg value="war" />

The first two targets are nearly identical. In App Engine projects, there is an XML file that contains the config

Side note: Versions in App Engine either can’t start with a capital or can’t contain capital letters altogether. This was one of my more aggravating annoyances because it ain’t ‘zactly trivial to make a string lowercase in Ant. I had to import a whole new Ant add-on, Antelope, to do it. Hence the <stringutil> task in setAppEngineVersionToDayOfWeek.

uration for uploading to App Engine. One of the tags is <version /> which is the name of the version in App Engine. These two targets will configure the version name to be the name of the day of the week, and the current date (in yyyy-MM-dd format) respectively.

The update target does the actual work of executing appcfg to upload to App Engine. In TeamCity, I have two build configurations. One executes nightly and calls the setAppEngineVersionToDayOfWeek target followed by the update target. The second configuration executes weekly and calls setAppEngineVersionToCurrentDate, then update.

The first crack in the veneer appears in the update target. App Engine comes with an Ant task, <appcfg>. But it requires you to be logged in and as far as I can tell, you can’t pass in that info to the <appcfg> task. So the target calls AppCfg using the <java> task and with a hard-coded email and password.

There’s an alternative. You can log into the CI server, run appcfg from the command line, and provide your email and password. This will be cached for 24 hours. As long as you hit App Engine at least once in 24 hours, your credentials will remain cached.

Problem with that is that you have to provide the credentials while logged in with the user account that runs the TeamCity agent, which in our case is the local system account. The other problem is that it’s a nightly build. Which means it runs roughly every 24 hours. Didn’t take long for the credentials to get evicted from the cache. So we created a special account so that we could hard-code the credentials into the build file (because, y’know, no one wanted their e-mail addresses and passwords stored in the version control system).

Other suggestions are included here. I went with the least resistance approach because frankly, a nighly build on a team of two in a start-up company shouldn’t require a custom Ant task.

The next issue we hit is that App Engine has a limit of ten versions per application. Put your fingers down, I’ll do the math for you. It means that after one week of nightly builds, that leaves three slots available for weekly builds. With none left over for others.

Ideally, I’d like to keep maybe only the last three nightly builds and possibly the last three weekly builds. But as far as I can tell, there’s no automated way to delete versions from App Engine. So this has forced us to examine exactly what we want to accomplish with our automated builds in App Engine.

Finally, if anyone knows how to secure versions of the app from prying eyes, I’d love to see that feature implemented as well. You need to log in to administer the application on App Engine along with all the versions. But as far as I’ve been able to figure out, if you know the URL to a version of the app, it’s available to the public. Which has stepped up the priority of our “Customer should be able to log in” story.

Kyle the Engineered

This entry was posted in Google App Engine. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Matt Burns

    For those that don’t want to store their credentials in the ant script, here is an interactive alternative (no use for CI obviously):


    UPDATE: something is going wrong with this comment engine and my closing xml tags, you can read it properly here:

  • Kyle Baley

    That’s what the nightly one does now. It creates nightly builds named: sun, mon, tue, wed, etc. Come next Sunday, the previous one gets overwritten. Never thought to apply a similar trick to weekly builds with that week_number mod m formula. Good tip.

    Having said that, it became a little cumbersome having to remember what day it was in order to see the latest version of the app so we’ve tweaked it a bit. Now, there’s a single nightly build that’s overwritten each night. We keep the weekly builds and every month-ish, we’ll just manually delete the ones from two months back.

  • Nick Johnson

    If you only want to keep the last week of nightly builds and the last m weekly builds, why not just name the version you upload “nightly_” or “weekly” where y = week_number mod m, and so forth?

    That way, the seventh nightly will overwrite the first, the eighth will overwrite the second, and so forth.

  • Kyle Baley

    To be clear, I meant that I read that section last night and totally misinterpreted it. I’ve added a web.xml file and it seems to do the trick. Too well as a matter of fact because it secures the app locally as well. You can type whatever you want as a username though so it’s not a big deal. Will have to see if it gets in my way enough as a work.

    Thanks for the tip!

  • Kyle Baley

    Yes, it does exactly that. And I certainly didn’t read that section last night and totally misinterpret what it said the first time around, if that’s what you think.

  • http:// Blair Conrad

    Oh, sorry about that. As you might guess, I live in Pythonland. Of course, you didn’t actually _say_ you were deploying a Java app (but I should’ve guessed).
    Anyhow, I’m not an expert, but doesn’t the Security and Authentication section of the web.xml documentation correspond to the original page that I mentioned?

  • Kyle Baley


    The documentation suggests that that works for Python apps on App Engine. I can’t find a corresponding page for the Java version.

  • http:// Blair Conrad

    Regarding your question about securing vresions of the app from prying eyes, is there a reason you can’t just use the login element in t he app.yaml, as specified at ?
    I would think could have your nightly/weekly builds set (selected) login element to administrator before deployment.