While the GAE SDK does provide the ability to run your web application locally, I’ve found that that the SDK’s local datastore doesn’t work precisely the same as the real one. Specially I’ve found that the local one is more permissive then the real one so its possible to get your app working locally and then have it fail once deployed to GAE.
Due GAE’s limitations on CPU time/cycles and an overall maximum limit of 10 seconds to fulfill a request a non-trivial set of unit tests will exceed the quotas if they are all run at the same time, so its necessary to run them in separate requests.
While GAEUnit provides server side infrastructure for creating the unit tests it is setup so that you either run all of the test in one request or run each test separately in a different request with the response being suitable for rendering in a browser. Thus it is designed to be run manually in a browser, which is quite tedious if you’ve developed numerous unit tests.
It seemed to me that running the tests via a REST API would be more convienent, so I went off in search of tools to help with this.
soapUI looks to be a full fledged web service (and just recently REST) testing tool, but it looks a little heavy weight to use as a client that just calls a series of REST APIs that are just providing access to a series of unit tests.
The command line tool curl could be used from a script/batch file; however, curl is just a wrapper over the C library libcurl and there are already Python binding for that provided by PycURL and in that same vein there is mechanize for “stateful programmatic web browsing in Python”, Python’s own urllib2, and urlgrabber.
There is Twill which is actually language developed for testing websites that also happens integrated with Python (I suspect its engine is written in Python, but didn’t delve into it to verify).
For those interested in additional tools, the end of this blog post and the comments on that posts have a number of useful links.
First I’m going to look at PycURL, urllib2, and urlgrabber in more depth. Then I’ll try one out with with a pair of REST APIs where the first provides the number of unit tests and the second takes the index of the test to run and returns either pass or the call stack of the exception causing the failure.
UPDATE: Well for what I was doing the urllib2 API was just fine:
count_of_tests = int(urllib2.urlopen(root_of_test_url + “count_of_tests”).read())