Light Web Strategy Game data model smacks into GAE limitations

As I mentioned in my last post, I had previously known that GAE could only operate on entities from one group in a transaction and had designed my data model to account for this, but recently came across information that Google App Engine locks an entire entity group while processing a transaction. As my light web strategy game is currently designed, the game world consists of isolated verses that can be connected by player actions. Once connected populous can be sent into verses beyond the one the player starts in, thus multiple players can have populous in any given verse at the same time and can in turn affect each other. I already had entities for Being (player), Verse, and VerseBeing with the latter holding all of the per being per verse information. Each VerseBeing is in the same entity group as the Verse it is associated with so no problem with transactions there and not really any with locking either (if necessary I’d limit the number of connections to any given realm).

The problem is that there a few properties on Being (e.g. A) that can modified as a result of another Being’s (e.g. B) orders in a Verse (e.g. X) where both A and B have populous. This isn’t dissimilar from the ‘global counter’ problem I ran across several solutions too while I was trying to find out more about the hard technical limits of GAE and while I could take those properties on Being and fracture them onto each VerseBeing so that the properties could be updated in the same transaction a Verse and its associated VerseBeing entities there would be a concurrency issue to solve. With the properties for Being entity factured across multiple VerseBeing entities there isn’t any way built in way to ensure that the whole of the properties (i.e. when you add the property value from each VerseBeing associated with a Being back up again to unfracture the property) are modified properly according to the game rules.

For example while Beings A and B both have populous in Verse X, Beings A and C may also have populous Verse Y. If both B and C simultaneously take actions that affect the same property of A, say each causing a automated response that use 60% of A’s available power then A will have universally expended more power then A actually had even those A’s expenditure was fine in each verse. Making sure that doesn’t happen requires that property needing to adhere to the game rules be modified a universal level, which means it can’t be fractured down to VerseBeing.

To get Beings A, B, and C and Verses X and Y and the associated VerseBeing entities all in the same transaction together would require that they all being in the same entity group, but that would in turn mean that all Beings and Verses would be in the same entity group which doesn’t scale since each player would need to lock the Being and Verse data every time they tried to take an action.

That isn’t to say there is no solution. For example a custom transaction model could be developed over the top of GAE’s data model or a command queue could be introduced that would serially process commands across multiple HTTP requests. This has me wondering if trying to deploy this game on to GAE is really worth while. On one hand, I could wait and try out GAE with an app more suited to its data model. On the other I could develop a platform layer on top of GAE and possibly make some coin off of that platform. I guess I’ll start off by making a list of my requirements for a platform layer on top of GAE and then see if anyone has done something similar already.

Update

After some thought I decided my requirements for an enhanced GAE data model layer were:

1. Transactions spanning entity groups
2. Checkpoints and journaling to allow rollback to a good state
3. Schema version management

Those later two are issues I raised in a previous post. I tried to locate an existing third party library that provided this, but didn’t turn anything up. I’ve got some concerns about what kind of perf overhead such a layer would have and I can’t allay those concerns without actually writing the library, so I think I’m going to go with a command queue and few one-off custom implementations for transactions were I need them in my current data model to see if GAE can handle that. If so then it’ll be worth looking into developing a generalized solution that meets my requirements above.