GAE docs hint at limits but provide no clue as to magnitude

This page of the GAE docs states:

To prevent the update of an entity from taking too long, the datastore limits the number of index entries that a single entity can have. The limit is large, and most applications will not notice. However, there are some circumstances where you might encounter the limit. For example, an entity with very many single-value properties can exceed the index entry limit.

Properties with multiple values, such as using a list value or a ListProperty model, store each value as a separate entry in an index. An entity with a single property with very many values (a very long list) can exceed the index entry limit.

Ambiguous statements such as “The limit is large”, “very many single-value properties”, and “a very long list” are not at all helpful. Give us a number please. If the actually limit is something like 128-256 depending on various factors, tell us that we’ll be safe with 30. Leaves you plenty of head room to change things, but it at least gives us a clue as to what kind of magnitude you’re talking about.

Update

And here is another example from this page:

Also, multiple users attempting to update entities in the same entity group at the same time will cause some users to retry their transactions, possibly causing some to fail to commit changes.

which is nice an ambiguous; however, the author of this blog post managed to state it without ambiguity (emphasis directly copied):

Write access is serialized per entity group. As opposed to a traditionnal RDBMS that provides row locking, the datastore only provides entity group locking. Writes to the a single entity group will always happen in sequence, even though changes concern different entities.

That’s a rather key piece of information when it comes to modeling you data. Something that is unambiguous in the GAE docs is that a transaction can only operate on a single entity group, so I setup my entity groups to ensure that any entities that would need to be modified by a user command would be in the same group. Now that I know this means the entirety of each group will be locked on a write and since every game action will result in a write my current data model isn’t going to scale to very many users even though there will be little overlap between which entities user actions are trying to write to.

Advertisements
This entry was posted in Web development and tagged , , , . Bookmark the permalink.

One Response to GAE docs hint at limits but provide no clue as to magnitude

  1. Pingback: Light Web Strategy Game data model smacks into GAE limitations « Josh Heitzman’s Impersonal Blog

Have something to say?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s