PerformanceEngine is a simple wrapper module that enables layered data model and query result storage on Google Application Engine.

More features coming soon!

• Layered data storage (local,memcache or datastore).
• Models that live in cache only (local or memcache).
• Cached queries!
• Lighweight (1 package, 2 files)
• Seamless integration into existing projects (call pdb.put instead of db.put).
• Different result types (list, key-model dict,name-model dict) to increase developer performance.
• Built-in handlers for common errors. (DeadlineExceededError,CapabilityDisabledError)
• Unified and simple API (if you figure out how to use pdb.get and pdb.put, you're set for good)

• cachepy

• Applications that have high amount and frequency of write operations.
• Applications that do batch data processing.
• Applications that like to stay under free quota of App Engine.

• Applications that do lots of transactions with highly sensitive data.
• Applications that rely heavily on queries rather than get and put operations.

If no additional parameters are given, PerformanceEngine works as memcache supported datastore. A get() function will try to get entities from memcache first, failing that, it tries to retrieve from datastore. It will then write entities that are fetched from datastore into memcache.

models = [model1,model2..]
keys = pdb.put()
#Key for an entity that's in datastore but not in memcache
keys.append(new_db_key) 

#model1 & model2 is retrieved from memcache but model for
# new_db_key is retrieved from datastore. It is also saved 
#into memcache for future calls.

pdb.get(keys)

Using different storage layers:

from PerformanceEngine import pdb

model1 = SomeModel(key_name='model1')
model2 = SomeModel(key_name='model2')
model3 = SomeModel(key_name='model3')

#Write to local cache
pdb.put(model1,_storage='local')

#Write to both memcache and local cache
pdb.put(model2,_storage=['local','memcache']

#Write to all layers
pdb.put(model3,_storage=['local','memcache','datastore'])

#Read from local cache
model1 = SomeModel.get_by_key_name('model1',_storage='local')

#Read from memcache 
model2 = SomeModel.get_by_key_name('model2',_storage='memcache')

#Read from datastore (same as db.get)
model3 = SomeModel.get_by_key_name('model3',_storage='datastore')

Please see wiki.

There aren't any example projects (yet) but PerformanceEngine is heavily used in TweetHit.

• If you try to write models without key_names into cache only without writing them to datastore first, it will fail as they won't have any keys created for them.

This module is released under the BSD License. You can find the full text of the license in the LICENSE file.

• Email: ocanbascil at gmail dot com
• Follow me on Twitter