I had a problem: too many fetches from the DB. So, how to reduce load on the database without major changes to the code? Cache Manager is the answer. I've managed to reduce number of DB hits as much as 80% in some cases (dictionaries, complex relations). It is using standard cache mechanisms. I'm using it with mathopd.
This is a very simple solution, instead of standard Manager, put this in your model definition as:
objects = CacheManager()
Then everythere elase in the code instead of all() or get(...) call all_cached() or get_cached().
I've kept original methods intact, to have an dual access, when you really, really must have frest data from the DB, and you can't wait for cache to expire.
This is much easier to work with, then manually getting fetched data from the cache.No change to your existing code 9except model) and voila!
Additionally if you have some data, you would like to store with your serialized object (e.g. related data, dynamic dictionaries), you can do this in the model method '_init_instance_cache').
Drop me an email if you find this useful. :)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
from django.core.cache import cache from django.db import models class CacheManager(models.Manager): def _getCacheKey(self, id, subset='s'): return "%s_%s_%s" % (self.model._meta.object_name, subset, id) def all_cached(self): if self.__class__.__name__ == 'RelatedManager': key = "".join(["%s%s" % el for el in self.core_filters.items()]) cacheKey = self._getCacheKey(0, key) else: cacheKey = self._getCacheKey(0, 'all') qs = cache.get(cacheKey) if qs is None: qs = list(self.get_query_set()) for element in qs: if hasattr(element,'_init_instance_cache'): element._init_instance_cache() cache.set(cacheKey, qs) return qs def get_cached(self, *args, **kwargs): cacheKey = cat = None if self.__class__.__name__ == 'RelatedManager': key = "".join(["%s%s" % el for el in self.core_filters.items()]) else: key = "" lst = map(lambda x:(x,x.pk if isinstance(x,models.Model) else str(x)),kwargs.items()) key += "_" + "_".join(list(map(unicode,args))+list(map(lambda x:"%s:%s"%x,lst))) key = key.replace(" ","") cacheKey = self._getCacheKey(key) # check if we have a result already ached if cacheKey is not None: element = cache.get(cacheKey) # no results, try to get it from cache if element is None: element = self.get_query_set().get(*args, **kwargs) if hasattr(element,'_init_instance_cache'): element._init_instance_cache() # if this should be cached, write it to the cache if cacheKey is not None: cache.set(cacheKey, element) return element
More like this
- Serializer factory with Django Rest Framework by julio 5 months, 3 weeks ago
- Image compression before saving the new model / work with JPG, PNG by Schleidens 6 months, 1 week ago
- Help text hyperlinks by sa2812 7 months, 1 week ago
- Stuff by NixonDash 9 months, 2 weeks ago
- Add custom fields to the built-in Group model by jmoppel 11 months, 2 weeks ago