This is the view-code to export a database-dump (hardcoded for mysql) or a media_root dump via the admin interface. just add 2 urls patterns to call the views and a small template with a simple form to send a http-post to the views. Note: The downloads are sort of streaming. I have successfully exportet a 2GB media_root as tar-ball without major increase of ram-usage of the django-process.
Every now and then you need to have items in your database which have a specific order. As SQL does not save rows in any order, you need to take care about this for yourself. No - actually, you don't need to anymore. You can just use this file - it is designed as kind-of plug-in for the Django ORM. Usage is (due to use of meta-classes) quite simple. It is recommended to save this snippet into a separate file called `positional.py`. To use it, you only have to import `PositionalSortMixIn` from the `positional` module and inherit from it in your own, custom model (but *before* you inherit from `models.Model`, the order counts). Usage example: Add this to your `models.py` from positional import PositionalSortMixIn class MyModel(PositionalSortMixIn, models.Model): name = models.CharField(maxlength=200, unique=True) Now you need to create the database tables: `PositionalSortMixIn` will automatically add a `postition` field to your model. In your views you can use it simply with `MyModel.objects.all().order_by('position')` and you get the objects sorted by their position. Of course you can move the objects down and up, by using `move_up()`, `move_down()` etc. In case you feel you have seen this code somewhere - right, this snippet is a modified version of [snippet #245](/snippets/245/) which I made earlier. It is basically the same code but uses another approach to display the data in an ordered way. Instead of overriding the `Manager` it adds the `position` field to `Meta.ordering`. Of course, all of this is done automatically, you only need to use `YourItem.objects.all()` to get the items in an ordered way. Update: Now you can call your custom managers `object` as long as the default manager (the one that is defined first) still returns all objects. This Mix-in absolutely needs to be able to access all elements saved. In case you find any errors just write a comment, updated versions are published here from time to time as new bugs are found and fixed.
I once needed to convert a Django project from PostgreSQL to SQLite. At that time I was either unaware of manage.py dumpdata/loaddata or it they didn't yet exist. I asked for advice on the #django IRC channel where ubernostrum came up with this plan: simple process: 1) Select everything. 2) Pickle it. 3) Save to file. 4) Read file. 5) Unpickle. 6) Save to db. :) Or something like that. First I thought it was funny, but then started to think about it and it made perfect sense. And so dbpickle.py was born. I've used this script also for migrating schema changes to production databases. For migration you can write plugins to hook on dbpickle.py's object retrieval and saving. This way you can add/remove/rename fields of objects on the fly when loading a dumped database. It's also possible to populate new fields with default values or even values computed based on the object's other properties. A good way to use this is to create a database migration plugin for each schema change and use it with dbpickle.py to migrate the project. See also [original blog posting](http://akaihola.blogspot.com/2006/11/database-conversion-django-style.html) and [my usenet posting](http://groups.google.com/group/django-users/browse_thread/thread/6a4e9781d08ae815/c5c063a288483e07#c5c063a288483e07) wondering about the feasibility of this functionality with manage.py dumpdata/loaddata. See [trac site](http://trac.ambitone.com/ambidjangolib/browser/trunk/dbpickle/dbpickle.py) for version history.
Here's a simple way to transparently encrypt a field (just repeat the idiom for more than one) so that it is not stored plaintext in the database. This might be useful for social security numbers, etc. The storage size for the ciphertext depends on which algorithm you use. Blowfish here requires 32 characters to store an encrypted 16 characters. Note also that Blowfish requires a block size of a multiple of 8, so that is what the repeat in the _set_ssn() is all about. The Crypto module is from http://www.amk.ca/python/code/crypto.html I make no claims as to how secure this scheme is overall, comments on this are welcome.
Hi, I made some small custom psycopg2 backend that implements persistent connection using global variable. With this I was able to improve the amout of requests per second from 350 to 1600 (on very simple page with few selects) Just save it in the file called base.py in any directory (e.g. postgresql_psycopg2_persistent) and set in settings DATABASE_ENGINE to projectname.postgresql_psycopg2_persistent This code is threadsafe, however because python don't use multiple processors with threads you won't get bit performance boost with this one. I really recommend using it in daemon mode. In apache mod_wsgi just set processes=8 threads=1
A django admin command that takes a fixture and makes the target database the same as that fixture, deleting objects that in the database but not in the fixture, updating objects that are different in the database, and inserting missing ones. Place this code in your_app/management/commands/syncdata.py You will need to use manage.py (not django-admin.py) for Django to recognise custom commands (see http://www.djangoproject.com/documentation/django-admin/#customized-actions). This snippet is the 'loaddata' command with this patch applied: http://code.djangoproject.com/ticket/7159 (with minor tweaks). The intention is that 'dumpdata' on system A followed by 'syncdata' on system B is equivalent to a database copy from A to B. The database structure in A and B must match.
*Incredibly useful for storing just about anything in the database (provided it is Pickle-able, of course) when there isn't a 'proper' field for the job:* A field which can store any pickleable object in the database. It is database-agnostic, and should work with any database backend you can throw at it. Pass in any object and it will be automagically converted behind the scenes, and you never have to manually pickle or unpickle anything. Also works fine when querying. **Please note that this is supposed to be two files, one fields.py and one tests.py (if you don't care about the unit tests, just use fields.py)**
[Based on snippet #513 by obeattie.](http://www.djangosnippets.org/snippets/513/) **Update 10/10/09:** [Further development is now occurring on GitHub, thanks to Shrubbery Software.](http://github.com/shrubberysoft/django-picklefield) Incredibly useful for storing just about anything in the database (provided it is Pickle-able, of course) when there isn't a 'proper' field for the job. `PickledObjectField` is database-agnostic, and should work with any database backend you can throw at it. You can pass in any Python object and it will automagically be converted behind the scenes. You never have to manually pickle or unpickle anything. Also works fine when querying; supports `exact`, `in`, and `isnull` lookups. It should be noted, however, that calling `QuerySet.values()` will only return the encoded data, not the original Python object. *Please note that this is supposed to be two files, one fields.py and one tests.py (if you don't care about the unit tests, just use fields.py).* This PickledObjectField has a few improvements over the one in [snippet #513](http://www.djangosnippets.org/snippets/513/). 1. This one solves the `DjangoUnicodeDecodeError` problem when saving an object containing non-ASCII data by base64 encoding the pickled output stream. This ensures that all stored data is ASCII, eliminating the problem. 2. `PickledObjectField` will now optionally use `zlib` to compress (and uncompress) pickled objects on the fly. This can be set per-field using the keyword argument "compress=True". For most items this is probably **not** worth the small performance penalty, but for Models with larger objects, it can be a real space saver. 3. You can also now specify the pickle protocol per-field, using the protocol keyword argument. The default of `2` should always work, unless you are trying to access the data from outside of the Django ORM. 4. Worked around a rare issue when using the `cPickle` and performing lookups of complex data types. In short, `cPickle` would sometimes output different streams for the same object depending on how it was referenced. This of course could cause lookups for complex objects to fail, even when a matching object exists. See the docstrings and tests for more information. 5. You can now use the `isnull` lookup and have it function as expected. A consequence of this is that by default, `PickledObjectField` has `null=True` set (you can of course pass `null=False` if you want to change that). If `null=False` is set (the default for fields), then you wouldn't be able to store a Python `None` value, since `None` values aren't pickled or encoded (this in turn is what makes the `isnull` lookup possible). 6. You can now pass in an object as the default argument for the field without it being converted to a unicode string first. If you pass in a callable though, the field will still call it. It will *not* try to pickle and encode it. 7. You can manually import `dbsafe_encode` and `dbsafe_decode` from fields.py if you want to encode and decode objects yourself. This is mostly useful for decoding values returned from calling `QuerySet.values()`, which are still encoded strings. The tests have been updated to match the added features, but if you find any bugs, please post them in the comments. My goal is to make this an error-proof implementation. **Note:** If you are trying to store other django models in the `PickledObjectField`, please see the comments for a discussion on the problems associated with doing that. The easy solution is to put django models into a list or tuple before assigning them to the `PickledObjectField`. **Update 9/2/09:** Fixed the `value_to_string` method so that serialization should now work as expected. Also added `deepcopy` back into `dbsafe_encode`, fixing #4 above, since `deepcopy` had somehow managed to remove itself. This means that lookups should once again work as expected in **all** situations. Also made the field `editable=False` by default (which I swear I already did once before!) since it is never a good idea to have a `PickledObjectField` be user editable.
Inspired by http://www.djangosnippets.org/snippets/159/ This context processor provides a new variable {{ sqldebug }}, which can be used as follows: {% if sqldebug %}...{% endif %} {% if sqldebug.enabled %}...{% endif %} This checks settings.SQL_DEBUG and settings.DEBUG. Both need to be True, otherwise the above will evaluate to False and sql debugging is considered to be disabled. {{ sqldebug }} This prints basic information like total number of queries and total time. {{ sqldebug.time }}, {{ sqldebug.queries.count }} Both pieces of data can be accessed manually as well. {{ sqldebug.queries }} Lists all queries as LI elements. {% for q in sqldebug.queries %} <li>{{ q.time }}: {{ q }}</li> {% endfor %} Queries can be iterated as well. The query is automatically escaped and contains <wbr> tags to improve display of long queries. You can use {{ q.sql }} to access the unmodified, raw query string. Here's a more complex example. It the snippet from: http://www.djangosnippets.org/snippets/93/ adjusted for this context processor. {% if sqldebug %} <div id="debug"> <p> {{ sqldebug.queries.count }} Quer{{ sqldebug.queries|pluralize:"y,ies" }}, {{ sqldebug.time }} seconds {% ifnotequal sql_queries|length 0 %} (<span style="cursor: pointer;" onclick="var s=document.getElementById('debugQueryTable').style;s.display=s.display=='none'?'':'none';this.innerHTML=this.innerHTML=='Show'?'Hide':'Show';">Show</span>) {% endifnotequal %} </p> <table id="debugQueryTable" style="display: none;"> <col width="1"></col> <col></col> <col width="1"></col> <thead> <tr> <th scope="col">#</th> <th scope="col">SQL</th> <th scope="col">Time</th> </tr> </thead> <tbody> {% for query in sqldebug.queries %}<tr class="{% cycle odd,even %}"> <td>{{ forloop.counter }}</td> <td>{{ query }}</td> <td>{{ query.time }}</td> </tr>{% endfor %} </tbody> </table> </div> {% endif %}
This is what I use to send simple status emails from my sites. Instead of a django.core.mail.send_mail call, which can take an irrritatingly, nondeterministically long time to return (regardless of error state), you can stow the emails in the database and rely on a separate interpreter process send them off (using a per-minute cron job or what have you). You then also have a record of everything you've sent via email from your code without having to configure your outbound SMTP server to store them or anything like that. Usage notes are in the docstring; share and enjoy.
This function is designed to make it easier to specify client-side query filtering options using JSON. Django has a great set of query operators as part of its database API. However, there's no way I know of to specify them in a way that's serializable, which means they can't be created on the client side or stored. `build_query_filter_from_spec()` is a function that solves this problem by describing query filters using a vaguely LISP-like syntax. Query filters consist of lists with the filter operator name first, and arguments following. Complicated query filters can be composed by nesting descriptions. Read the doc string for more information. To use this function in an AJAX application, construct a filter description in JavaScript on the client, serialize it to JSON, and send it over the wire using POST. On the server side, do something like: > `from django.utils import simplejson` > `filterString = request.POST.get('filter', '[]')` > `filterSpec = simplejson.loads(filterString)` > `q = build_query_filter_from_spec(filterSpec)` > `result = Thing.objects.filter(q)` You could also use this technique to serialize/marshall a query and store it in a database.
This is useful especially during the model-creation stage, when things are in constant flux. The `freshdb` command will drop the project's database, then create a new one. A common use case: manage.py freshdb manage.py syncdb
I hate when my unittest hits database. Especially when each test case needs different dataset. So I wrote this db mock, that's local to specific test and uses sqlite3 in-memory db. Usage (nosetests): class TestMainNoData(DbMock): 'testing main function with no meaningful data' def test_no_posts(self): 'there are no posts' assert models.Post.objects.count() == 0, 'should be no feeds'
First off: This snippet is more or less obsoleted by [snippet #259](/snippets/259/). The way that snippet #259 uses feels cleaner and more usable. It is basically the same code but the magic works a little bit different. In case you prefer this way, I've left this snippet as-is. Maybe you know this problem: you have some objects in your database and would like to display them in some way. The problem: the Django ORM does not provide any way to sort model instances by a user-defined criterion. There was already a plug-in in SQLAlchemy called [orderinglist](http://www.sqlalchemy.org/docs/plugins.html#plugins_orderinglist), so implementing this in Django was basically not a big deal. Usage is (due to use of meta-classes) quite simple. It is recommended to save this snippet into a separate file called `positional.py`. To use it, you only have to import `PositionalSortMixIn` from the `positional` module and inherit from it in your own, custom model (but *before* you inherit from `models.Model`, the order counts).
..