Login

Database cleanup

Author:
skyjur
Posted:
July 28, 2010
Language:
Python
Version:
1.2
Score:
3 (after 3 ratings)

About

I tried to dump data from my database (manage.py dumpdata) and I couldn't do it because of error: 

User matching query does not exists

I found out that my database was filled with garbage: entries those foreigners were deleted. My table's engine is MyISAM so it allows for these lost entries to exist. I had to cleanup my database before I do datadump, so I've written a script which worked fine for me.

Usage

Place this script in same directory with your settings.py and run it: python db_cleanup.py

Disclaimer

Backup your data :) 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
#! /usr/bin/env python
'''Cleans database from garbage: objects which lost their required relations.'''

import setup_environment

from django.db import connection
from django.db.models.fields.related import RelatedField

from django.core.exceptions import ObjectDoesNotExist

def main(argv=None):
    tables = connection.introspection.table_names()
    seen_models = connection.introspection.installed_models(tables)
    for model in seen_models:
        cleanup_model(model)
        
def cleanup_model(model):
    print 'Cleaning ' + str(model) + '...'
    
    #get relational fields:
    fields = []
    for f in model._meta.fields:
        if isinstance(f, RelatedField):
            fields.append(f)
    
    for obj in model.objects.all():
        for f in fields:
            try:
                getattr(obj, f.name)
            except ObjectDoesNotExist:
                print 'Object #%i has unrelated field: %s == %s' % \
                    (obj.pk, f.name, f.value_from_object(obj))
                if f.null == False: 
                    print ' * Removing this object...'
                    obj.delete()
                    break
                else:
                    print ' * Fixing object with null value...'
                    setattr(obj, f.name, None)
            
            
if __name__ == '__main__':
    main()

More like this

  1. Template tag - list punctuation for a list of items by shapiromatron 3 months, 1 week ago
  2. JSONRequestMiddleware adds a .json() method to your HttpRequests by cdcarter 3 months, 2 weeks ago
  3. Serializer factory with Django Rest Framework by julio 10 months, 2 weeks ago
  4. Image compression before saving the new model / work with JPG, PNG by Schleidens 11 months ago
  5. Help text hyperlinks by sa2812 12 months ago

Comments

Please login first before commenting.