I often need to dump data from a database to csv. This little snippet should make it easy enough to do without having to worry too much about character encodings, though it does assume you want your csv file to be utf-8 encoded. Note that this dumps just one table from the database. Trying to dump all the tables in your app will raise an exception.
This is a basic view for exporting data from models. It is designed so that you can provide the GET variables used for searching/filtering within listdisplay pages in the admin, and the code will filter the same way. It allows ouputting of model data in various formats using serializers. This should be used as a urlpattern through get_url, so that you can use the admin_view decorator to properly secure the data.
This is a basic model view for exporting data from models. It is designed so that you can provide the GET variables used for searching/filtering within listdisplay pages in the admin, and the code will filter the same way. It allows the output of model data in various formats using serializers or templates.
A generic admin action to export selected objects as csv file. The csv file contains a first line with header information build from the models field names followed by the actual data rows. Access is limited to staff users. Requires django-1.1. **Usage:** Add the code to your project, e.g. a file called actions.py in the project root. Register the action in your apps admin.py: from myproject.actions import export_as_csv class MyAdmin(admin.ModelAdmin): actions = [export_as_csv]
**This script converts a CSV file into a JSON file ready to be imported via `manage.py loaddata` like any other fixture data.** It can be used manually to do a one-time conversion (for placing into a /fixtures folder), or used in a fabric script that automatically converts CSV to JSON live then runs `loaddata` to import as fixture data. To run script: >`csv2json.py input_file_name model_name` > >e.g. csv2json.py airport.csv app_airport.Airport > >Note: input_file_name should be a path relative to where this script is. **Scripts depends on simplejson module.** The module can just be placed in a sub-folder to the script to make it easy to import. If you use the same Python binary that you use for your Django site, you could use the Django import instead: `from django.utils import simplejson` **File Input/Ouptut formats:** Assumes CSV files are saved with LF line endings, and that first line has field values. First column is the model's pk field. Sample CSV input: id,ident,name,city,state 1,00C,Animas Air Park,Durango,CO 6,00V,Meadow Lake,Colorado Springs,CO 7,00W,Lower Granite State,Colfax,WA 12,01J,Hilliard Airpark,Hilliard,FL Output file name is input name + ".json" extension. Sample JSON output: [ { "pk": 1, "model": "app_airport.Airport", "fields": { "name": "Animas Air Park", "city": "Durango", "ident": "00C", "state": "CO", } } ] **Debugging Conversion Problems** If JSON import errors out with "ValidationError: This value must be an integer", you probably have a blank in an Integer field within your CSV file, but if can't figure out, try setting a breakpoint in file: ./django/django/db/models/fields/__init__.py e.g. 688 try: 689 return int(value) 690 except (TypeError, ValueError): 691 import pdb; pdb.set_trace() 692 -> raise exceptions.ValidationError( 693 _("This value must be an integer.")) To figure out what field caused the error, while in the debugger: (Pdb) u <- to go UP the callstack (Pdb) field.name
A subclass of `HttpResponse` which will transform a `QuerySet`, or sequence of sequences, into either an Excel spreadsheet or CSV file formatted for Excel, depending on the amount of data. All of this is done in-memory and on-the-fly, with no disk writes, thanks to the StringIO library. **Requires:** [xlwt](http://pypi.python.org/pypi/xlwt/) **Example:** def excelview(request): objs = SomeModel.objects.all() return ExcelResponse(objs)
Exporting unicode data to Excel in a CSV file is surprisingly difficult. After much experimentation, it turns out the magic combination is UTF-16, a byte order mark and tab-delimiters. This snippet provides two classes - UnicodeWriter and UnicodeDictWriter - which can be used to output Excel-compatible CSV.
This will generically add csv exporting to your views in the admin. It will default to exporting the entire table you see (without paging). If the table only has one column, it will export the fields the the model. You can overide this functionality. I ended up creating my own admin/change_list.html to apply this functionality universally: {% extends "admin/base_site.html" %} {% load adminmedia admin_list i18n %} {% block stylesheet %}{% admin_media_prefix %}css/changelists.css{% endblock %} {% block bodyclass %}change-list{% endblock %} {% if not is_popup %}{% block breadcrumbs %}<div class="breadcrumbs"><a href="../../">{% trans "Home" %}</a> › {{ cl.opts.verbose_name_plural|capfirst|escape }}</div>{% endblock %}{% endif %} {% block coltype %}flex{% endblock %} {% block content %} <div id="content-main"> {% block object-tools %} <ul class="object-tools"> <li><a href="csv/{%if request.GET%}?{{request.GET.urlencode}}{%endif%}" class="addlink">Export to CSV</a></li> {% if has_add_permission %} <li><a href="add/{% if is_popup %}?_popup=1{% endif %}" class="addlink">{% blocktrans with cl.opts.verbose_name|escape as name %}Add {{ name }}{% endblocktrans %}</a></li> {% endif %} </ul> {% endblock %} <div class="module{% if cl.has_filters %} filtered{% endif %}" id="changelist"> {% block search %}{% search_form cl %}{% endblock %} {% block date_hierarchy %}{% date_hierarchy cl %}{% endblock %} {% block filters %}{% filters cl %}{% endblock %} {% block result_list %}{% result_list cl %}{% endblock %} {% block pagination %}{% pagination cl %}{% endblock %} </div> </div> {% endblock %}
This is an awesome script! The original can be found here: http://www.djangosnippets.org/snippets/633/ I had to make a couple minor changes to get it working for me, so I thought I would share the love...
Importing data from other sources than SQL can be an annoyance. This script serves as a general tool for importing data from CSV files.
When given a model this utility method will export all data in that model to a CSV file. I use this method to place an "Export" button on a particular model's change list page in the Django administrator.
The first function (ftype_batch) is a view that passes the first part of the CSV filename and a queryset intended to write out to the file. run_batch prepares the HttpResponse for incrementally writing, and write_batch actually writes out the data. The logic in write_batch is custom to what I need done, but as long as the csv writer receives a sequence to write, it should work.
Use this script to import the Maxmind GeoIP lite CSV datasets into your database. This takes at least 200MB of RAM; the resulting database will be ~400MB. Stick in the same directory as the [models](http://www.djangosnippets.org/snippets/327/). Make sure to set `DEBUG=False` to prevent running out of memory during import.
A class called MetaOptions that enables decoration of Django models with meta classes in similar style to Admin and Meta. Included is an example usage to enable CSV export of any set of models. The package installs into django.contrib.options and is available for download at the [Python Cheeseshop](http://cheeseshop.python.org/pypi/django_options/r6)
This code generates a CSV file for a model. The URL is http://example.com/spreadsheets/app_label/model_name/ (replace spreadsheets for admin, from the URL for the model's admin page.)
30 snippets posted so far.