Login

Tag "import"

Snippet List

create or update, then get, model instances from JSON/py dict

Basically the idea is to import/update model instances from a json data that closely matches the model structure (i.e. identical field names) From my answer to this question: [http://stackoverflow.com/a/8377382/202168](http://stackoverflow.com/a/8377382/202168) See the original question for sample data format.

  • json
  • import
Read More

CSV to JSON Fixture

**This script converts a CSV file into a JSON file ready to be imported via `manage.py loaddata` like any other fixture data.** It can be used manually to do a one-time conversion (for placing into a /fixtures folder), or used in a fabric script that automatically converts CSV to JSON live then runs `loaddata` to import as fixture data. To run script: >`csv2json.py input_file_name model_name` > >e.g. csv2json.py airport.csv app_airport.Airport > >Note: input_file_name should be a path relative to where this script is. **Scripts depends on simplejson module.** The module can just be placed in a sub-folder to the script to make it easy to import. If you use the same Python binary that you use for your Django site, you could use the Django import instead: `from django.utils import simplejson` **File Input/Ouptut formats:** Assumes CSV files are saved with LF line endings, and that first line has field values. First column is the model's pk field. Sample CSV input: id,ident,name,city,state 1,00C,Animas Air Park,Durango,CO 6,00V,Meadow Lake,Colorado Springs,CO 7,00W,Lower Granite State,Colfax,WA 12,01J,Hilliard Airpark,Hilliard,FL Output file name is input name + ".json" extension. Sample JSON output: [ { "pk": 1, "model": "app_airport.Airport", "fields": { "name": "Animas Air Park", "city": "Durango", "ident": "00C", "state": "CO", } } ] **Debugging Conversion Problems** If JSON import errors out with "ValidationError: This value must be an integer", you probably have a blank in an Integer field within your CSV file, but if can't figure out, try setting a breakpoint in file: ./django/django/db/models/fields/__init__.py e.g. 688 try: 689 return int(value) 690 except (TypeError, ValueError): 691 import pdb; pdb.set_trace() 692 -> raise exceptions.ValidationError( 693 _("This value must be an integer.")) To figure out what field caused the error, while in the debugger: (Pdb) u <- to go UP the callstack (Pdb) field.name

  • json
  • loaddata
  • fixtures
  • csv
  • import
  • fixture
Read More

Dynamic import from an installed app

This is an example how to dynamically import modules other than `models.py` from installed apps. It allows you to define the full module path just once in `INSTALLED_APPS` in the settings. Using this technique makes it possible to write extensible and reusable apps as mentioned in [Abstract Models and Dynamicly Assigned Foreign Keys](http://djangotricks.blogspot.com/2009/02/abstract-models-and-dynamicly-assigned.html). Probably, it would be great to have some helpers for doing this in django itself. For example: from django.db import models from django.utils import importlib def import_installed(path): """ Imports a module from an installed app >>> import_installed("myapp.forms") <module 'myproject.apps.myapp.forms'> """ app_name, module = path.split(".", 1) app = models.get_app(app_name) return importlib.import_module(app.__name__[:-6] + module) UPDATE: Reported as ticket [#10703](http://code.djangoproject.com/ticket/10703)

  • dynamic
  • import
Read More

syncdata command

A django admin command that takes a fixture and makes the target database the same as that fixture, deleting objects that in the database but not in the fixture, updating objects that are different in the database, and inserting missing ones. Place this code in your_app/management/commands/syncdata.py You will need to use manage.py (not django-admin.py) for Django to recognise custom commands (see http://www.djangoproject.com/documentation/django-admin/#customized-actions). This snippet is the 'loaddata' command with this patch applied: http://code.djangoproject.com/ticket/7159 (with minor tweaks). The intention is that 'dumpdata' on system A followed by 'syncdata' on system B is equivalent to a database copy from A to B. The database structure in A and B must match.

  • admin
  • database
  • import
  • commands
  • copy
  • command
  • synchronise
  • publish
Read More

Dynamic Template Loader

This snippet provides support for dynamically importing templates. This helps you to avoid naming collisions and other problems. The format is as follows: 1. `module.submodule.app:template.html` 2. `module.submodule.app:subfolder/template.html` 3. `module.submodule.app:/subfolder/template.html` Assuming the module is located in '/var/', these would map (respectively) to: 1. `/var/module/submodule/app/templates/template.html` 2. `/var/module/submodule/app/templates/subfolder/template.html` 3. `/var/module/submodule/app/templates/subfolder/template.html` The colon splits the the python module from the template directory, meaning you can import from anything that has a "templates" directory. This helps me to avoid naming collisions by specifying the application I'm referring to, without having to put long paths in my extends and include tags inside other templates. It's also dynamic in that if I move a library outside the old path, it has no effect on the templates. To get this rolling, in your `settings.py` file, add the following:: >TEMPLATE_LOADERS = ( > 'addons.template_loader.load_template_source', # <--- add this > 'django.template.loaders.app_directories.load_template_source', > 'django.template.loaders.filesystem.load_template_source', ># 'django.template.loaders.eggs.load_template_source', >)

  • template
  • dynamic
  • import
  • loader
Read More

CSVImport

This is an awesome script! The original can be found here: http://www.djangosnippets.org/snippets/633/ I had to make a couple minor changes to get it working for me, so I thought I would share the love...

  • csv
  • import
Read More

module_from_path

I needed to dynamically import a module based on a path to that file on disk, without it necessarily being on the Python Path.

  • import
Read More

Automagically import settings from installed applications

Use this snippet at the end of your main settings.py file to automagically import the settings defined in each app of `INSTALLED_APPS` that begins with `APPS_BASE_NAME`. Set `APPS_BASE_NAME` to the base name of your Django project (e.g. the parent directory) and put `settings.py` files in every app directory (next to the `models.py` file) you want to have local settings in. # works in the Django shell >>> from django.conf import settings >>> settings.TEST_SETTING_FROM_APP "this is great for reusable apps" Please keep in mind that the imported settings will overwrite the already given and preceding settings, e.g. when you use the same setting name in different applications. Props to [bartTC](http://www.djangosnippets.org/users/bartTC/) for the idea.

  • settings
  • import
  • reusable
  • apps
  • applications
Read More

Simple Plone Migration

This is a very simple script to do a **simple** migration from plone content to django. ATNewsItems and PloneArticles are imported into the django model *Article* (with Foreignkey to the django models *File* and *Image*). ATDocuments are imported into the django model *Page*. **Usage** 1. Make sure that the Python version running Zope can import django 2. The django database should be writeable from within the Zope environment 3. Set the shell variable *DJANGO_MODULE_SETTING* to the settings file of the django project you want to import your Plone content to 4. Start the Zope server in the debug modus: $ bin/zopectl debug Then import the python module above, and pass the site you want to migrate to the start function: import simple_migration mysite = app.mysite simple_migration.start(mysite) Found 1253 objects Saved Test Document Type: ATNewsItem ... Hope it helps someone. Used for the migration of **Plone-2.5.3** content in a **Python-2.4.4** environment.

  • migration
  • import
  • plone
Read More

Autoload Django Models When Using ./manage.py shell

It's a pain to import all the Django models you want to use in the Python shell every time you start it. Here's how you can get IPython to autoload all your Django models for you every time you start the shell using ./manage.py shell. Put the code in a .py file in the root of your project. Then tell IPython to load the script in ~/.ipython/ipythonrc in the "Python files to load and execute" section.

  • import
  • shell
  • interpreter
  • autoload
Read More

Movable Type Import

This is a quick and dirty way to import blog entries and comments from Movable Type. Sorry I don't have time to document it better. Hopefully it will help you get started if you're looking to move from MT to Django.

  • type
  • import
  • movable-type
  • movable
  • mt
Read More

MaxMind(R) GeoIP Lite CSV Import

Use this script to import the Maxmind GeoIP lite CSV datasets into your database. This takes at least 200MB of RAM; the resulting database will be ~400MB. Stick in the same directory as the [models](http://www.djangosnippets.org/snippets/327/). Make sure to set `DEBUG=False` to prevent running out of memory during import.

  • log
  • csv
  • gis
  • ip
  • geolocation
  • maxmind
  • geodjango
  • import
Read More

14 snippets posted so far.