Login

Quick script to convert json data to csv

Author:
stephenemslie
Posted:
April 28, 2010
Language:
Python
Version:
1.1
Score:
0 (after 0 ratings)

I often need to dump data from a database to csv. This little snippet should make it easy enough to do without having to worry too much about character encodings, though it does assume you want your csv file to be utf-8 encoded.

Note that this dumps just one table from the database. Trying to dump all the tables in your app will raise an exception.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
"""convert json to csv
Author: Stephen Emslie
"""

USAGE = """
    django-admin.py dumpdata myapp.mymodel > /path/to/in.json
    python %prog /path/to/in.json /path/to/out.csv

    or in one command:

    django-admin.py dumpdata myapp.mymodel | python %prog - /tmp/out.csv
"""

import csv
import sys
import tempfile
import simplejson
from optparse import OptionParser

if __name__ == '__main__':
    parser = OptionParser(usage=USAGE)
    options, args = parser.parse_args()
    path, outpath = args[0], args[1]
    if path == '-':
        source = sys.stdin
    else:
        source = open(path)
    s = source.read()
    objects = [object['fields'] for object in simplejson.loads(s)]
    writer = csv.DictWriter(open(outpath, 'w'), objects[0].keys())
    objects.insert(0, dict(zip(objects[0].keys(), objects[0].keys())))
    for d in objects:
      for key, value in d.items():
        if isinstance(value, basestring):
          d[key] = value.encode('utf8')
    writer.writerows(objects)

More like this

  1. Generate and render HTML Table by LLyaudet 6 days ago
  2. My firs Snippets by GutemaG 1 week, 2 days ago
  3. FileField having auto upload_to path by junaidmgithub 1 month, 2 weeks ago
  4. LazyPrimaryKeyRelatedField by LLyaudet 1 month, 3 weeks ago
  5. CacheInDictManager by LLyaudet 1 month, 3 weeks ago

Comments

Please login first before commenting.