Login

Tag "database"

47 snippets

Snippet List

Friendly ID(Python 3.X)

This is just modified version of [friendly id](https://djangosnippets.org/snippets/1249/) for make this script compatible with python 3.x Invoice numbers like "0000004" are a little unprofessional in that they expose how many sales a system has made, and can be used to monitor the rate of sales over a given time. They are also harder for customers to read back to you, especially if they are 10 digits long. This is simply a perfect hash function to convert an integer (from eg an ID AutoField) to a unique number. The ID is then made shorter and more user-friendly by converting to a string of letters and numbers that wont be confused for one another (in speech or text). To use it: import friendly_id class MyModel(models.Model): invoice_id = models.CharField(max_length=6, null=True, blank=True, unique=True) def save(self, *args, **kwargs): super(MyModel, self).save(*args, **kwargs) # Populate the invoice_id if it is missing if self.id and not self.invoice_id: self.invoice_id = friendly_id.encode(self.id) self.save() if self.id and not self.invoice_id When an object from this model is saved, an invoice ID will be generated that does not resemble those surrounding it. For example, where you are expecting millions of invoices the IDs generated from the AutoField primary key will be: obj.id obj.invoice_id 1 TTH9R 2 45FLU 3 6ACXD 4 8G98W 5 AQ6HF 6 DV3TY ... 9999999 J8UE5 The functions are deterministic, so running it again sometime will give the same result, and generated strings are unique for the given range (the default max is 10,000,000). Specifying a higher range allows you to have more IDs, but all the strings will then be longer. You have to decide which you need: short strings or many strings :-) This problem could have also been solved using a random invoice_id generator, but that might cause collisions which cost time to rectify, especially when a decent proportion of the available values are taken (eg 10%). Anyhow, someone else has now already written this little module for you, so now you don't have to write your own :-)

  • database
  • field-id
  • invoice-id
  • invoice
Read More

Django chunked queryset iterator

The function slices a queryset into smaller querysets containing chunk_size objects and then yield them. It is used to avoid memory error when processing huge queryset, and also database error due to that the database pulls whole table at once. Concurrent database modification wouldn't make some entries repeated or skipped in this process.

  • django
  • python
  • database
  • queryset
  • iterator
  • memoryerror
Read More

Database backup with admin command

Detect type of database (MySQL, PostgreSQL or SQLite) and make backup. In this moment ONLY WORK in GNU/Linux, NOT WIN.

  • database
  • admin-actions
  • backup
  • MySQL
  • admin-command
  • SQLite
  • PostgreSQL
Read More
Author: jhg
  • 1
  • 3

Validation for full e-mails (e.g. "Joe Hacker <[email protected]>")

Out of the box, Django e-mail fields for both database models and forms only accept plain e-mail addresses. For example, `[email protected]` is accepted. On the other hand, full e-mail addresses which include a human-readable name, for example the following address fails validation in Django: Joe Hacker <[email protected]> This package adds support for validating full e-mail addresses. **Database model example** from django import models from full_email.models import FullEmailField class MyModel(models.Model): email = FullEmailField() **Forms example** from django import forms from full_email.formfields import FullEmailField class MyForm(forms.Form): email = FullEmailField(label='E-mail address') I maintain this code in a [GitHub gist](https://gist.github.com/1505228). It includes some unit tests as well.

  • forms
  • model
  • email
  • validation
  • orm
  • database
Read More

JsonObjectField

This fields.py file defines a new model field type, "JsonObjectField," which is designed to allow the storage of arbitrary Python objects in Django TextFields. It is intended primarily to allow the storage of Python dictionaries or list objects. As the name implies, it converts objects to JSON for storage; this conversion happens transparently, so from your model's perspective, the field stores and retrieves the actual objects.

  • model
  • json
  • database
  • object
Read More

PostgreSQL ON DELETE CASCADE

Have you always been annoyed by how you set up this elaborate big database schema and weren't able to have **ON DELETE CASCADE ON UPDATE CASCADE** in dbshell? This solves the problem; create the two files and and empty *__init__.py* and put them somewhere in your path. Then say DATABASE_ENGINE='postgresql_psycopg2_cascade' in settings. Really I'd like this to be in the ForeignKey object, were it upstream Django or an own version of it, but it doesn't seem possible. Ideas on how to make this configurable are more than welcome! Props go out to Ari Flinkman for the inspiration to do this!

  • database
  • postgres
  • foreign-key
  • postgresql
  • databases
Read More
Author: mjt
  • 1
  • 1

Models with database views

This example shows, how to use database views with django models. NewestArticle models contains 100 newest Articles. Remember, that NewestArticle model is read-only. Tested with mysql.

  • sql
  • models
  • views
  • view
  • model
  • mysql
  • database
Read More

Improved YAML serializer for large databases

I needed the ability to serialize and deserialize my database, which contains millions of objects. The existing XML serializer encountered spurious parse errors; the JSON serializer failed to handle UTF-8 even when it was asked to; and both the JSON and YAML serializers tried to keep all the representations in memory simultaneously. This custom serializer is the only one that has done the job. It uses YAML's "stream of documents" model so that it can successfully serialize and deserialize large databases.

  • serialize
  • database
  • yaml
Read More

Database file storage

Class DatabaseStorage can be used with either FileField or ImageField. It can be used to map filenames to database blobs: so you have to use it with a **special additional table created manually**. The table should contain: *a pk-column for filenames (I think it's better to use the same type that FileField uses: nvarchar(100)) *a blob column (image type for example) *a size column (bigint type). You can't just create blob column in the same table, where you defined FileField, since there is no way to find required row in the save() method. Also size field is required to obtain better perfomance (see size() method). So you can use it with different FileFields and even with different "upload_to" variables used. Thus it implements a kind of root filesystem, where you can define dirs using "upload_to" with FileField and store any files in these dirs. Beware saving file with the same "virtual path" overwrites old file. It uses either settings.DB_FILES_URL or constructor param 'base_url' (@see __init__()) to create urls to files. Base url should be mapped to view that provides access to files (see example in the class doc-string). To store files in the same table, where FileField is defined you have to define your own field and provide extra argument (e.g. pk) to save(). Raw sql is used for all operations. In constractor or in DB_FILES of settings.py () you should specify a dictionary with db_table, fname_column, blob_column, size_column and 'base_url'. For example I just put to the settings.py the following line: DB_FILES = {'db_table': 'FILES', 'fname_column': 'FILE_NAME', 'blob_column': 'BLOB', 'size_column': 'SIZE', 'base_url': 'http://localhost/dbfiles/' }" And use it with ImageField as following: player_photo = models.ImageField(upload_to="player_photos", storage = DatabaseStorage() ) DatabaseStorage class uses your settings.py file to perform custom connection to your database. The reason to use custom connection: http://code.djangoproject.com/ticket/5135 Connection string looks like "cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=localhost;DATABASE=testdb;UID=me;PWD=pass')" It's based on pyodbc module, so can be used with any database supported by pyodbc. I've tested it with MS Sql Express 2005. Note: It returns special path, which should be mapped to special view, which returns requested file: **View and usage Example:** def image_view(request, filename): import os from django.http import HttpResponse from django.conf import settings from django.utils._os import safe_join from filestorage import DatabaseStorage from django.core.exceptions import ObjectDoesNotExist storage = DatabaseStorage() try: image_file = storage.open(filename, 'rb') file_content = image_file.read() except: filename = 'no_image.gif' path = safe_join(os.path.abspath(settings.MEDIA_ROOT), filename) if not os.path.exists(path): raise ObjectDoesNotExist no_image = open(path, 'rb') file_content = no_image.read() response = HttpResponse(file_content, mimetype="image/jpeg") response['Content-Disposition'] = 'inline; filename=%s'%filename return response **Warning:** *If filename exist, blob will be overwritten, to change this remove get_available_name(self, name), so Storage.get_available_name(self, name) will be used to generate new filename.* For more information see docstrings in the code. Please, drop me a line if you've found a mistake or have a suggestion :)

  • files
  • database
  • storage
  • filestorage
Read More

MySQL "Text" Type Model Field

Custom field for using MySQL's `text` type. `text` is more compact than the `longtext` field that Django assigns for `models.TextField` (2^16 vs. 2^32, respectively)

  • text
  • models
  • mysql
  • db
  • database
  • field
  • custom-field
Read More

Pull ID from arbitrary sequence

Django does not currently allow one to pull ID values from arbitrarily named sequences. For example, if you did not create your ID column using the serial data type in PostgreSQL, you likely will not be able to use your sequences. This is quite a problem for those integrating with legacy databases. While ultimately the best place to fix this is django proper, this decorator will help people get by for now. Note that in this case, all of my sequences are named "pk_TABLENAME". You'll likely have a different convention and should update the decorator appropriately. While I could have made the pattern a parameter, it didn't seem like that would gain much here.

  • database
  • sequence
  • postgresql
Read More

autocompleter with database query

This is an improvement of snippet 253 in that it supports database queries. Implementing autocompletion for foreign keys takes a few steps: 1) Put the snippet above into <app>/widgets/autocomplete.py. 2) Create a view of your foreign key model (here: Donator) in <app>/donator/views.py: from models import Donator from widgets.autocomplete import autocomplete_response def autocomplete(request): return autocomplete_response( request.REQUEST['text'], Donator, ( 'line_1', 'line_2', 'line_3', 'line_4', 'line_5', 'line_6', 'line_7', 'line_8', '^zip_code', 'location' ) ) This view returns the autocompletion result by searching the fields in the tuple. Each word from the form field must appear at least in one database field. 3) Create a URLconf that points to this new view. 4) In the form where you need the autocompletion, define the widget of the foreign key field as an instance of AutoCompleteField: from widget.autocomplete import AutoCompleteField field.widget = AutoCompleteField( url='/donator/autocomplete/'), options={'minChars': 3} ) The url parameter is the URL connected to the view in step 3), the options dict is passed on to the Ajax.Autocompleter JavaScript object. Links: * [Snippet 253](http://www.djangosnippets.org/snippets/253/) * [Django and scriptaculous integration](http://wiki.script.aculo.us/scriptaculous/show/IntegrationWithDjango) * [Ajax.Autocompleter](http://wiki.script.aculo.us/scriptaculous/show/Ajax.Autocompleter)

  • ajax
  • selection
  • database
  • autocomplete
  • query
  • foreign-key
  • prototype
  • scriptaculous
  • protoculous
Read More