Django Tips #20 Working With Multiple Settings Modules

Usually, it’s a good idea to avoid multiple configuration files, instead, keep your project setup simple. But that’s not always possible, as a Django project starts to grow, the settings.py module can get fairly complex. In those cases, you also want to avoid using if statements like if not DEBUG: # do something.... For clarity and strict separation of what is development configuration and what is production configuration, you can break down the settings.py module into multiple files.

Basic Structure

A brand new Django project looks like this:

mysite/
 |-- mysite/
 |    |-- __init__.py
 |    |-- settings.py
 |    |-- urls.py
 |    +-- wsgi.py
 +-- manage.py

First thing we want to do is to create a folder named settings, rename the settings.py file to base.py and move it inside the newly created settings folder. Make sure you also add a __init__.py in case you are working with Python 2.x.

mysite/
 |-- mysite/
 |    |-- __init__.py
 |    |-- settings/         <--
 |    |    |-- __init__.py  <--
 |    |    +-- base.py      <--
 |    |-- urls.py
 |    +-- wsgi.py
 +-- manage.py

As the name suggests, the base.py will provide the common settings among all environments (development, production, staging, etc).

Next step now is to create a settings module for each environment. Common use cases are:

ci.py
development.py
production.py
staging.py

The file structure would look like this:

mysite/
 |-- mysite/
 |    |-- __init__.py
 |    |-- settings/
 |    |    |-- __init__.py
 |    |    |-- base.py
 |    |    |-- ci.py
 |    |    |-- development.py
 |    |    |-- production.py
 |    |    +-- staging.py
 |    |-- urls.py
 |    +-- wsgi.py
 +-- manage.py

Configuring a New Settings.py

First, take as example the following base.py module:

settings/base.py

from decouple import config

SECRET_KEY = config('SECRET_KEY')

INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    'mysite.core',
    'mysite.blog',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'mysite.urls'

WSGI_APPLICATION = 'mysite.wsgi.application'

There are a few default settings missing, which I removed so the example doesn’t get too big.

Now, to create a development.py module that “extends” our base.py settings module, we can achieve it like this:

settings/development.py

from .base import *

DEBUG = True

INSTALLED_APPS += [
    'debug_toolbar',
]

MIDDLEWARE += ['debug_toolbar.middleware.DebugToolbarMiddleware', ]

EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'

DEBUG_TOOLBAR_CONFIG = {
    'JQUERY_URL': '',
}

And a production.py module could be defined like this:

settings/production.py

from .base import *

DEBUG = False

ALLOWED_HOSTS = ['mysite.com', ]

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': '127.0.0.1:11211',
    }
}

EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'smtp.mailgun.org'
EMAIL_PORT = 587
EMAIL_HOST_USER = config('EMAIL_HOST_USER')
EMAIL_HOST_PASSWORD = config('EMAIL_HOST_PASSWORD')
EMAIL_USE_TLS = True

Two important things to note: avoid using star imports (import *). This is one of the few exceptions. Star imports may put lots of unecessary stuff in the namespace which in some cases can cause issues. Another important thing, even though we are using different files for development and production, you still have to protect sensitive data! Make sure you keep passwords and secret keys in environment variables or use a library like Python-Decouple which I highly recommend!

How to Use It

Since we no longer have a settings.py in the project root, running commands like python manage.py runserver will no longer work. Instead, you have to pass which settings.py module you want to use in the command line:

python manage.py runserver --settings=mysite.settings.development

python manage.py migrate --settings=mysite.settings.production

The next step is optional, but since we use manage.py often during the development process, you can edit it to set the default settings module to your development.py module.

To do that, simply edit the manage.py file, like this:

manage.py

#!/usr/bin/env python
import os
import sys

if __name__ == "__main__":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings.development")  # <-- Change here!
    try:
        from django.core.management import execute_from_command_line
    except ImportError:
        # The above import may fail for some other reason. Ensure that the
        # issue is really that Django is missing to avoid masking other
        # exceptions on Python 2.
        try:
            import django
        except ImportError:
            raise ImportError(
                "Couldn't import Django. Are you sure it's installed and "
                "available on your PYTHONPATH environment variable? Did you "
                "forget to activate a virtual environment?"
            )
        raise
    execute_from_command_line(sys.argv)

So, basically we changed the line from:

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")

To:

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings.development")

Now you can run the manage.py commands again without using the --settings argument. But remember to refer to the correct settings module in production!

Some Things You Can Do

Since now we have different settings modules, one thing you can do is removing the AUTH_PASSWORD_VALIDATORS from the settings/base.py and only add it to settings/production.py module. This way you use simple passwords like “123” during development but in the production environment it will be protected by the validators.

In your settings/tests.py or settings/ci.py you can override the following configuration so your tests run faster:

DATABASES['default'] = {
    'ENGINE': 'django.db.backends.sqlite3'
}

PASSWORD_HASHERS = (
    'django.contrib.auth.hashers.MD5PasswordHasher',
)

Conclusions

I hope you found this post useful somehow! Remember to use it carefully!

If you have any questions, please post in the comments below!

tips