In this tutorial I’m going to show you how I usually start and organize a new Django project nowadays. I’ve tried many different configurations and ways to organize the project, but for the past 4 years or so this has been consistently my go-to setup.
Please note that this is not intended to be a “best practice” guide or to fit every use case. It’s just the way I like to use Django and that’s also the way that I found that allow your project to grow in healthy way.
Index
- Premises
- Environments/Modes
- Project Configuration
- Apps Configuration
- Code style and formatting
- Conclusions
Premises
Usually those are the premises I take into account when setting up a project:
- Separation of code and configuration
- Multiple environments (production, staging, development, local)
- Local/development environment first
- Internationalization and localization
- Testing and documentation
- Static checks and styling rules
- Not all apps must be pluggable
- Debugging and logging
Environments/Modes
Usually I work with three environment dimensions in my code: local, tests and production. I like to see it
as a “mode” how I run the project. What dictates which mode I’m running the project is which settings.py
I’m currently
using.
Local
The local dimension always come first. It is the settings and setup that a developer will use on their local machine.
All the defaults and configurations must be done to attend the local development environment first.
The reason why I like to do it that way is that the project must be as simple as possible for a new hire to clone the repository, run the project and start coding.
The production environment usually will be configured and maintained by experienced developers and by those who are more familiar with the code base itself. And because the deployment should be automated, there is no reason for people being re-creating the production server over and over again. So it is perfectly fine for the production setup require a few extra steps and configuration.
Tests
The tests environment will be also available locally, so developers can test the code and run the static checks.
But the idea of the tests environment is to expose it to a CI environment like Travis CI, Circle CI, AWS Code Pipeline, etc.
It is a simple setup that you can install the project and run all the unit tests.
Production
The production dimension is the real deal. This is the environment that goes live without the testing and debugging utilities.
I also use this “mode” or dimension to run the staging server.
A staging server is where you roll out new features and bug fixes before applying to the production server.
The idea here is that your staging server should run in production mode, and the only difference is going to be your static/media server and database server. And this can be achieved just by changing the configuration to tell what is the database connection string for example.
But the main thing is that you should not have any conditional in your code that checks if it is the production or staging server. The project should run exactly in the same way as in production.
Project Configuration
Right from the beginning it is a good idea to setup a remote version control service. My go-to option is Git on GitHub. Usually I create the remote repository first then clone it on my local machine to get started.
Let’s say our project is called simple
, after creating the repository on GitHub I will create a directory named
simple
on my local machine, then within the simple
directory I will clone the repository, like shown on the
structure below:
Then I create the virtualenv
outside of the Git repository:
Then alongside the simple
and venv
directories I may place some other support files related to the project which I
do not plan to commit to the Git repository.
The reason I do that is because it is more convenient to destroy and re-create/re-clone both the virtual environment or the repository itself.
It is also good to store your virtual environment outside of the git repository/project root so you don’t need to bother ignoring its path when using libs like flake8, isort, black, tox, etc.
You can also use tools like virtualenvwrapper
to manage your virtual environments, but I prefer doing it that way
because everything is in one place. And if I no longer need to keep a given project on my local machine, I can delete
it completely without leaving behind anything related to the project on my machine.
The next step is installing Django inside the virtualenv so we can use the django-admin
commands.
Inside the simple
directory (where the git repository was cloned) start a new project:
Attention to the .
in the end of the command. It is necessary to not create yet another directory called simple
.
So now the structure should be something like this:
At this point I already complement the project package directory with three extra directories for templates
, static
and locale
.
Both templates
and static
we are going to manage at a project-level and app-level. Those are refer to the global
templates and static files.
The locale
is necessary in case you are using i18n
to translate your application to other languages. So here
is where you are going to store the .mo
and .po
files.
So the structure now should be something like this:
Requirements
Inside the project root (2) I like to create a directory called requirements
with all the .txt
files, breaking down
the project dependencies like this:
base.txt
: Main dependencies, strictly necessary to make the project run. Common to all environmentstests.txt
: Inherits frombase.txt
+ test utilitieslocal.txt
: Inherits fromtests.txt
+ development utilitiesproduction.txt
: Inherits frombase.txt
+ production only dependencies
Note that I do not have a staging.txt
requirements file, that’s because the staging environment is going to use the
production.txt
requirements so we have an exact copy of the production environment.
Now let’s have a look inside each of those requirements file and what are the python libraries that I always use no matter what type of Django project I’m developing.
base.txt
- dj-database-url: This is a very handy Django library to create an one line database connection string which is convenient for storing in
.env
files in a safe way - Django: Django itself
- psycopg2-binary: PostgreSQL is my go-to database when working with Django. So I always have it here for all my environments
- python-decouple: A typed environment variable manager to help protect sensitive data that goes to your
settings.py
module. It also helps with decoupling configuration from source code - pytz: For timezone aware datetime fields
tests.txt
The -r base.txt
inherits all the requirements defined in the base.txt
file
- black: A Python auto-formatter so you don’t have to bother with styling and formatting your code. It let you focus on what really matters while coding and doing code reviews
- coverage: Lib to generate test coverage reports of your project
- factory-boy: A model factory to help you setup complex test cases where the code you are testing rely on multiple models being set in a certain way
- flake8: Checks for code complexity, PEPs, formatting rules, etc
- isort: Auto-formatter for your imports so all imports are organized by blocks (standard library, Django, third-party, first-party, etc)
- tox: I use tox as an interface for CI tools to run all code checks and unit tests
local.txt
The -r tests.txt
inherits all the requirements defined in the base.txt
and tests.txt
file
- django-debug-toolbar: 99% of the time I use it to debug the query count on complex views so you can optimize your database access
- ipython: Improved Python shell. I use it all the time during the development phase to start some implementation or to inspect code
production.txt
The -r base.txt
inherits all the requirements defined in the base.txt
file
- gunicorn: A Python WSGI HTTP server for production used behind a proxy server like Nginx
- sentry-sdk: Error reporting/logging tool to catch exceptions raised in production
Settings
Also following the environments and modes premise I like to setup multiple settings modules. Those are going to serve as the entry point to determine in which mode I’m running the project.
Inside the simple
project package, I create a new directory called settings
and break down the files like this:
Note that I removed the settings.py
that used to live inside the simple/ (3)
directory.
The majority of the code will live inside the base.py
settings module.
Everything that we can set only once in the base.py
and change its value using python-decouple
we should keep in the
base.py
and never repeat/override in the other settings modules.
After the removal of the main settings.py
a nice touch is to modify the manage.py
file to set the
local.py
as the default settings module so we can still run commands like python manage.py runserver
without any
further parameters:
manage.py
Now let’s have a look on each of those settings modules.
base.py
A few comments on the overall base settings file contents:
- The
config()
are from thepython-decouple
library. It is exposing the configuration to an environment variable and retrieving its value accordingly to the expected data type. Read more aboutpython-decouple
on this guide: How to Use Python Decouple - See how configurations like
SECRET_KEY
,DEBUG
andALLOWED_HOSTS
defaults to local/development environment values. That means a new developer won’t need to set a local.env
and provide some initial value to run locally - On the database settings block we are using the
dj_database_url
to translate this one line string to a Python dictionary as Django expects - Note that how on the
MEDIA_ROOT
we are navigating two directories up to create amedia
directory outside the git repository but inside our project workspace (inside the directorysimple/ (1)
). So everything is handy and we won’t be committing test uploads to our repository - In the end of the
base.py
settings I reserve two blocks for third-party Django libraries that I may install, such as Django Rest Framework or Django Crispy Forms. And the first-party settings refer to custom settings that I may create exclusively for our project. Usually I will prefix them with the project name, likeSIMPLE_XXX
local.py
Here is where I will setup Django Debug Toolbar for example. Or set the email backend to display the sent emails on console instead of having to setup a valid email server to work on the project.
All the code that is only relevant for the development process goes here.
You can use it to setup other libs like Django Silk to run profiling without exposing it to production.
tests.py
Here I add configurations that help us run the test cases faster. Sometimes disabling the migrations may not work if you have interdependencies between the apps models so Django may fail to create a database without the migrations.
In some projects it is better to keep the test database after the execution.
production.py
The most important part here on the production settings is to enable all the security settings Django offer. I like to do it that way because you can’t run the development server with most of those configurations on.
The other thing is the Sentry configuration.
Note the simple.__version__
on the release. Next we are going to explore how I usually manage the version of the
project.
Version
I like to reuse Django’s get_version
utility for a simple and PEP 440 complaint version identification.
Inside the project’s __init__.py
module:
You can do something like this:
The only down side of using the get_version
directly from the Django module is that it won’t be able to resolve the
git hash for alpha versions.
A possible solution is making a copy of the django/utils/version.py
file to your project, and then you import it
locally, so it will be able to identify your git repository within the project folder.
But it also depends what kind of versioning you are using for your project. If the version of your project is not really relevant to the end user and you want to keep track of it for internal management like to identify the release on a Sentry issue, you could use a date-based release versioning.
Apps Configuration
A Django app is a Python package that you “install” using the INSTALLED_APPS
in your settings file. An app can live pretty
much anywhere: inside or outside the project package or even in a library that you installed using pip
.
Indeed, your Django apps may be reusable on other projects. But that doesn’t mean it should. Don’t let it destroy your project design or don’t get obsessed over it. Also, it shouldn’t necessarily represent a “part” of your website/web application.
It is perfectly fine for some apps to not have models, or other apps have only views. Some of your modules doesn’t even need to be a Django app at all. I like to see my Django projects as a big Python package and organize it in a way that makes sense, and not try to place everything inside reusable apps.
The general recommendation of the official Django documentation is to place your apps in the project root (alongside
the manage.py file, identified here in this tutorial by the simple/ (2)
folder).
But actually I prefer to create my apps inside the project package (identified in this tutorial by the simple/ (3)
folder). I create a module named apps
and then inside the apps
I create my Django apps. The main reason why is that
it creates a nice namespace for the app. It helps you easily identify that a particular import is part of your
project. Also this namespace helps when creating logging rules to handle events in a different way.
Here is an example of how I do it:
In the example above the folders accounts/
and core/
are Django apps created with the command django-admin startapp
.
Those two apps are also always in my project. The accounts
app is the one that I use the replace the default Django
User
model and also the place where I eventually create password reset, account activation, sign ups, etc.
The core
app I use for general/global implementations. For example to define a model that will be used across most
of the other apps. I try to keep it decoupled from other apps, not importing other apps resources. It usually is a good
place to implement general purpose or reusable views and mixins.
Something to pay attention when using this approach is that you need to change the name
of the apps configuration,
inside the apps.py
file of the Django app:
accounts/apps.py
You should rename it like this, to respect the namespace:
Then on your INSTALLED_APPS
you are going to create a reference to your models like this:
The namespace also helps to organize your INSTALLED_APPS
making your project apps easily recognizable.
App Structure
This is what my app structure looks like:
The first thing I do is create a folder named tests
so I can break down my tests into several files. I always add a
factories.py
to create my model factories using the factory-boy
library.
For both static
and templates
always create first a directory with the same name as the app to avoid name collisions
when Django collect all static files and try to resolve the templates.
The admin.py
may be there or not depending if I’m using the Django Admin contrib app.
Other common modules that you may have is a utils.py
, forms.py
, managers.py
, services.py
etc.
Code style and formatting
Now I’m going to show you the configuration that I use for tools like isort
, black
, flake8
, coverage
and tox
.
Editor Config
The .editorconfig
file is a standard recognized by all major IDEs and code editors. It helps the editor understand
what is the file formatting rules used in the project.
It tells the editor if the project is indented with tabs or spaces. How many spaces/tabs. What’s the max length for a line of code.
I like to use Django’s .editorconfig
file. Here is what it looks like:
.editorconfig
Flake8
Flake8 is a Python library that wraps PyFlakes, pycodestyle and Ned Batchelder’s McCabe script. It is a great toolkit for checking your code base against coding style (PEP8), programming errors (like “library imported but unused” and “Undefined name”) and to check cyclomatic complexity.
To learn more about flake8, check this tutorial I posted a while a go: How to Use Flake8.
setup.cfg
isort
isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections.
To learn more about isort, check this tutorial I posted a while a go: How to Use Python isort Library.
setup.cfg
Pay attention to the known_first_party
, it should be the name of your project so isort can group your project’s
imports.
Black
Black is a life changing library to auto-format your Python applications. There is no way I’m coding with Python nowadays without using Black.
Here is the basic configuration that I use:
pyproject.toml
Conclusions
In this tutorial I described my go-to project setup when working with Django. That’s pretty much how I start all my projects nowadays.
Here is the final project structure for reference:
You can also explore the code on GitHub: django-production-template.