Serving multiple Django applications on the same domain with Nginx and Gunicorn

Plenty of articles explain how to use Nginx and Gunicorn to serve Django applications. The knowledge base quickly dries up, however, if you want to serve up multiple applications using the same domain. With a lot of help from Mitchell Thorson, an awesome developer at USA Today, I managed to configure the Data Team’s production server to do just that.

To get started, I worked through this excellent post on using Gunicorn and Nginx to serve Django applications, then read this follow-up post on serving multiple Django applications. But this second article assumes each application has its own domain. I’m cheap. I’m lazy. I don’t want to buy each app it’s own domain. So let’s see how to wire everything to work harmoniously on one domain and instead give each Django application its own subdomain.

This tutorial assumes you have Nginx installed and that all the files for your Django application are on the server. We’re going to create a virtual environment for each application and then install Django and Gunicorn inside each environment. Per Two Scoops of Django, I prefer to store my virtual environments independently of my Django applications and recommend you do the same. For example, I might store a Django application called hello_world in /usr/local/apps/hello_world while the virtual environment that runs hello_world is saved in /usr/local/envs/hello_world.

You need superuser permissions to run most if not all of the commands below, so if you’re not running as root, you probably will need to prefix each command with sudo.

Step 1: Create your virtual environment
To create virtual environments, you’ll need virtualenv and virtualenvwrapper installed on your server. If you don’t have them installed already, you can run this command:

$ pip install virtualenv virtualenvwrapper

If you have multiple versions of Python installed on your system, you’ll need to set VIRTUALENVWRAPPER_PYTHON to specify which version to use. I did this in my bash profile. Once you do that, navigate to the parent directory you want to use for your virtual environments and create them:

$ cd /usr/local/envs
mkvirtualenv my_env1
mkvirtualenv my_env2

Now let’s move into our first virtual environment, which has Python installed, and add django, gunicorn and any other packages your application needs. We’ll also upgrade pip:

$ workon my_env1
(my_env1) $ pip install pip --upgrade
(my_env1) $ pip install django
(my_env1) $ pip install gunicorn

While we’re here, let’s create a log directory and a log file:
(my_env1) $ mkdir /usr/local/envs/my_env1/logs
touch /usr/local/envs/my_env1/logs/gunicorn_supervisor.log

When you’re done, simply type deactivate to exit the virtual environment, and then repeat the above steps to create as many virtual environments as you need.

Step 2: Create users to run your applications and virtual environments
For security reasons, I recommend you follow Michal Karzynski’s advice to create a system user with limited permissions for each application and virtual environment. It’s a really simple habit to get into. Once you create your users, you’ll want to make those users the owners of your applications and virtual environments. For example:

$ chown -R django_user_1 /usr/local/envs/my_env1
$ chown -R django_user_2 /usr/local/envs/my_env2
$ chown -R django_user_1 /usr/local/apps/hello_world
$ chown -R django_user_2 /usr/local/apps/goodnight_moon

Step 3: Create gunicorn_start files in your virtual environments
Starting with code found in the Gunicorn section of Michal Karzynski’s blog post, you’re going to create a file called gunicorn_start for each virtual environment and save it in the environment’s bin directory. For example: /usr/local/envs/my_env1/bin/gunicorn_start

Now let’s make a few changes to Karzynski’s code:

  1. First, change the settings in lines 3 through 10 to map to your Django application, Django settings and virtual environment. For the SOCKFILE setting, point to the run subdirectory of your environment. For example:
  2. Next, add a VIRTUALENV setting after DJANGO_WSGI_MODULE and set it to point to the root directory for your virtual environment:
  3. Finally, modify what should now be line 17 of your code to use your VIRTUALENV setting:
    source $VIRTUALENV/bin/activate
  4. If you are setting up your app in production, you also might want to change the log-level near the bottom of the code from debug to info once you’re sure everything is working correctly.

Rinse and repeat the above steps to create a gunicorn_start file for each virtual environment.

Step 4: Install supervisor service
Next, if you don’t have it already, install the supervisord service outside of your virtual environments. This service will make sure all of your Django applications are up and running.

$ sudo yum install supervisor

Now we need to create an initialization file for each Django application. This is one of those cases where the directory structure on CentOS did not match Karzynski’s blog post. For example, he says to create a .conf file for each application and place it in /etc/supervisor/conf.d. But on CentOS, I put these files in /etc/supervisord.d and gave them a .ini extension based on two lines of code in /etc/supervisord.conf:

files = supervisord.d/*.ini

Otherwise, your code should look almost identical to what’s in Karzynski’s post:

command = /usr/local/envs/my_env1/bin/gunicorn_start
user = django_user_1
stdout_logfile = /usr/local/envs/my_env1/logs/gunicorn_supervisor.log
redirect_stderr = true

You might need to start the supervisor service:

service supervisord start

When I was figuring out how to set all this up, I found that I often needed to restart services and check their statuses using various commands. I also discovered that even if supervisor says my Django application is running, it might not be if an error in my Python code prevented it from starting. To test your Django app, you can go to the root directory of your virtual environment and enter this command:


Here are some other commands you might find useful. Again, they might be slightly different if you’re not running on CentOS 7:

$ supervisorctl update
$ supervisorctl status hello_world
$ supervisorctl restart goodnight_moon
$ systemctl status supervisord.service -l

Step 5: Create one Nginx initialization file for all our environments
In Karzynski’s post, he tells you to create an Nginx configuration file for each virtual environment, save those files in /etc/nginx/sites-available and create a simlink in /etc/nginx/sites-enabled for each site you want to make accessible.

We’re going to create just one configuration file called sites that will house the configuration settings for all our applications and save it in /etc/nginx/sites-enabled. This costs us a little flexibility: If you want to disable a site, you need to delete or comment out the related code in the sites file rather than just remove a simlink.

You’ll see an upstream server setting at the very top of Karzynski’s Nginx configuration file. You can change the name from hello_app_server to anything you like. Inside the brackets, point to your first application’s gunicorn_sock file. When you’re done, make a copy of this code for each additional application, give each server a unique name and point to that application’s gunicorn_sock file. Make sure all your upstream server settings are at the top of the file. For example:

upstream hello_world_app_server {
  server unix:/usr/local/envs/my_env1/run/gunicorn.sock fail_timeout=0;

upstream goodnight_moon_app_server {
  server unix:/usr/local/envs/my_env2/run/gunicorn.sock fail_timeout=0;

The rest of this file is going to be a single server{} setting, with a location{} setting for each application.

First, near the top of the server{} setting, set server_name to your server’s domain name or IP address. For example:

server {
    listen   80;

    . . .

Next, I decided to create an access log and error log independent of my applications and virtual environments and store them inside an Nginx directory. If you want to do the same, modify these two lines of code to point to where you’d like to store these log files. For example:

server {
    . . .

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    . . .

Finally, create a location setting inside the server setting for each application. For this, you’ll largely mirror the default code, making the following changes:

  1. Set the location to whatever subdomain you want to use for that application. For example, if you want the root URL for your application to be

        location /hello_world {
            . . .

    Note that you do not want a closing forward slash after the subdomain.

  2. Inside the nested if statement inside your location{}, set proxy_pass to point to the name of your upstream server:
        location /hello_world {
            . . .
            if (!-f $request_filename) {
                proxy_pass http://hello_world_app_server;
  3. Repeat the above steps to create a location{} for each of your applications. Remember: All of these locations must be inside the server{} setting.

Static admin files
This is a bit of a segue from the focus of this article, but I want to pass on something I struggled with for hours before I figured it out: If you are using the default admin site for a Django application, you might also want to use the default css, js and other files. This doesn’t just work out of the box.

I ended up creating a location{} inside my Nginx configuration file to point to those static files outside all my applications and virtual environments. You’ll want to verify this directory on your system, but once you do and point to it for admin static files, everything should just work. Here’s the code that worked on my box:

server {
    . . .
    location /static/admin/ {
        alias /usr/local/lib/python3.4/site-packages/django/contrib/admin/static/admin/;
    . . .

Note that in this case, you do use a closing forward slash at the end of the location name.

Step 6: Restart Nginx
After you save your Nginx configuration file, restart Nginx:

$ sudo service nginx restart

If you set up everything correctly, you now should be able to run multiple Django apps on the same domain.

Have fun!

Viewing FEC campaign finance reports in Excel

While I was preparing a series of campaign finance panels for the National Institute for Computer-Assisted Reporting conference in Baltimore this weekend, Chris Keller sent an email to the listserv asking for advice on how to view a campaign finance report submitted to the FEC in Excel.

At USA TODAY, I’ve got a massive FEC database parked on a SQL Server box and fed by a small Army of Python scripts I’ve developed over the last two years. It’s a labor of love that has netted me a ton of co-bylines with my colleague Fredreka Schouten. But as Chris’ post pointed out, not everybody has the time, resources or even the desire, for that matter, to build such a complex system. He just wanted a way to get the data for a single campaign finance report into Excel so he could analyze it.

Anyone who has dealt with electronic filings knows how difficult it can be to work with that data. There are two header rows at the top of each report, followed by data for up to 16 schedules. Each schedule has its own record layout, none of which is included in the data files. And the FEC periodically changes the data headers. (Presently, they’re using v8.1: the fourteenth iteration.) It’s a hassle to work with these reports when you know what you’re doing and next to impossible when you don’t.

I realized in preparing for my panels that I not only had to show other journalists how to find data on the FEC website; I also had to show them how to easily view and analyze that data with the tools they have. After thinking about it, I came up with the idea to build an Excel template. Here’s what you do.

Build your template: (or download v8.1 here)

  1. Download the FEC’s .zip file containing all header information, decompress it and open “vX.X e-filing headers.xlsx,” where X.X is the header version you want. (You can use these instructions to build additional templates for other header versions as well.)
  2. Make sure you’re on the first tab., click cell B1, then press Ctrl-Shift-End to highlight all of your data from B1 to the bottom-right corner.
  3. Copy this data (Ctrl-C).
  4. Create a new worksheet (Ctrl-N).
  5. Click on cell A2 of your new worksheet (we’re leaving a blank row for headers) and Paste the data (Ctrl-V).
  6. We don’t need Column B, so let’s delete it. Click the B heading to highlight that column, right-click the column and select Delete.
  7. Go to A1 and create a generic header, such as Col1 or just 1. Then go to B1 and type Col2, 2 or something similar. Highlight these two cells, grab the fill handle (the little square in the bottom-right corner of the frame around the two highlighted cells) and drag it all the way to the right to make sure every column has a header. As of version 8.1, this requires you to go to cell GX1, generating 206 column headings.
  8. Save your file as an Excel template. You should give it a name that identifies the header version you’re using, such as: Form3Template_v81.xltx

Use your template:

  1. Go to the FEC website, find the report you want and download it in CSV format.
  2. Open your template, then immediately save the file as a regular Excel workbook to make sure you don’t overwrite your template later. Give it a descriptive name that reflects the report you are about to view.
  3. Click on the first blank cell. If you are using the v8.1 headers, this should be A50.
  4. Go to Data, Get External Data, From Text and browse to your .csv file.
  5. Step 1: Select Delimited data, and make sure the “My data has headers” box is unchecked. Click Next.
  6. Step 2: Check the Comma delimiter and make sure all other delimiters are unchecked. Text qualifier should be quotation marks (“). Click Next.
  7. Step 3: Leave the defaults and click Finish. If an Import Data window pops up, Make sure “Existing worksheet” is selected and the cell address is the first blank cell (A50, if you’re using v8.1). To break the data connection between your Excel file and the CSV file (which I recommend), click Properties, uncheck “Save query definition” and click OK twice to close both dialog boxes.
  8. Here comes the fun part: Click a non-empty cell in your data and turn on AutoFilter (Data, Filter). You’ll see our generic columns in Row 1 have dropdown arrows.
  9. Now you can filter your data to see only what you want. To see all receipts (from Schedule A), for example, click the dropdown arrow in A1 and select Text Filters, then Begins With. In the dialog that opens, enter “SA” (without the quotes), then click OK. You are now looking at just your Schedule A data with a nice row of headers in the second row.
  10. If you want to make sure your headers remain visible at all times, click the first cell in Column A containing data (rather than headers), then go to View, Freeze Panes.

I recommend you don’t sort your data while using AutoFilter. If you really want to do that, copy all of your data to a separate worksheet, skipping the generic column headings in Row 1. This will put the headers you really care about in the first row and will make sorting much easier and less likely to scramble your data.

Have fun!