As a software developer that mainly works with web applications, there’s nothing I find more frustrating than working on a new feature or fixing a bug and not seeing my changes reflected in the web browser because it is using an out-of-date cached version of the JS or CSS files I updated. And even after realizing that I may have already fixed the problem and refreshing the browser, it’s not an option to explain to puzzled customers that doing a full-page refresh or emptying the browser cache will fix the problem. At HumanGeo, we’ve developed a technique to get around this problem by using Nginx to dynamically “smart cache” some of our static files by using the built-in URL regular expressions to control what files are served and cached by browsers. We were already using Nginx to serve our static content, so with just a couple of updates to the Nginx configuration and the Django code, we’re guaranteed to always get the most up-to-date version of specific static files and ensuring that the browser will properly cache the file.

Here’s an example from one of our hosted CSS files:

After doing a basic get request (Note: The requested file was taken from the cache):

After doing a full-page refresh (Note: The requested file got the appropriate status code 304 from the server):

The technique is actually pretty straightforward: using the URL regular expression format that Nginx provides, we look for a pattern within the given URL and find the corresponding file on the file system. So essentially, the browser will request /static/js/application.<pattern>.js and Nginx will actually serve /static/js/application.js. Since the browser is requesting a filename with a particular pattern, it will still cache files with that pattern, but we have much more control over the user experience and what files they see when they hit the page, without requiring them to clear their cache. To further automate the process, the pattern we use is the MD5 hash of the specified file. Using the MD5 and a little help from a Django context processor, we can ensure that the pattern is always up-to-date and that we’re always using the most recent version of the file. However, the pattern could be any unique string you’d like – such as a file version number, application version number, the date the file was last modified, etc.

Details:

Nginx configuration – how to use the URL regex:

# alias for /static/js/application.js
location ~ ^/static/js/application\.(.*)\.js$ {
    alias /srv/www/project/static_files/js/application.js;
    expires 30d;
}# serve remaining static files
location /static/ {
    alias /srv/www/project/static_files/;
    expires 30d;
}

Getting the MD5 for specific files: Django settings.py

A function to generate the MD5 hash:

def md5sum(filename):
    md5 = hashlib.md5()
    with open(filename,'rb') as f:
        for chunk in iter(lambda: f.read(128*md5.block_size), b''):
            md5.update(chunk)
    return md5.hexdigest()

Generate and save the MD5 hash for application.js:

try:
    JS_FILE_PATH = os.path.join(os.path.join(STATIC_ROOT, 'js'), 'application.js')
    JS_MD5 = md5sum(JS_FILE_PATH)
except:
    JS_MD5 = ""

Ensure that a custom template context processor has been added to TEMPLATE_CONTEXT_PROCESSORS:

TEMPLATE_CONTEXT_PROCESSORS = (
    ....
    "project.template_context_processors.settings_context_processor",
)

Make the JS_MD5 variable visible to the templates: template_context_processors.py

template_context_processors.py:

from django.conf import settings
....
def settings_context_processor(request):
    return {
        'JS_MD5': settings.JS_MD5,
    }

Reference the JS_MD5 variable in html templates:

base.html

<script type="text/javascript" src="{{ STATIC_URL }}js/application.{{ JS_MD5 }}.js"></script>

When the HTML templates are loaded, the script tag will now be generated as:

<script type="text/javascript" src="{{ STATIC_URL }}js/application.<MD5 hash>.js"></script>

Summary

With a couple of updates to Django and a few lines of Nginx configuration, it’s easy to create dynamic smart caching of static resources that will work across all browsers. This will also ensure that project stakeholders and all of the application’s users will also have the correct version of the static resource, as well as their browsers correctly caching each resource.