Speed Up Your Website – Django Compressor with S3 and cloudfront

In today’s world where the response time of your website determines how many people are going to stay on your website, you cannot ignore the steps needed to improve your website load time. After all, you want your users to get the content as fast as possible, right? And static components of the website are usually one of the bottlenecks and can be optimized to a great extent. There are many ways to optimize distribution of static content, a few of them are

  1. Minification (in case of js and css)
  2. Gzipping
  3. Merging multiple files into one
  4. Serving through CDN

While top 3 methods (Magnification, gzipping and merging) reduces the total data flow size to the website, a content distribution system reduces the network latency by localizing the data to the nearest node. Serving content directly through cdn also ensures that your web server doesn’t get loaded with requests to serve static data.

In this post, I will be covering ways to achieve all these optimizations and make your website blazing fast. Although this post is for django website, the concept applies to any other framework as well.

My origin server is amazon S3, cdn is cloudfront, django version is 1.8.11. I am also using django-compressor app.

First things first, create an S3 bucket and also configure cloudfront to point to this S3 bucket. Once that is done, you need to setup django to use S3 as your origin server, this section is explained next.

Setup Django to handle S3 storage

In order to get the S3 working with django, I am going to use django-s3-folder-storage app. To install this app, use the following command

pip install django-s3-folder-storage

Now add this app to your installed_apps and configure the following variables in your settings file

INSTALLED_APPS = (
    ...
    's3_folder_storage'
    ...
)

# All my static files resides in a directory 'static_files'                            STATICFILES_DIRS = (
     location('static_files'),
)

AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
AWS_SECRET_ACCESS_KEY = os.environ['AWS_SECRET_ACCESS_KEY']
AWS_STORAGE_BUCKET_NAME = <bucketname>

S3_DOMAIN = s3-ap-southeast-1.amazonaws.com/<bucketname> 
CLOUDFRONT_DOMAIN = 'xyz.cloudfront.net'
DEFAULT_FILE_STORAGE = 's3_folder_storage.s3.DefaultStorage'
STATICFILES_STORAGE = 's3_folder_storage.s3.StaticStorage'
STATIC_S3_PATH = 'static'
DEFAULT_S3_PATH = "media"

MEDIA_ROOT = '/%s/' % DEFAULT_S3_PATH
STATIC_ROOT = "/%s/" % STATIC_S3_PATH
STATIC_URL = '//%s/%s/' % (CLOUDFRONT_DOMAIN, STATIC_S3_PATH)         MEDIA_URL = '//%s/%s/' % (CLOUDFRONT_DOMAIN, DEFAULT_S3_PATH)

ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
AWS_S3_CUSTOM_DOMAIN = CLOUDFRONT_DOMAIN     #important: no "http://"

Once you are done with this setup, just run

./manage.py collectstatic

and check if the files are properly uploading to S3 bucket. Now that we are serving static content from S3 using cloudfront, lets further optimize the distribution of static content.

Setup Gzipping of Static Content

Once the S3 and cloudfront are setup, its time to gzip your content. The advantage of gzipping is to minimize the data transfer required to the website. Its a very minimal setup, just configure the following variables in django settings file

AWS_IS_GZIPPED = True
GZIP_CONTENT_TYPES = (
   'text/css',
   'application/javascript',
   'application/x-javascript',
   'text/javascript',
   'application/vnd.ms-fontobject',
   'application/font-sfnt',
   'application/font-woff',
)

You can modify content types variable according to your requirement. Now run the collect static command, the uploaded content should be gzipped. Now as we have setup the S3, cloudfront and also enabled the gzip compression; lets jump on to the minification and merging of js and css files.

Setup Django-compressor app

Django Compressor is an amazing app which does the minification, compression and merging of css and js files into single file. It even compresses the inline scripts and styles and doesn’t enforce any condition on what should be your static files server. More details can be found on this link. First install this app via pip

pip install django_compressor

Add this app to your installed_apps and configure the following django settings

INSTALLED_APPS = (
 ...
 'compressor'
 ...
)

STATICFILES_FINDERS = (
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'compressor.finders.CompressorFinder',
)

COMPRESS_ENABLED = True
COMPRESS_CSS_FILTERS = ["compressor.filters.cssmin.CSSMinFilter"]
COMPRESS_JS_FILTERS = ["compressor.filters.jsmin.JSMinFilter"]
COMPRESS_URL = STATIC_URL
COMPRESS_ROOT = STATIC_ROOT
COMPRESS_STORAGE = STATICFILES_STORAGE

Now that all the setup is done, any content you want to compress can be done by using compress template tag, an example is shown below

{% load compress %}     # do this at the top of a page
{% compress js %}
    <script src="{% static 'js/common.js' %}"></script>
    <script src="{% static 'js/index.js' %}"></script>
{% endcompress %}

The above code will minify both common.js & index.js, merge them into a single js file. This file will reside in the CACHE directory, this setting can be changed by setting the variable COMPRESS_OUTPUT_DIR. There are other useful settings as well which can be found here.

Let me know if you have any suggestions/feedback in the comments section below.

Fun Fact: Game of thrones season 6 is back, and its episode 4 is also titled as the book of stranger 🙂

2 thoughts on “Speed Up Your Website – Django Compressor with S3 and cloudfront

  1. Pingback: Architecting Scalable and Fast Asset Delivery System – The book of Stranger

  2. Pingback: oakley scalpel

Come on, I know you want to say it