S3 or Simple Storage Service is a type of cloud storage where you access your data over the internet. It is an object-storage service where files are stored as objects rather than in a filesystem structure.
An S3 bucket is a container or a top-level folder that is used to store objects. An object consists of data, metadata and an identifier. It might appear that S3 buckets store files as folders because there is a slash in the file name, but that is not the case.
Storing you site's static files in S3 frees up your server from handling more static requests. And it is also used in case you are choosing the AWS Lambda or other serverless architectures for your site.
In this article, I will assume that you already have an AWS IAM (Identity and Access Management) user setup with a group. This user has AmazonS3FullAccess permission and it also has a Key ID and a secret.
I. Create an S3 Bucket
- Go to the S3 service in the services menu
- Click "Create bucket"
- Add a name for your bucket and click "Next"
- Keep the default options and click "Next"
- Click "Create bucket"
II. Edit the CORS Permissions
CORS (cross-origin resource sharing) allows web apps from different domains to have access to the S3 resources.
To edit it, go to the "Permissions" tab and choose "CORS configuration" and add the rule below and save.
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>*</AllowedHeader>
</CORSRule >
<CORSConfiguration >
AllowedOrigin is used to specify the origin we want to allow cross-domain requests from.
AllowedMethod can be GET, PUT, POST, DELETE or HEAD.
MaxAgeSeconds is the amount of time in seconds that the browser caches the response obtained.
AllowedHeader specifies the headers allowed in the request header.
III. Edit the Bucket Policy
Given that the bucket has Public access enabled, we need to create a policy that would only grant read access to the static files in the bucket. Therefore in the "Permissions" tab under "Bucket Policy", add the following policy:
{
"Version":"2012-10-17",
"Statement":[{
"Sid":"PublicReadGetObject",
"Effect":"Allow",
"Principal": "*",
"Action":["s3:GetObject"],
"Resource":["arn:aws:s3:::my_bucket_name/*"
]
}
]
}
Version is the latest defined policy language version.
Sid is the statement id which identifies the policy statement.
Effect can be Allow or Deny.
Principal used to specify the user.
Action is the action to be allowed or denied.
Resource are the objects that this policy concerns using "arn" or the Amazon Resource Name.
IV. Configure your Django App to use S3
In development environments, Django typically would be configured to use the local filesystem for storing static files. To start using your S3 bucket, install the 2 modules below:
pip install boto3 pip install django-storages
After that you need to add "storages" in your settings files under the installed apps.
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'storages',
'my_app',
]
Define the environment variables below and then use them from your settings file as to avoid pushing code to version control that has sensitive information.
## in your setting.py add:
AWS_STORAGE_BUCKET_NAME = os.getenv('AWS_STORAGE_BUCKET_NAME')
AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_ACCESS_KEY')
AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_KEY_ID')
AWS_S3_CUSTOM_DOMAIN = os.getenv('AWS_S3_CUSTOM_DOMAIN')
STATIC_URL = 'https://my_bucket_name.s3.eu-west-2.amazonaws.com/'
STATICFILES_DIRS = [ os.path.join(BASE_DIR, 'static') ]
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
After that, you can upload your static files into the bucket using the AWS console interface or programmatically by typing the command below in your terminal:
python manage.py collectstatic
To verify that your website is now using S3, you can inspect an image or look in the page's source code.