Amazon S3

Amazon S3 provides data object storage through a web service interface.

To provide any feedback on this feature, contact the GOV.UK PaaS team at gov-uk-paas-support@digital.cabinet-office.gov.uk.

Set up the service

Provision an Amazon Web Service S3 bucket

Creating a service instance provisions an Amazon Web Service (AWS) Simple Storage Service (S3) bucket.

The name of the service overall is aws-s3-bucket. There’s one type of plan available for the service named default. This plan has unlimited data, storage, and resources.

S3 buckets can be either private or public:

private S3 buckets are only accessible using apps that you bind to that bucket
files in public S3 buckets can be downloaded by anyone with the URL for those files

AWS encrypts S3 buckets using Amazon S3-managed encryption keys. You do not have to enable this encryption because it is enabled by default.

Provision a private AWS S3 bucket

Run the following in the command line to provision a private AWS S3 bucket:

cf create-service aws-s3-bucket default SERVICE_NAME

where SERVICE_NAME is a unique descriptive name for this S3 bucket.

Provision a public AWS S3 bucket

Public S3 buckets can be useful for tasks such as:

storing static objects that you do not want to include with your app
hosting 404 error pages that work independently of your app

Run the following in the command line to provision a public AWS S3 bucket:

cf create-service aws-s3-bucket default SERVICE_NAME -c '{"public_bucket":true}'

where SERVICE_NAME is a unique descriptive name for this S3 bucket. The configuration option '{"public_bucket":true}' makes sure that the S3 bucket is public.

Bind an AWS S3 bucket to your app

You must bind the S3 bucket to your app so you can get credentials to use the AWS S3 API to read from and write to the bucket.

You can bind an S3 bucket using the command line or the manifest file. You must use the command line when you want to specify permissions for the bucket.

Run the following to bind the S3 bucket to your app:

cf bind-service APP_NAME SERVICE_NAME -c '{"permissions": "PERMISSION"}'

where:

APP_NAME is the name of a deployed instance of your app, as specified in your app’s manifest or push command
SERVICE_NAME is a unique descriptive name for this S3 bucket
PERMISSION is the permission level and can be either read-write or read-only

You can use the app’s manifest to bind an S3 bucket to the app with default read and write permissions only. It will bind automatically when you next deploy your app. An example manifest:

--
applications:
- name: APP_NAME
  services:
  - SERVICE_NAME

By default, you can only connect to an S3 bucket from your GOV.UK PaaS-hosted app.

To connect to an S3 bucket from outside the GOV.UK PaaS, you must create a service key.

Connect to an S3 bucket from your app

Restart your app to populate your VCAP_SERVICES environment variable with the credentials needed to connect to the S3 bucket.

An example VCAP_SERVICES environment variable:

{
 "VCAP_SERVICES": {
  "aws-s3-bucket": [
   {
    "binding_name": null,
    "credentials": {
     "aws_access_key_id": "XXXXXXXXXXXXXXXXXXXX",
     "aws_region": "eu-west-2",
     "aws_secret_access_key": "XXXX/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
     "bucket_name": "paas-s3-broker-prod-lon-XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
     "deploy_env": ""
    },
    "instance_name": "my-bucket",
    "label": "aws-s3-bucket",
    "name": "my-bucket",
    "plan": "default",
    "provider": null,
    "syslog_drain_url": null,
    "tags": [
     "s3"
    ],
    "volume_mounts": []
   }
  ]
 }
}

You must pass credentials from your VCAP_SERVICES environment variable to your AWS software development kit (SDK). Use the appropriate Amazon SDK documentation to do this.

Connect to an S3 bucket from outside of the GOV.UK PaaS

To connect to an S3 bucket from outside of the GOV.UK PaaS, you must create a service key and credentials that allow external access to the bucket.

Create a service key and credentials to allow external access:
```
cf create-service-key SERVICE_NAME SERVICE_KEY -c '{"allow_external_access": true}'
```
where:
- SERVICE_NAME is the unique descriptive name for the S3 bucket
- SERVICE_KEY is a unique descriptive name for the service key

Get the service key credentials:

cf service-key SERVICE_NAME SERVICE_KEY

You will see output similar to the following:

{
 "aws_access_key_id": "AWS_ACCESS_KEY",
 "aws_region": "AWS_REGION",
 "aws_secret_access_key": "AWS_SECRET_ACCESS_KEY",
 "bucket_name": "BUCKET_NAME",
 "deploy_env": ""
}

Use these credentials to connect to the S3 bucket from outside of the GOV.UK PaaS.

Listing and revoking service keys

You can list all service keys for a service by running the following:

cf service-keys SERVICE_NAME

These credentials persist and do not get automatically deleted when you sign out of the GOV.UK PaaS. If you do not need these credentials, you should delete them by running:

cf delete-service-key MY_SERVICE MY_KEY

Use the service

You can take different actions on an S3 bucket using the AWS S3 API. The actions you can take depend on your permission level for that bucket.

Description	Command	Permission
Download an object from S3 bucket	`s3:GetObject`	`read-write`, `read-only`
Upload an object to S3 bucket	`s3:PutObject`	`read-write`
Delete an object from S3 bucket	`s3:DeleteObject`	`read-write`
Get AWS region of S3 bucket	`s3:GetBucketLocation`	`read-write`, `read-only`
List objects in S3 bucket	`s3:ListBucket`	`read-write`, `read-only`
Upload a CORS policy	`s3:PutBucketCORS`	`read-write`
Download a CORS policy	`s3:GetBucketCORS`	`read-write`, `read-only`

Access objects in a public AWS S3 bucket

Once you have uploaded an object to a public S3 bucket, you can find that object at:

https://BUCKET_NAME.s3.amazonaws.com/KEY

where:

BUCKET_NAME is the bucket_name value in the bucket environment variables
KEY is the key you gave to the object when uploading that object to the bucket

Access objects in a private AWS S3 bucket using presigned URLs

You can generate a presigned URL that gives temporary access to an object in a private bucket. Presigned URLs are useful in scenarios where you want to:

grant limited access to a resource in the bucket after validating the user’s authorisation (for example, downloading a signed document)
allow users to upload files directly to the bucket, without passing the file through your application (for example, large file uploads)

You can generate a presigned URL in your application using the AWS SDK package for your application’s programming language. You can also generate them with the AWS CLI tool by using the S3 bucket credentials from a service key.

You need to allow external access to the bucket if you intend to use a presigned URL from outside of the GOV.UK PaaS. For example, if you want to download the generated urls directly from the browser.

Run the following in the command line to bind the S3 bucket to your app and allow external access:

cf bind-service APP_NAME SERVICE_NAME -c '{"allow_external_access": true}'

Remove the service

Unbind an S3 bucket from your app

Run the following in the command line to unbind the S3 bucket from the app:

cf unbind-service APP_NAME SERVICE_NAME

where:

APP_NAME is the name of a deployed instance of your app, as specified in your app’s manifest or push command
SERVICE_NAME is a unique descriptive name for this S3 bucket

If you unbind the S3 bucket from your app but do not delete the bucket, that bucket will persist even after you delete your app. You can re-bind or re-connect to the bucket in future.

Delete an S3 bucket

You must delete all objects from the S3 bucket before you can delete the bucket itself.

You must have read and write permissions for an S3 bucket to delete objects from that bucket.

Use the AWS S3 API to delete objects from the bucket.

Run the following in the command line to delete the S3 bucket:

cf delete-service SERVICE_NAME

where SERVICE_NAME is a unique descriptive name for this S3 bucket.