Table of contents

Technical documentation for GOV.UK PaaS

GOV.UK Platform as a Service (PaaS) is a cloud-hosting platform being built by the Government Digital Service (GDS). GOV.UK PaaS manages the deployment of your apps, services and background tasks so you don’t need to hire people with specialist cloud skills.

GOV.UK PaaS is currently in private beta.

If you’d like to be one of the early applications going live on the platform, please contact us by emailing gov-uk-paas-support@digital.cabinet-office.gov.uk. There is limited availability for these early applications.

Benefits

Right now, development teams spend considerable time and money setting up all the components required to host a government service. With GOV.UK PaaS, development teams can save on this effort by using a hosting stack that’s already been developed.

Your development team can use PaaS to deploy and run government applications written in a range of languages and web frameworks. You’ll simply need to push your source code to the PaaS environment and the code will be compiled and deployed for you.

PaaS makes user management easy by providing development teams with privilege separation options, eg an ‘admin’ user will be able to assign other team members certain permissions from their PaaS console.

PaaS is deployed to multiple availability zones, making it resilient and accredited for information up to OFFICIAL.

The platform itself will be supported 24/7 by GDS, although this does not include application support.

Features of the PaaS platform currently include:

  • PostgreSQL support and will include database back-ups
  • language support through the standard Cloud Foundry buildpacks [external link] except .NET Core
  • limited support for custom buildpacks and Docker images
  • ability to stream application logs to Software as a Service logging platforms

Coding in the open

We are making all new GOV.UK PaaS source code open and reusable. You can use our source code if you want to support a different backing service (any networked attached service that your application consumes to do its job, for example a MongoDB instance or a PostgreSQL database).

Characteristics of GOV.UK PaaS

This table summarises the core characteristics of the PaaS offering.

PaaS characteristic Meaning
Multi-tenant architecture Applications running on the platform are isolated from each other and can’t read or change each other’s code, data or logs (eg the Digital Marketplace application can’t access the data of the GOV.UK publishing platform).
Application development teams manage their own user support A platform where people developing applications also support the application out of hours leads to better software and a better user experience.
Self-service model PaaS makes it easy for development teams to get started, and to make frequent changes to their applications without requiring support from a member of the PaaS team (eg they can create a Postgres instance). Because application teams have this complete control, they won’t experience any unnecessary delays.
Can use multiple public clouds PaaS isn’t locked into a single provider in order to encourage price competition, and to also remove the risk of a single point of failure

Technology

GOV.UK PaaS uses Cloud Foundry and Amazon Web Services (AWS).

Cloud Foundry

The PaaS platform is built using the open source Cloud Foundry project. Multiple open source products were evaluated before Cloud Foundry was selected as the best fit for PaaS.

We chose Cloud Foundry due to its maturity and because it offered:

  • a selection of user permissions
  • support for a wide variety of languages and frameworks through buildpacks
  • a very active community
  • an ability to scale easily, performing well under heavy load

For more information on why we chose Cloud Foundry, read our blog post.

GOV.UK PaaS addresses common usability, security, compliance and scalability concerns experienced by government development teams. This makes the platform a superior option to your development team deploying their own version of the Cloud Foundry software, where they’d need to address cloud security implications and build a continuous integration pipeline.

Amazon Web Services

Currently GOV.UK PaaS runs on Amazon Web Services. As a tenant, you do not have to interact with AWS directly; all interactions required to deploy and manage your service are carried out through Cloud Foundry commands.

The platform may change cloud provider in the future, depending on cost. Your interface to the PaaS will always remain the same, no matter which provider is in use.

Pricing

GOV.UK PaaS will initially operate using a freemium pricing model.

We will select underlying hosting providers with cost in mind. We can change to a more competitively-priced hosting provider without changing how you interface with the PaaS.

Please contact us for pricing details at gov-uk-paas-support@digital.cabinet-office.gov.uk.

Get started

PaaS requirements

To be hosted by GOV.UK PaaS, your application must:

  • follow the twelve-factor application principles (described in more detail below) - this will be the case if your app was written to be deployed to another PaaS like Heroku
  • not require any backing service apart from a database (currently the only services available are PostgreSQL and MySQL)
  • not carry data at SECRET or above (this is currently out of scope for GOV.UK PaaS)
  • be written in one of these languages:
    • Go
    • Nodejs
    • Java
    • PHP
    • Python
    • Ruby
    • or be a static HTML/CSS/Javascript site or compiled binary

Cloud Foundry buildpacks provide runtime and framework support for your application. For most languages, you will need to provide configuration files to describe your app’s dependencies. Note that most buildpacks will support a limited range of versions of the language.

12-factor application principles

These principles were formulated by Adam Wiggins, the cofounder of the Heroku platform. They outline practices for modern apps to follow during development to make them scalable and easy to deploy. Your app must follow these practices to work on the Cloud Foundry technology which is used by GOV.UK PaaS.

We have summarised the practices in the table below, and noted the relevance of each principle to GOV.UK PaaS.

Visit the 12factor.net website to further ensure your application supports these practices.

Principle Meaning Relevance to Cloud Foundry
One codebase many deploys Each of your applications needs its own source-controlled repository. You can push the same codebase to many Cloud Foundry applications with these commands:
$ cf push myapp-staging
$ cf push myapp-production
Isolate dependencies All required dependencies (eg a database or image library) must be vendored into your software system. If you don’t declare your dependencies, you probably won’t be able to deploy your application on Cloud Foundry. How you specify dependencies depends on which language and buildpack you use. For example, the python buildpack expects you to provide a requirements.txt file which it will pass to pip.
Store your configuration in the environment Ensure you separate the storage of your code and configuration (this is anything that may vary between environments, such as passwords). Cloud Foundry provides environment variables to tell an application how to configure itself, e.g. VCAP_SERVICES tells applications what services are available and how to connect to them. You can create your own environment variables for each application.
Backing Services All your backing services (e.g. an email service or a monitoring system) should be loosely coupled to your code so you can easily change services if you need to. A change in service should not require a code change. In Cloud Foundry, backing services are referred to as ‘services’. Users can create services, bind them to applications, and delete them.
Strictly separate build, release and run stages There should be a strict separation between building, releasing and running the code. The build stage is carried out by the buildpack and Cloud Foundry refers to this as ‘staging’ an app. Cloud Foundry combines the release and run stage into a single ‘push’ stage.
Stateless processes Build stateless applications where intermediate data is stored in your backing services (eg your databases) and not in your running code. Also, have you applications run on many servers so you can ensure continuity in the event of server downtime. Cloud Foundry treats all application processes are stateless and expendable.
Port binding Your application should interface to the world via an API. You can have a separate URL for your customers than the one you use for internal calls. This means your app can be used as a backing service for another app via its URL. Cloud Foundry expects applications to have this and provides an environment variable called PORT for the application to use as part of its bootstrapping process.
Concurrency Ensure all your processes (e.g. web requests or API calls) are running separately so your application can scale easily. Cloud Foundry expects applications to behave according to this principle. To increase the number of processes running, use the cf scale command.
Disposability You should be able to rapidly release new code. Also, applications should be able to start back up fast and cleanly following shut down. Cloud Foundry expects applications to follow this principle.
Development parity Applications should be rapidly deployed from their development environment to production. To ensure this rapid deployment, keep a developer’s environment similar to that of production (eg both environments should use the same backing services). CF helps you achieve this development/production parity by letting you create and deploy similar services for development, test and production environments
Logs Maintain and archive log files so you have visibility of how your application works over time. Cloud Foundry describes how to log files here [external link]. GOV.UK PaaS will offer the Loggregator function in beta.
Administrative processes Run one-off administrative processes (eg running analytics) in your production environment.

Quick setup guide

Getting an account

GOV.UK PaaS is currently in private beta.

If your organisation is already taking part in the private beta and you need an account, please talk to your manager who will need to authorise creating your account.

If you’d like to find out about taking part in the beta, please contact us by emailing gov-uk-paas-support@digital.cabinet-office.gov.uk.

In order to provide you with an account, we need to store some personal data about you. Please see our Privacy Policy for details.

Setting up the command line

GOV.UK PaaS uses a hosting technology called Cloud Foundry. As a tenant (that is, someone who is hosting an application on the PaaS), you will use the Cloud Foundry command line client to interact with the PaaS. To set it up:

  1. Download and install the Cloud Foundry CLI for your platform [external page, opens in new tab]

    Note: On macOS Sierra, installing with Homebrew does not work. We recommend using the Mac binary or installer.

  2. To check that it installed correctly, go to the Terminal/command line/Command Prompt and run:

    cf -v
    

    You should get a message like this, confirming the version that’s installed:

    cf version X.X.X...
    

    Note: depending on your network configuration you might need to set an HTTP_PROXY environment variable [external link] for the CLI to connect. Contact your network administrators to work out the correct settings for your configuration.

  3. Log in by running:

    cf login -a api.cloud.service.gov.uk -u USERNAME
    

    Your USERNAME is your email address your account was created with.

    You will then be prompted to enter your password. You set this when you clicked on the invite link in your welcome email.

Once logged in, you can see the available commands by running cf.

Deploying a test app

To practice deploying an app, try following the process to deploy a static site.

To clean up look at our instructions on stopping and deleting apps.

Choosing passwords

When setting a password for your GOV.UK PaaS account, it is important to choose one that is both unique and resistant to attack.

We reserve the right to suspend your account if we detect that it has an insecure password.

Follow these rules when setting your password:

  • Your password for the PaaS should be unique. You should not have used it as a password anywhere else.
  • Your password must be longer than eight characters, although it may not be longer than 255 characters.
  • Your password should not be common (such as 12345678 or qwertyuiop), or a single dictionary word (such as football or welcome).
  • If you must use a memorable password, use three words in a short sentence.
  • We recommend that you generate and store your passwords with password manager software, like Lastpass, 1Password, or KeePass.

Your session will expire 7 days after logging in using the cf command line tool, and you will be prompted to log in again.

System status, alerts and updates

Platform availability

GOV.UK PaaS has a system status page showing the availability of live applications and database connectivity.

We use this system status service to let tenants know about problems with the platform, and so we recommend that you sign up to receive alerts and updates - particularly if you are using the platform to host production applications.

Platform updates

You can find out about changes, fixes and upgrades, and hear about new features, by joining our announcement mailing list. If you have trouble signing up, please contact gov-uk-paas-support@digital.cabinet-office.gov.uk and we will add you manually.

Limitations

While GOV.UK PaaS is built using Cloud Foundry technology, we don’t support all Cloud Foundry features. This section explains some Cloud Foundry features that are not enabled, as well as some limitations of the beta phase.

.NET is not supported with standard buildpacks

Cloud Foundry uses buildpacks to provide runtime and framework support for applications in different languages.

GOV.UK PaaS supports both standard buildpacks [external link] and custom buildpacks [external link]. Please note that the .NET Core buildpack [external link] is only available as a custom buildpack.

404s after commands that restart the app

After you use a command that restarts application instances, such as cf push or cf restart, your app may briefly return incorrect 404 errors. Apart from the brief downtime, this may lead to problems if the 404 is cached, or visiting web crawling bots (as used by search engines) receive a 404.

Commands known to do this are:

  • cf push
  • cf restage
  • cf restart
  • cf scale when changing disk or memory limits or forcing a restart

We are working on a fix to prevent this happening.

In the meantime, we suggest that you use a blue-green deployment process [external link], where you have two versions of an app, one that is ‘live’ and one that is undergoing an update or restart. There are plugins for the Cloud Foundry command line client to facilitate this process. We can recommend the cf-blue-green-deploy plugin.

API access may have brief outages during beta

During the beta period, there may be occasional brief periods where API access is unavailable during a platform update, causing commands sent from the command line client to fail.

If you find that a valid command is failing and the error message does not explain the problem, please wait 5 minutes before trying the command again. If the error persists for more than 5 minutes, it is unlikely to be caused by a platform update and you should contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk.

We are working on a fix to prevent the interruption of API access when we update the platform.

Privacy policy

This privacy policy relates to the data we store about you as a GOV.UK PaaS tenant. It does not relate to the privacy of the end users of any service you host on the PaaS: you are responsible for the data stored by your service.

If this privacy policy changes, all PaaS tenants will be notified.

Introduction

GOV.UK Platform as a Service (GOV.UK PaaS) is provided by the Government Digital Service (GDS), which is part of the Cabinet Office. We use your personal data to provide you with access to PaaS systems, and to administer that access.

This privacy policy explains the kinds of data that we process, how it’s used, how it’s stored, how it’s protected and how you can find out about it.

The data we process

We collect and process some data about you to provide you with access to PaaS systems, as part of your work on HMG services and products that utilise the PaaS platform.

This data includes:

  • your name - so we know who is using the account
  • your email address - which allows us to contact you about your account
  • your mobile telephone number - which allows us to contact you about your account, and in future, support multi-factor authentication
  • your organisational role - which allows us to understand what you do, and what access you should have

How we use and share your data

We use this information to create and administer user accounts with our platform.

We don’t share this information with anyone else.

We will store the data you provide in order for your user account with PaaS to function correctly, and so we can administer that access - including sending you updates and notices.

We will delete your personal data when your user account is no longer required. We keep records of the user account activity for up to an additional 2 years.

Where we store your data

We store your data on our servers in the European Union. We’ll never transfer or your data on servers outside of the European Economic Area.

This means you are covered by EU Data Protection regulations.

Keeping your data secure

We are committed to protecting the information you provide. We ensure that appropriate steps are taken to prevent the disclosure, theft or unauthorised modification of your information. We understand how important it is to protect personal data and other sensitive information. This includes encrypting the data, as well as ensuring that data is sent via encrypted channels (such as Transport Layer Security). We have procedures and security features in place to try and keep your data secure once we receive it.

Because it’s impossible to guarantee data security, any data you transmit is at your own risk.

We won’t share your information with any other organisations for marketing, market research or commercial purposes, and we will only share your information with other organisations to make it possible to facilitate and administer platform functionality.

Disclosing your information

We will usually only share your information as set out above.

Additionally, we may pass on your data if we have a lawful reason to do so, for example as part of a criminal investigation or fraud prevention activity.

Your rights and how to make a complaint

You can find out what information we hold about you, and ask us not to use any of the information we collect.

You can email us on gov-uk-paas-support@digital.cabinet-office.gov.uk. You’ll get a confirmation within 5 working days that we have received your complaint, and a full answer within 20 working days. We will inform you if there’s going to be a delay.

If you’re unhappy with the answer you receive, or need any advice regarding the use of your personal data, you can contact the Information Commissioner’s Office (ICO), https://ico.org.uk/.

Deploying apps

Deployment overview

The cf push command is used both to create a new app and to push a new version of an existing one. The basic steps:

  1. Put the code you want to deploy in a directory. This is usually accomplished by checking it out of version control.

  2. Target the appropriate organisation and space.

    cf target -o SOMEORG -s SOMESPACE
    
  3. Deploy the application by running:

    cf push APPNAME
    

    from the directory which contains the code and configuration files for your app.

The app should now be live at https://APPNAME.cloudapps.digital.

There are many options available when you push an app. You can optionally set them in a manifest.yml file in the directory from which you are running the push command. See the Cloud Foundry documentation on Deploying with Application Manifests [external link] for details.

For a production app, you should run at least two instances to ensure availability.

After deployment, you can increase the running instances to two using:

cf scale APPNAME -i 2

Caveats

  • Your app should not write to local storage. Cloud Foundry local storage is ephemeral and can be deleted at any time.
  • You may need to set environment variables for your app to work. All configuration information should be stored in environment variables, not in the code.
  • Instances will be restarted if they exceed memory limits.
  • Your application should write all its log messages to STDOUT/STDERR, rather than a log file.

Buildpacks

About Buildpacks

Cloud Foundry uses buildpacks to provide runtime and framework support for applications in different languages (for example ensuring your app code has both the Ruby runtime and the Rails framework available to help it run). GOV.UK PaaS supports both standard and custom buildpacks:

  • standard buildpacks [external link] are buildpacks for common languages and frameworks that are supported by Cloud Foundry
  • custom buildpacks are developed by the wider community to enable hosting of applications in additional languages or frameworks
  • docker images are a packaging format, and the requirements for the app and its runtime environment are the same as for apps deployed using buildpacks

Which Buildpack should I use?

We recommend using the standard buildpacks to maximise the support you will receive from GOV.UK PaaS. Using a custom buildpack or docker image will mean additional setup and maintenance costs.

Refer to the Guidance section for further information on the different options available.

How to use Custom Buildpacks

There are many application attribute options available when you push an app. You can use the buildpack attribute to specify a custom buildpack for your app through:

  • specifying the buildpack in the manifest.yml file
  • using the command line

Specifying buildpacks in the Manifest File

You can set application attribute options in a manifest.yml file in the directory from which you are running the push command. This is done in one of three ways:

  1. By name: MY-BUILDPACK.
  2. By GitHub URL: https://github.com/cloudfoundry/java-buildpack.git.
  3. By GitHub URL with a branch or tag: https://github.com/cloudfoundry/java-buildpack.git#v3.3.0 for the v3.3.0 tag.
---
  ...
  buildpack: buildpack_URL

Command line options override the manifest; the option that overrides the custom buildpack attribute is -b.

Using the Command Line to choose buildpacks

Once a custom buildpack has been created, it can be pushed to either a public or private git repository. The repository URL can then be included in the command line to push your app.

If the repository is private, the push command must include https and username/password authentication

  • Public: $ cf push my-new-app -b git://github.com/johndoe/my-buildpack.git
  • Private: $ cf push my-new-app -b https://username:password@github.com/johndoe/my-buildpack.git

The app will then be deployed to Cloud Foundry, and the buildpack will be cloned from the repository and applied to the app.

If a buildpack is specified using cf push -b the detect step will be skipped and as a result, no buildpack detect scripts will be run.

Excluding files

Cloud Foundry isn’t version-control-aware, so cf push will deploy the working state of whatever files you have in that directory. In most cases, you will want to exclude files [external link] ignored by Git. From within your project directory, run

ln -s .gitignore .cfignore

and commit the .cfignore to your repository. However, read on if you have a more advanced CF setup.

A couple of important points on the .cfignore:

  1. if you have a more advanced app setup and have apps with a path other than the project root (where you run cf push from), you will need an additional .cfignore file located in each app path;

  2. also note that more advanced .gitignore syntax, such as the ** recursive subdirectory wildcard, are not supported by .cfignore.

Environment variables

All the configuration information for your app must be stored as environment variables, not in the code.

This includes credentials for external services that your app uses, such as a Twitter account, as well as values that will vary with each deployment of the app, like the canonical URL.

To view an app’s current environment variables, run:

cf env APPNAME

To create or update a variable, use:

cf set-env APPNAME ENV_VAR_NAME ENV_VAR_VALUE

If you’re deploying a pre-existing app that was written with 12-factor in mind, check the app’s documentation for any environment variables you need to set.

If the app has instructions to deploy to Heroku, and tells you to do something like:

heroku config:set VARIABLE=value

then you should do the equivalent command with cf set-env:

cf set-env APPNAME VARIABLE value

System-provided environment variables

As well as environment variables you set yourself, there are a number of system-provided variables which give you information about configuration details handled by the PaaS: the port on which the application is listening, the maximum memory each instance can use, the external IP address of the instance, and so on.

Do not attempt to change the values of these system-provided variables with the CLI or your app’s code.

For a full list, see Cloud Foundry’s Cloud Foundry Environment Variables [external link] documentation.

Two important variables for initial setup are:

  • VCAP_SERVICES contains details (including credentials) of any backing services bound to the app
  • VCAP_APPLICATION provides details of the currently running application (for example, language runtime version)

To see the values of the system-provided variables, use:

cf env APPNAME

If your app connects to a backing service, you may need to have it parse VCAP_SERVICES to get the credentials and other settings relating to that service and set the appropriate environment variables.

However, some buildpacks will do this for you automatically. See the deploy instructions for the language/framework you are using for details.

Organisations, spaces & targets

Organisations

Your tenant account belongs to at least one organisation (“org” for short) within the PaaS. This typically represents the real-world organisation, department or team you work for. Your co-workers’ tenant accounts will belong to the same org.

Typically a team has a single organisation, but you can have more than one.

To list the orgs your account can access, run:

cf orgs

To see details about an org, run:

cf org ORGNAME

where ORGNAME is the name of the org.

Spaces

Each organisation is divided into one or more spaces, which are used to organise app development, deployment, and maintenance. For example, you might have spaces for development and production versions of your app.

Diagram showing that an organisation contains multiple spaces

Different accounts can have different permissions to access each space, which are granted through user roles. For example, you may choose to give a junior developer access to your development space, but not to production. To change which accounts have access, your account needs to have the Org Manager role. See the section on Managing users for more about roles.

When we set up your organisation, we create a default sandbox space you can use for experimenting with the PaaS.

To see the spaces you can access in your current org, run:

cf spaces

Setting a target

To deploy an app, you need to specify a combination of an organisation and a space: this is called the target.

Set the target with:

cf target -o ORGNAME -s SPACENAME

Once you set the target, the Cloud Foundry client remembers it until you change it.

You can change space without changing org using:

cf target -s SPACENAME

Managing spaces

You can create new spaces within an org if your account has the Org Manager role. To find out who has that role, run:

cf org-users ORGNAME

where ORGNAME is the name of the org. You will see a list of users and their roles.

If your account does not have the Org Manager role and you need to create a new space, you should ask a co-worker who is an Org Manager for your organisation.

As an Org Manager, you can use:

cf create-space SPACENAME -o ORGNAME

to create a new space. You will then need to grant access to any tenant accounts who should be able to use that space.

Granting access

You grant access to a space by assigning a role to an account. To assign roles, you need to be an Org Manager or Space Manager.

In most cases, the role you will assign is Space Developer, which gives the account an ability to deploy and manage apps in the specified space.

To grant access to a space to another account, run:

cf set-space-role USERNAME ORGNAME SPACENAME ROLE

The USERNAME is the email address the user logs in with when using the command line client.

For example, say you had just created a space called test within your org called acme, and you wanted your co-worker ana@example.com to be able to use that space as a developer, you would run:

cf set-space-role ana@example.com acme test SpaceDeveloper

Learning more

See the Cloud Foundry documentation on Orgs, Spaces, Roles, and Permissions [external link] for more details.

Names, routes and domains

App names and cloudapps.digital hostname clash

When you push an app, you have to assign an app name, either in the manifest or using the command line. The app name is also used as the default hostname of the cloudapps.digital URL where the app will be hosted.

For example, if you push an app called myapp, the PaaS tries to host it at myapp.cloudapps.digital. This is done using Cloud Foundry’s route functions.

The cloudapps.digital domain is shared across all PaaS tenants. If another tenant is using myapp.cloudapps.digital, and you try to push an app called myapp, you will receive an error like this:

Using route myapp.cloudapps.digital
Binding myapp.cloudapps.digital to myapp...
FAILED
The route myapp.cloudapps.digital is already in use.

This will also happen if there is an app with the same name within your own organisation, but in a different space.

There are a few possible solutions to this problem:

  1. Minimise the chance of name clash by the way you name your apps; incorporate the name of your team, department or service to produce names that are unlikely to clash with other PaaS tenants. For example, if you work for the Ministry of Peace, instead of myapp, you could use minipeace-myapp.

    If you run different versions of an app in different spaces, you will also need to name them differently: for example, you might have minipeace-myapp-dev, minipeace-myapp-test, minipeace-myapp-production and so on.

  2. Manually specify a hostname that is different from the app name. You can do this as a command line option when you push:

    cf push myapp -n HOSTNAME

    or with a line in the app’s manifest.yml file:

    ---
        ...
        host: HOSTNAME
    

    You will still need to pick a hostname that is not in use.

    This solution means you can change the hostname while keeping the app name the same. This is more flexible, but means you need to keep track of both a hostname and an app name for each app. It could potentially make it easier to deploy to the wrong target.

  3. Use the random-route option. This appends a couple of random words to the hostname, to avoid clashes. For example, if you run:

    cf push myapp --random-route

    the app will be hosted at something like https://myapp-mummifying-giraffe.cloudapps.digital.

    You can also specify this in your app’s manifest.yml file:

    random-route: true

    This is a convenient way to avoid clashes automatically. The disadvantage is that if your end users can see the resulting URLs, they may find the random words strange.

Custom domains

In production, you will probably want your app to be available through your own url (for example, yourapp.service.gov.uk). Go to using a custom domain to see how to set this up.

Deploy a static site

This section explains how to create and deploy a static HTML page. It’s worth testing that you can carry out this process before you try to deploy a more complex app.

When you deploy an app, you must select a combination of an organisation and a space (see Orgs, Spaces and Targets for more information). This is called the target.

We have provided a sandbox space in your organisation for you to use for learning about the PaaS. You may want to target the sandbox while you are testing by running:

cf target -s sandbox

It’s also important to realise that if you deploy an app using the same name and target as an existing app, the original will be replaced. If you are not sure about where to deploy your app, consult the rest of your team.

These steps assume you have already carried out the setup process explained in the Quick Setup Guide section.

  1. In an empty directory, create an index.html file.

  2. Add some markup to the index.html file:

    <html>
      <head>
        <title>Static Site</title>
      </head>
      <body>
        <p>Welcome to the static site!</p>
      </body>
    </html>
    
  3. Create a manifest.yml file in the same directory. The manifest file tells Cloud Foundry what to do with your app.

    ---
    applications:
    - name: my-static-site
      memory: 64M
      buildpack: staticfile_buildpack
    

    Replace my-static-site with a unique name for your app. (You can use cf apps to see apps which already exist).

    The memory line tells the PaaS how much memory to allocate to the app.

    A buildpack provides any framework and runtime support required by an app. If your app was written in Ruby, you would use ruby_buildpack. In this case, we just want to serve a static file, so we use staticfile_buildpack.

  4. From the directory where you created the files, run:

    cf push

    If you do not specify an app name with the push command, the name specified in the manifest file is used.

The site should now be live at https://APPNAME.cloudapps.digital.

Note: The http:// version of the URL will not work. You must enter https://.

Adding more instances

For a production service, you should run at least two instances of the app to ensure availability.

You can add another instance of this static app by running:

cf scale APPNAME -i 2

or by adding this to the manifest and pushing again:

---
  ...
  instances: 2

Deploy a Ruby on Rails app

This section explains minimal steps for deploying a basic Rails app. For full details of how to deploy Ruby on Rails apps, see the official Cloud Foundry guide Getting Started Deploying Ruby on Rails Apps [external link].

These steps assume you have already carried out the setup process explained in the Quick Setup Guide section.

When you deploy an app, you must select a combination of an organisation and a space (see Orgs and spaces for more information). This is called the target.

We have provided a sandbox space for you to use for learning about the PaaS. You may want to target the sandbox while you are testing by running:

cf target -s sandbox

It’s also important to realise that if you deploy an app using the same name and target as an existing app, the original will be replaced. If you are not sure about where to deploy your app, consult the rest of your team.

Deploying a non-database app

This is how to deploy a Rails app that doesn’t require a database.

  1. Put the code for your Rails app into a local directory (for example, by checking it out of version control).

  2. Exclude files ignored by Git.

  3. If you’re using Rails 4, add the rails_12factor gem for better logging. Rails 5 has this functionality built in by default.

  4. Create a manifest.yml file in the folder where you checked out your app.

    ---
    applications:
    - name: my-rails-app
      memory: 256M
      buildpack: ruby_buildpack
    

    Replace my-rails-app with a unique name for your app. (You can use cf apps to see apps which already exist).

    The memory line tells the PaaS how much memory to allocate to the app.

    A buildpack provides any framework and runtime support required by an app. In this case, because the app is written in Ruby, you use the ruby_buildpack.

  5. Upload and start the application by running:

    cf push APPNAME
    

    from the directory which contains all the code and configuration files for your app.

    If you do not specify a name for the app after the cf push command, the name from the manifest file is used.

  6. Set any additional environment variables required by your app. For example:

    cf set-env APPNAME VARIABLE `value`
    

    where VARIABLE is a unique name for the variable, and value is the value to set.

Your app should now be live at https://APPNAME.cloudapps.digital!

For a production service, you should run at least two instances of the app to ensure availability.

You can add another instance of your app by running:

cf scale APPNAME -i 2

Deploying with a PostgreSQL database

Note that the only database service currently supported by PaaS is PostgreSQL. If your Rails app requires a database, it must be able to work with PostgreSQL.

The Cloud Foundry buildpack for Ruby automatically gets the details of the first available PostgreSQL service from the VCAP_SERVICES environment variable and sets the Ruby DATABASE_URL environment variable accordingly. Ensure that your app is configured to use DATABASE_URL to set its database configuration when deployed to the PaaS.

  1. Put the code for your Rails app into a local directory (for example, by checking it out of version control).

  2. If you are using Git, you may wish to exclude files ignored by Git.

  3. If you’re using Rails 4, add the rails_12factor gem for better logging. Rails 5 has this functionality built in by default.

  4. Create a manifest.yml file in the directory where you checked out your app.

    ---
    applications:
    - name: my-rails-app
      memory: 256M
      buildpack: ruby_buildpack
    

    Replace my-rails-app with a unique name for your app. (You can use cf apps to see apps which already exist).

    The memory line tells the PaaS how much memory to allocate to the app.

    A buildpack provides any framework and runtime support required by an app. In this case, because the app is written in Ruby, you use the ruby_buildpack.

  5. Upload the app without starting it using this command:

    cf push --no-start APPNAME
    

    from the directory which contains all the code and configuration files for your app.

    If you do not specify a name for the app after the cf push command, the name from the manifest file is used.

  6. Set any additional environment variables required by your app. For example:

    cf set-env APPNAME VARIABLE `value`
    

    where VARIABLE is a unique name for the variable, and value is the value to set.

  7. Create a PostgreSQL backing service (if required) and bind it to your app.

    To enable Rails support for database migrations, you may wish to create a Procfile in the same directory as your manifest.yml and Gemfile. The Procfile is a way to specify commands to be run when deploying your app.

    This is a minimal example of the Procfile content for Rails 5.0:

    web: rake db:migrate && bin/rails server
    
  8. Start your app by running:

    cf start APPNAME
    

Your app should now be live at https://APPNAME.cloudapps.digital!

For a production service, you should run at least two instances of the app to ensure availability.

You can add another instance of your app by running:

cf scale APPNAME -i 2

Web servers

By default, the Cloud Foundry Ruby buildpack runs bin/rails server to spin up the application. In Rails 4 and below, this will use WEBrick as the web server. In Rails 5 and above, the default is puma.

You may want to use a different web server in production. See the Cloud Foundry docs for more information on configuring a production server [external link].

Troubleshooting asset precompilation

By default, the Rails buildpack performs asset precompilation during the staging phase. This is fine for most Rails apps, but it won’t work for those which need to connect to services (such as the database) during asset compilation. (For an example, see MyUSA issue #636)

There are multiple potential solutions for this. For more advice, see the Cloud Foundry document on the subject [external link].

Deploy a Node.js app

This section covers how to deploy a basic Node.js application to GOV.UK PaaS. See the Cloud Foundry Tips for Node.js Applications [external link] for more details.

Note that the only database service currently supported by PaaS is PostgreSQL. If your Node.js app requires a database, it must be able to work with PostgreSQL.

These instructions assume you have already carried out the setup process explained in the Quick Setup Guide section.

This is the code for the example app we are going to use. It is a basic web server that responds with a ‘Hello World’ message.

const http = require('http');

const port = process.env.PORT || 3000;

const server = http.createServer((req, res) => {
  res.statusCode = 200;
  res.setHeader('Content-Type', 'text/plain');
  res.end('Hello World\n');
});

server.listen(port, () => {
  console.log(`Server running on ${port}/`);
});
  1. Save the code to a new local directory as example.js.

  2. Add this manifest.yml file to the same directory:

    ---
    applications:
    - name: my-node-app
      command: node example.js
      memory: 256M
      buildpack: nodejs_buildpack
    

    Replace my-node-app with a unique name for your app. (You can use cf apps to see apps which already exist).

    The memory line tells the PaaS how much memory to allocate to the app.

    A buildpack provides any framework and runtime support required by an app. In this case, because the app is written in Node.js, you use the nodejs_buildpack.

  3. Include an npm package.json file to specify dependencies. The file should also specify a start command used to launch the app.

    This is a package.json file for our example app:

    {
      "name": "example",
      "version": "0.0.1",
      "author": "Demo",
      "engines": {
        "node": "6.11.x"
      }
    }
    

    The "engines" values specify the versions of Node.js and npm that the PaaS should use to run your app. Note that older versions may not be available: if your version is not supported, you will see an error message when you try to upload and start the app.

  4. You can optionally run npm install to preinstall dependencies rather than having them added during the PaaS staging process.

  5. Run cf push APPNAME from the top level of the directory which contains all the code and configuration files.

If you want to upload the app without starting it (for example, if you need to create a PostgreSQL service), run cf push --no-start APPNAME, then when you are ready to start the app, run cf start APPNAME.

See Tips for Node.js Applications [external link] in the Cloud Foundry documentation for more information.

PostgreSQL setup with Node.js

If your app depends on a backing service such as PostgreSQL, it will need to parse the VCAP_SERVICES environment variable to get required details, such as service URLs and credentials.

You must create the service and bind it to your Node.js app as described in the Deploy a backing service section.

You can use the cfenv module to assist with parsing the environment variables.

In your package.json file, you would specify cfenv as a dependency:

    {
      // ...
      "dependencies": {
        "cfenv": "*",
        // ...
      }
    }

Then in your app, you can easily get configuration information for backing services. This is an example of how to connect to a PostgreSQL service.

    var cfenv = require("cfenv");
    var pg = require('pg');
    var appEnv = cfenv.getAppEnv();
    var connectionString = appEnv.getServiceURL(/.*/);
    var client = new pg.Client(connectionString);
    client.ssl = true;
    client.connect();

Note that in the above you should replace “my-postgres” with the exact name of the PostgreSQL service you created. The getServiceURL function returns a connection string which includes the username and password required to connect to the database.

You should also remember to include dependencies for any service bindings in package.json.

```
{
  // ...
  "dependencies": {
    "pg": "*",
    // ...
  }
}
```

Deploy a Django app

This section explains how to deploy an app using the Django framework. You may also need to refer to the Cloud Foundry documentation about the Python buildpack [external link].

Note that the only database service currently supported by PaaS is PostgreSQL. If your Django app requires a database, it must be able to work with PostgreSQL.

These steps assume you have already carried out the setup process explained in the Quick Setup Guide section.

If you are just getting started learning CloudFoundry, you can use the sandbox space by running: cf target -s sandbox

  1. Put the code for your Django app into a local directory (for example, by checking it out of version control).

  2. If you are using Git, add *.pyc and local_settings.py to your .gitignore, file, then exclude files ignored by Git so Cloud Foundry will ignore them too.

  3. Tell Cloud Foundry which Python runtime to use by creating a runtime.txt file in the root of the local folder. The contents of the file should
    be:

    python-3.5.1  
    

    replacing “3.5.1” with the version of Python you want to use (it must be supported by the buildpack: currently versions 2.7.11 to 3.5.2 are supported.)

  4. Make sure you have all the required modules for your project installed into your virtual environment (including Django).

  5. Generate a requirements.txt file if your project doesn’t already have one by running pip freeze > requirements.txt in the root of the local folder. Add the following lines to the requirements.txt file.

    whitenoise==1.0.6  #manages static assets
    waitress==0.8.9 #a pure python WSGI server that is a replacement for gunicorn
    
  6. Edit your wsgi.py file.

    When you create a Django project, a default wsgi.py file should be created for you in the project folder. Excluding the opening comments, the default wsgi.py looks like this:

    import os
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "PROJECTNAME.settings")
    
    from django.core.wsgi import get_wsgi_application
    application = get_wsgi_application()
    

    You’ll need to add a few lines to import the whitenoise package and wrap the middleware around the WSGI application so that all static files are served using whitenoise. Edit your wsgi.py to:

    import os
    from django.core.wsgi import get_wsgi_application
    
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "PROJECTNAME.settings")
    # important that the whitenoise import is after the line above
    from whitenoise.django import DjangoWhiteNoise
    
    application = get_wsgi_application()
    application = DjangoWhiteNoise(application)
    

    The order here is important. The DJANGO_SETTINGS_MODULE environment variable must be set before importing DjangoWhiteNoise.

  7. You should now tell Django where to look for static files. In settings.py within the project folder, add these lines below the import os statement.

    PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))
    
    STATIC_ROOT = os.path.join(PROJECT_ROOT, 'static')
    STATIC_URL = '/static/'
    

    In this case, the STATICROOT variable tells Django to look for static files in a directory called static inside the project folder, and the STATICURL variable sets the path where those files will be served.

    You may need to alter these values depending on how static files are handled in your app.

    If your static files are located across multiple folders, you may need to use the STATICFILES_DIRS variable. See the Django documentation for full details on managing static files.

  8. Create a file called Procfile in the root of your local folder, and put in it:

        web: python manage.py migrate && waitress-serve --port=$PORT PROJECTNAME.wsgi:application
    

    The Procfile is a way to specify commands to be run when deploying your app; in this case, for database migration.

    PROJECTNAME should be replaced with whatever the name of your WSGI module is. By default, this is the same as the name of your project module, but it may be changed using the DJANGOSETTINGSMODULE environment variable. Using this configuration will automatically apply any database migrations.

  9. Create a manifest.yml file in the root of your local folder.

    ---
    applications:
    - name: my-app
      memory: 512M
      buildpack: python_buildpack
    

    where my-django-app is the name that will be used for the app within GOV.UK PaaS.

    Replace my-django-app with a unique name for your app. (You can use cf apps to see apps which already exist).

    The memory line tells the PaaS how much memory to allocate to the app.

  10. If your app requires a database, create a PostgreSQL backing service and bind it to your app. Then see the section on PostgreSQL setup below.

  11. To push your app, do:

    cf push APPNAME

    from the directory which contains all the code and configuration files.

    If you want to upload the app without starting it (for example, if you need to create a PostgreSQL service), run cf push --no-start APPNAME, then when you are ready to start the app, run cf start APPNAME.

You can now view your app at https://APPNAME.cloudapps.digital.

PostgreSQL setup with Django

Add these lines to your requirements.txt:

psycopg2==2.6.2 #installs the postgres driver
dj-database-url==0.3.0 #grabs environment variables and dumps them into a Django settings file

In your settings.py file, make sure you import the dj_database_url package we added to the requirements.txt file above:

    import dj_database_url

This package will automatically parse the VCAP_SERVICES environment variable and set DATABASE_URL to the first database found.

Then you’ll need to add a DATABASES setting. It’s best to add this to the settings.py file.

    DATABASES = {}
    DATABASES['default'] =  dj_database_url.config()

Your local_settings.py file will override this when you’re working locally.

The Procfile configuration provided in the section above will automatically apply database migrations.

Stopping and deleting apps

It’s important to know that when you perform a cf push command, there are three things that happen on the platform:

  1. your code is uploaded, after which a buildpack converts it to a single container, called a ‘droplet’, that can be run on PaaS as an app
  2. the droplet is used to start the requested number of instances of that app
  3. a route is created, connecting your app to the internet

When deleting apps you need to remember aspects 1 and 3.

Databases, CDNs and other services have a different lifecycle and need to be removed separately.

Stopping and starting apps

If you temporarily want to stop your app, freeing up memory from your quota, you can use the command

cf stop [appname]

This will stop the app running (although databases and other provisioned services will still be running and chargeable). Users visiting your app’s URL will get the error 404 Not Found: Requested route ('[appname].cloudapps.digital') does not exist.

You can start a stopped app with

cf start [appname]

Deleting apps

BEFORE YOU START: This is irreversible. We strongly recommend running the cf target command before you start: check you are where you think you are, and working on what you think you are working on.

If you want to remove your app completely it’s tempting to jump in with the cf delete command, but there are a few things to beware of:

  • Services that are used by apps do not automatically get deleted, and would still be chargeable
  • Routes between the internet and your apps need to be explicitly removed.

If you have a simple app without any services the best way to delete it is

cf delete -r [APPNAME]

which will delete the app and its routes in one go. If your app does have services, please delete them first.

If you accidentally delete your app without the -r option, you can delete the route manually. First confirm the details of the orphaned route by typing the cf routes command, to get a list of all active routes in the current space. This will list the space, the hostname, the overall domain (normally cloudapps.digital), port, path, type and any bound apps or services. You will see your hostname listed but without an associated app. Use this information to populate the following command:

cf delete-route [domain name] --hostname [hostname]

Deleting services

Before deleting your app, you will need to delete any services you have provisioned for it.

You can check the details of these by using the cf services command to see all active services in the current space. You will be able to see which services are bound only to your app - don’t delete services that other apps are also using.

You can then use this information to run

cf delete-service [SERVICE INSTANCE]

When all services are removed you can then delete your app, as above.

Deploy a docker image (experimental)

This section explains how to deploy an app from a docker image. The role of the docker image is to serve as a packaging format and the requirements for the app and its runtime environment are the same as for apps deployed using buildpacks. Configuration via manifest.yml also stays the same, albeit you should not specify buildpack. If you do, it will be ignored, which can create some confusion to anyone reading the manifest. Our platform currently only supports deploying images from docker hub or Docker Trusted Registry [external page] servers. You cannot currently deploy from local images.

To deploy an app using a docker image stored in docker hub via CF cli:

cf push myapp --docker-image dockerhuborg/app_image:tag

To deploy an app using a docker image stored in Docker Trusted Registry (DTR) via CF cli:

cf push myapp --docker-image MY-PRIVATE-REGISTRY.DOMAIN:5000/image/name:tag

You might notice that the staging process is simplified when deploying from a docker image - this is because there is no need to build the app from the pushed code using a buildpack. Also, the buildpack for apps deployed from a docker image will be reported as unknown in the app information (cf app <myapp>).

There are a few more specifics about docker image support in Cloud Foundry:

  • Local storage in each container instance is ephemeral. Every container launch will start with unmodified image.
  • Traffic will be routed to the lowest exposed port (by EXPOSE command in Dockerfile). The PORT environment variable will be also set to this number.
  • Buildpack based apps use bash to execute the app start command (bash -c <command>), but docker image based apps use sh (sh -c <command>) - as bash presence cannot be assumed in your docker image.
  • Privileged container mode is not supported. Features depending on this will not work.
  • Only registries that use Docker Registry API V2 are supported.
  • There is currently no support for deploying images from private/non DTR registries.

Experimental feature

Please note that this is currently only an experimental feature, meaning you can potentially encounter some unexpected behaviour or glitches. It is also possible that we might turn the feature off in the future. It should not be used in live services at this stage.

Lastly, our support for this functionality has lower priority than usual requests, so it is possible that responses to your queries regarding this will take a bit longer. However we are turning on this feature in order get feedback from service teams, so please contact support with any comments or to talk about use in production systems.

Applying security updates

When your app is built with a non-custom buildpack, you are responsible only for the code of your app. The platform provides both the buildpacks and also a safe and secure root filesystem (cflinuxfs2). Buildpack based apps never have root access in their container, which adds another layer of security. We do periodically apply security and funcionality updates to the buildpacks, container root FS and the platform itself.

When your app is deployed from docker image, responsibility for the root filesystem and for the complete build of your app shifts to you. You need to ensure proper measures are applied to make your app and root image safe and with the latest security updates. We recommend you use one of the major and well supported linux distributions as your base container. This way you will be able to build an image with latest security patches easily.

Production checklist

Before deploying an app for production use, check the following:

  1. If your app uses a PostgreSQL service, make sure that you have selected the high-availability plan (M-HA-dedicated-9.5) for the bound postgres service instance.
  2. You are running at least two instances of the app to ensure availability. Use cf scale APPNAME -i 2 to add an extra running instance.
  3. You are prepared to use a blue-green deployment process [external link] for when the app needs to be updated or restarted (this avoids problems due to a known issue with the PaaS that can generate transient 404 errors).

Configuring a custom continuous integration (CI) system

Setting up a CI service helps you make sure your team are delivering new features without breaking existing functionality. CI will deploy software automatically, but only if the tests pass. This helps you to iterate code faster and with greater confidence, a huge benefit once the codebase becomes larger and more complex.

Tools we have seen used to manage the tests and deployments include Travis, Circle, TeamCity and Jenkins. Detailed instructions on configuring Travis or Jenkins are below.

  • Travis CI is cloud-based, has a free tier, and is commonly used when a team want to start deploying prototypes from shared GitHub repositories - rather than code running locally on a particular machine. This is equivalent to linking GitHub repositories to Heroku, and takes less than ten minutes. Set up Travis
  • Jenkins is more complex, needs detailed technical knowledge, and requires its own infrastructure. Set up Jenkins

Security is a key concern when setting up a CI system that interacts with a PaaS. You should protect confidential data and your systems by:

  • running all tests as a low-privilege PaaS user
  • not exposing your PaaS credentials publicly; you should store them in encrypted form and decrypt only during runtime.
  • running your tests in a dedicated environment before deploying to production

For security reasons, GOV.UK PaaS will lock your account if your CI system makes multiple failed login attempts in a short period of time. This can happen if you provide incorrect or expired login details. See the Failed login rate limit section for more information.

Credentials for automated accounts

In order to avoid developer credentials leaking, particularly those with admin rights, you should should request a dedicated PaaS user account for use by your CI service. If you want to deploy to multiple spaces with your CI tool, use a different user account for each. Each user should only have SpaceDeveloper access to a single space within your organisation.

To request a PaaS user account for a CI service, contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk and tell us the email address of the CI user. This should be a unique email address, not already used by another PaaS account. If other people within your team need to be able to reset the password then we recommend that you use a group email address that they also receive the email for.

We set up user accounts for CI services in the same way as user accounts for people; the accounts have the SpaceDeveloper role in the sandbox space.

Use Travis

As at April 2017, Cloudfoundry support is currently included in the latest ‘edge’ version of Travis, which means it will work with GOV.UK PaaS. We expect it will become part of the core product soon. Likewise, our support for this is also new, so please contact our support team with any problems so we can continue to improve our documentation.

In order to do this, you will need to register for a Travis account. If you link this to your GitHub account, it will be able to see all your personal code repositories as well as those of the GitHub organisations you belong to.

Use the switch icons allow you to turn on the repos Travis will manage - remember that you’ll need administrator access on any shared repos. This helps stop you deploying any code you shouldn’t!

You will need to know the the particular PaaS organisation and space you want Travis to deploy your app into. If you don’t know, you can get these by logging into PaaS with a terminal window and using the cf orgs and cf spaces commands.

If you haven’t already done so, we also strongly recommend that you add a name entry to your manifest.yml file so that you’ll know the address of your app once published. Find out about names and URLs on PaaS.

Plan for dedicated CI accounts

Note that using your own PaaS login details within a CI service should only be a temporary measure. Dedicated user accounts should be set up for testing as soon as reasonably possible - by the end of Alpha at the latest. If you are already an Org Manager role on another project in Beta or Live, you should do this anyway. Find out more about credentials for automated accounts.

Download the Travis Gem

Make sure you use at least ruby 1.9.3 and download the gem:

gem install travis  --no-rdoc --no-ri

Run the setup to configure it to work with Cloudfoundry. It will ask for a username and password it can use to login to PaaS, the space and organisation you wish to deploy to, and the API URL. It will encrypt the data for you in .travis.yml file.

travis setup cloudfoundry

Manual setup

Create .travis.yml and put Cloudfoundry configuration in it. If you already have a .travis.yml file please add the appropriate lines:

deploy:
  edge: true
  provider: cloudfoundry
  username: tests@example.com
  api: https://api.cloud.service.gov.uk
  organization: mydepartament
  space: ci

Now you can add encrypted credentials to .travis.yml with the following command

travis encrypt --add deploy.password

Commit travis.yml to Github

Whichever setup approach you took, now commit your code to GitHub. After a few moments, the new .travis.yml file will be detected by Travis CI. If you look on the Travis website, you will see it start to automatically build and deploy your application.

Go to your application

Once Travis has finished building and deploying, go to the URL you set with the name field in your manifest.yml - the application should be at https://[name].cloudapps.digital.

As you continue to develop your prototypes, you can also ask Travis to start running automated tests for you, or deploy to different environments for each branch. More details can be found in the Travis documentation.

Push an app with Jenkins

There are two main approaches to pushing applications to GOV.UK PaaS with Jenkins:

  1. Use custom scripts (allows full scripting of deployments)
  2. Use the Cloud Foundry plugin for Jenkins (less flexible, relies on application manifest for configuration)

Both approaches require you to set up the credentials plugin first.

Set up dedicated accounts

CI systems should not use normal user accounts. Find out more about setting credentials for automated accounts in PaaS.

Setting up the credentials plugin

To install the credentials plugin manually:

  1. In the Jenkins web interface, click on Manage Jenkins, then Manage Plugins.
  2. Click on the Installed tab and check if “Credentials Plugin” is listed. If it’s listed, skip the next step.
  3. Click on the Available tab and find “Credentials Plugin”. Check the box to select the plugin, then click either Install without restart or Download now and install after restart at the bottom of the interface.

Once the plugin is installed, you will see a Credentials link in the left-hand navigation menu.

Now you can add credentials for the Cloud Foundry user you will be using to push apps from Jenkins.

  1. Click on Credentials, then System. By default, you will see a ‘Global credentials (unrestricted)’ domain. You should set up a different domain for each deployment target/PaaS user account.
  2. Click Add domain and enter a name and description, then click Save.
  3. Click Add credentials and enter the PaaS user account credentials. The kind of credentials is ‘Username with password’, which is typically selected by default.

You can now go on to either:

Setting up custom scripts

Before you set up custom scripts, make sure you first set up the credentials plugin.

Note that using the custom scripts approach exposes the password via the process command line, so it can be read by other processes running on the same machine. If this risk is not acceptable, please use the Cloud Foundry plugin described below. The Cloud Foundry project is aware of the problem and we expect they will provide a more secure login mechanism soon.

The custom scripts approach needs the Jenkins credentials binding plugin. To install it manually:

  1. In the Jenkins web interface, click on Manage Jenkins, then Manage Plugins.
  2. Click on the Available tab and find “Credentials Binding Plugin”. Check the box to select the plugin, then click either Install without restart or Download now and install after restart at the bottom of the interface.
  3. In your build configuration, select the Use secret text(s) or file(s) checkbox in the “Build Environment” section.
  4. Click on the Add drop down menu and select Username and password (separated).
  5. Choose your variable names and select the user whose credentials will be used.

Once this is set up, any shell scripts you configure in Jenkins will have the credentials available as environment variables.

To protect these credentials and prevent them leaking to other Jenkins jobs, we suggest you set your CF_HOME to a temporary directory.

A basic 'execute shell’ buildstep would look like this:

# Set CF_HOME in a temp dir so that jobs do not share or overwrite each others' credentials.
export CF_HOME="$(mktemp -d)"
trap 'rm -r $CF_HOME' EXIT

cf api https://api.cloud.service.gov.uk

# Note: the actual name of the environment variable is determined
# by what you enter into the Credentials Binding Plugin
cf auth "$CF_USER" "$CF_PASSWORD"

cf target -o myorg -s myspace
cf push

# Destroy token
cf logout

Setting up the Cloud Foundry Jenkins plugin

Using the Cloud Foundry plugin only allows Jenkins to push your application to GOV.UK PaaS as a post-build action: the equivalent of doing a cf login followed by a cf push. There is little scope for configuration beyond using the application manifest.

Before you do these steps, make sure you first set up the credentials plugin.

To install the Cloud Foundry plugin manually:

  1. In the Jenkins web interface, click on Manage Jenkins, then Manage Plugins.
  2. Click on the Available tab and find “Cloud Foundry Plugin”. Check the box to select the plugin, then click either Install without restart or Download now and install after restart at the bottom of the interface.

An extra post-build action called “Push to Cloud Foundry” is now available in the dropdown menu when you configure a job.

  1. In your job’s configuration, click the Add post-build action dropdown menu and select Push to Cloud Foundry.
  2. In the Target field, enter https://api.cloud.service.gov.uk.
  3. In Credentials, select the user you created using the credentials plugin.
  4. Enter your organisation and the space the application will be deployed to. See Orgs, Spaces and Targets for more details about organisations and spaces. You do not need to tick “Allow self-signed certificate” or “Reset app if already exists”.
  5. The rest of the fields can be left with their default values. The plugin expects you to have a manifest file called manifest.yml in the root of the application folder. If you do not, you can provide the path to the application manifest, or enter a manifest configuration directly into the plugin.
  6. Click Save.

Further information can be found on the Cloud Foundry plugin’s wiki page.

Deploy a backing or routing service

Services and plans

Many 12-factor applications rely on backing services such as a database, an email delivery service or a monitoring system. Routing services can be used to proxy and perform preprocessing on application requests such as caching, rate limiting or authentication.

In Cloud Foundry, backing and routing services are referred to as ‘services’ and are available through the Cloud Foundry cf marketplace command.

Currently, the following database services are available in the marketplace:

  • PostgreSQL
  • MySQL
  • MongoDB (currently only available on private beta)

Each service in the marketplace can have multiple plans available. For example, there are different PostgreSQL plans which vary by availability, storage capacity and encryption.

Users can also define their own external services that are not available in the marketplace by using User-Provided Service Instances.

Some service plans are paid: that is, you can potentially be billed by us for using them.

By default, access to paid plans is not enabled for a new organisation. Whether this is enabled or not is controlled by your organisation’s quota settings.

If you try to use a paid service and receive an error stating “service instance cannot be created because paid service plans are not allowed”, please contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk.

Accessing services

Your app can find out what backing services are available, and obtain credentials for the services, by parsing the VCAP_SERVICES environment variable.

User-Provided Service Instance

Cloud Foundry enables tenants to define User-Provided Service Instance [external link]. They can be used to deliver service credentials to an application, and/or to trigger streaming of application logs to a syslog compatible consumer. Once created, user-provided service instances behave like service instances created through the marketplace.

Future services

We are going to add more services in future based on demand, including Elasticsearch and Redis. If you need a particular backing service that we don’t yet support, please let us know at gov-uk-paas-support@digital.cabinet-office.gov.uk.

Using Database Services

GOV.UK PaaS enables you to create a database service and bind it to your app.

In Cloud Foundry, each service may have multiple plans available with different characteristics.

Currently, GOV.UK PaaS offers a service for PostgreSQL, MySQL and MongoDB, with multiple plans available. Please note that MongoDB is currently only available through private beta.

PostgreSQL

PostgreSQL is an object-relational database management system. It is open-source and designed to be extensible; currently the postgis and uuid-ossp extensions are enabled.

MySQL

MySQL is an open source relational database management system that uses Structured Query Language (SQL) and is backed by Oracle.

MongoDB

MongoDB is an open-source cross-platform document-oriented database program. It uses JSON-like documents with schemas, and is often used for content management such as articles on GOV.UK.

Using Database Services

To see the available plans, run:

cf marketplace -s SERVICE

Where SERVICE is the service you want; it can currently take the following values:

  • postgres
  • mysql
  • mongodb

Here is a shortened example of the sort of output you will see for the PostgreSQL service (the exact plans will vary):

service plan             description                                                                                                                                                       free or paid
M-dedicated-X.X          20GB Storage, Dedicated Instance, Max 500 Concurrent Connections. Postgres Version X.X. DB Instance Class: db.m4.large.                                           paid
M-HA-dedicated-X.X       20GB Storage, Dedicated Instance, Highly Available, Max 500 Concurrent Connections. Postgres Version X.X. DB Instance Class: db.m4.large.                         paid
...
Free                     5GB Storage, NOT BACKED UP, Dedicated Instance, Max 50 Concurrent Connections. Postgres Version X.X. DB Instance Class: db.t2.micro.                              free

Free and paid service plans

Most plans are paid, meaning that we will bill you based on your usage of the service.

There is a free plan available with limited storage. This should only be used for development or testing, not for production.

Paid services may not be enabled for your organisation. If they’re not enabled, when you try to set up a paid service, you’ll receive the error “service instance cannot be created because paid service plans are not allowed”. One of your Org Managers must contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk to request that we enable paid services.

Failover process

High availability plans - PostgreSQL and MySQL

We recommend you use one of the high availability plans (indicated by HA in the name) for your PostgreSQL and MySQL apps. These plans use Amazon RDS Multi-AZ instances which are designed to be 99.95% available (see Amazon’s SLA for details).

When you use the high availability plan, Amazon RDS provides a hot standby service for failover in the event that the original service fails.

The failover process means that Amazon RDS will automatically change the DNS record of the database instance to point to the standby instance. You should make sure that your app doesn’t cache the database IP, and any DNS caching should be configured with a low time to live (TTL). Consult the documentation for the language/framework you are using to find out how to do this.

During failover, there will be an outage period (from tens of seconds to a few minutes).

See the Amazon RDS documentation on the failover process for more details.

You should test how your app deals with a failover to make sure you are benefiting from the high availability plan. We can trigger a failover for you. Please contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk to arrange this.

High availability plans - MongoDB

MongoDB uses replica sets i.e. clusters of MongoDB servers that replicate the contents of the master database. Replica sets provide high availability through sharding, a method for distributing data across multiple machines. A shard is a single mongod instance or replica set that stores some portion of a sharded cluster’s total data set. In the standard configuration for MongoDB on our service provider, you get a single-shard with three nodes in a replica-set. Mongo will push all write operations on to the primary data node and then propagate them to the secondary data node on the shard. Should the primary node fail, the secondary will be promoted to primary providing fault-tolerance and redundancy for your databases. The Mongo routers are responsible for dictating any failover, along with load-balancing operations to any of the nodes in the cluster.

Your Mongo instances are backed up automatically, but are not yet directly available to tenants. Please contact us if you need to recover a backup or want to know when related features will be in our roadmap.

Encrypted plans

Plans with enc in the name include encryption at rest of the database storage. This means that the data on the disk and in snapshots is encrypted.

We recommend that you use an encrypted plan for production services or those that use real data.

Once you’ve created a service instance, you can’t enable or disable encryption. There’s no way to convert an unencrypted service instance to an encrypted one later.

This applies to PostgreSQL and MySQL services. There are currently no encrypted plans for the MongoDB service.

Read replicas

Amazon RDS has the capability to provide a read replica: a read-only copy of your database. This can be useful for performance, availability or security reasons.

See the Amazon RDS documentation on read replicas to learn more.

GOV.UK PaaS doesn’t currently support read replicas, but if you think you would find them useful, please contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk, providing details of your use case.

This applies to PostgreSQL and MySQL services.

Setting up a service

To create a service and bind it to your app:

  1. From the command line, run:

    cf marketplace -s SERVICE

    where SERVICE is the service you want, to see details of the available plans.

  2. Run:

    cf create-service SERVICE PLAN SERVICE_INSTANCE

    where PLAN is the plan you want, and SERVICE_INSTANCE is a unique, descriptive name for this instance of the service; for example:

    cf create-service postgres M-dedicated-9.5 my-pg-service

    Note that for production usage, we recommend you select a high-availability encrypted plan (one with HA-enc in the name).

  3. It may take some time (5 to 10 minutes) for the service instance to be set up. To find out its status, run:

    cf service SERVICE_INSTANCE

    for example:

    cf service my-pg-service

    where SERVICE_INSTANCE is the name you gave the instance when you created it.

  4. Wait until the service status reported by the above command is ‘create succeeded’. Here is an example of the type of output you will see once the service is created:

    Service instance: my-pg-service
    Service: postgres
    Bound apps:
    Tags:
    Plan: M-dedicated-9.5
    Description: AWS RDS PostgreSQL service
    Documentation url: https://aws.amazon.com/documentation/rds/
    Dashboard:
    
    Last Operation
    Status: create succeeded
    Message: DB Instance 'rdsbroker-9f053413-97a5-461f-aa41-fe6e29db323e' status is 'available'
    Started: 2016-08-23T15:34:41Z
    Updated: 2016-08-23T15:42:02Z
    
  5. You can now bind the service to your app. Run:

    cf bind-service APPLICATION SERVICE_INSTANCE

    where APPLICATION is the name of a deployed instance of your application (exactly as specified in your manifest or push command), for example:

    cf bind-service my-app my-pg-service

  6. If the app is already running, you should restage the app to make sure it connects:

    cf restage APPLICATION

  7. To confirm that the service is bound to the app, you can run:

    cf service SERVICE_INSTANCE

    and check the Bound apps: line of the output.

Accessing the service from your app

Your app must make a TLS connection to the service. Some libraries use TLS by default, but others will need to be explicitly configured.

MongoDB uses self-signed certificates. In order for your app to verify a TLS connection to the MongoDB service, the app must use a CA certificate included in a VCAP_SERVICES environmental variable.

GOV.UK PaaS will automatically parse the VCAP_SERVICES environment variable to get details of the service and then set the DATABASE_URL variable to the first database found.

Use cf env APPNAME to see the app’s environment variables and confirm that the variable has been set correctly.

If your app writes database connection errors to STDOUT or STDERR, you can view recent errors with cf logs APPNAME --recent. See the section on Logs for details.

PostgreSQL maintenance & backups

PostgreSQL service maintenance times

The PaaS PostgreSQL service is currently provided by Amazon Web Services RDS. Each PostgreSQL service you create will have a randomly-assigned weekly 30 minute maintenance window, during which there may be brief downtime. (To minimise this downtime, select a high availability plan with HA in its name).

Minor version upgrades (for example from 9.4.1 to 9.4.2) will be applied during the maintenance window.

For more details, see the Amazon RDS Maintenance documentation [external page].

If you need to know the time of your maintenance window, please contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk. Window start times will vary from 22:00 to 06:00 UTC. We will add the ability to set the time of the maintenance window in a future version of GOV.UK PaaS.

PostgreSQL service backup

The data stored within any PostgreSQL service you create is backed up using the standard Amazon RDS backup system, except if you are using the free plan.

Backups are taken nightly at some time between 22:00 and 06:00 UTC. Data is retained for 7 days.

There are two ways you can restore data to an earlier state:

  1. You can restore to the latest snapshot yourself. See Restoring a PostgreSQL service snapshot for details.

  2. We can manually restore to any point from 5 minutes to 7 days ago, with a resolution of one second. Data can be restored to a new PostgreSQL service instance running in parallel, or it can replace the existing service instance.

    To arrange a manual restore, contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk. We will need approval from your organization manager if restoring will involve overwriting data.

Note that data restore will not be available in the event of an RDS outage affecting the entire Amazon availability zone.

For more details about how the RDS backup system works, see Amazon’s DB Instance Backups documentation [external page].

Restoring a PostgreSQL service snapshot (experimental)

You can create a copy of any existing PostgreSQL service instance using the latest snapshot of the RDS instance. These snapshots are taken during the nightly backups described above.

This can be useful if you want to clone a production database to be used for testing or batch processing.

This feature is experimental; we expect it to work and we’d like people who use it to tell us if it worked for them. It has the following limitations:

  • You can only restore the most recent snapshot from the latest nightly backup
  • You cannot restore from a service instance that has been deleted
  • You must use the same service plan for the copy as for the original service instance
  • You must create the new service instance in the same organisation and space as the original. This is to prevent unauthorised access to data between spaces. If you need to copy data to a different organisation and/or space, you can use pg_dump and pg_restore via SSH tunnels.

To restore from a snapshot:

  1. Get the GUID of the existing instance by running:

    cf service SERVICE_INSTANCE --guid

    where SERVICE_INSTANCE is the name of the PostgreSQL service instance you want to copy.

    for example:

    cf service my-pg-service --guid

    This returns a GUID like 32938730-e603-44d6-810e-b4f12d7d109e.

  2. Trigger the creation of a new service based on the snapshot with

    cf create-service postgres PLAN NEW_SERVICE_INSTANCE -c '{"restore_from_latest_snapshot_of": "GUID"}'

    where: + PLAN is the identifier for the plan used in the original instance; you can find out what this is using cf service SERVICE_INSTANCE + NEWSERVICEINSTANCE is a unique, descriptive name for this new instance (not the name of the original) + GUID is the GUID from step 1

    For example:

    cf create-service postgres M-dedicated-9.5 my-pg-service-copy -c '{"restore_from_latest_snapshot_of": "32938730-e603-44d6-810e-b4f12d7d109e"}'

  3. It may take some time (5 to 10 minutes) for the new service instance to be set up. To find out its status, run:

    cf service NEW_SERVICE_INSTANCE

    for example:

    cf service my-pg-service-copy

  4. Once the status reported by the above command is “create succeeded”, you can use the instance. See Setting up a PostgreSQL service for more details.

Route services

Tenants may wish to apply some processing to requests before they reach an application. Common examples of use cases are authentication, rate limiting, and caching services.

Cloud Foundry allows the tenants to bind application routes to Route Services [external link]. A Route Service acts as a full proxy. Once it is bound to a route, the platform routing layer will send every request for that route to the Route Service endpoint. The Route Service can then process the incoming request, proxy it back to the original application, and finally process the response request before returning it to the original client.

Using Route Services has some consequences to be aware of:

  • Every request will include additional latency as it will be proxied via the Routing Service.
  • The Route Service will be able to access all the request content in clear.
  • The Route Service would become a critical point of failure, and if it is not available, the application will not be available for the end users.

User-Provided Route Services

Tenants can define their own Route Service instance by using a User-Provided Service Instance that points to any HTTPS service. This endpoint must fulfill the following requirements:

  • It must be a HTTPS endpoint with a valid certificate.
  • It can be a application running in the platform itself or an external service on the Internet.
  • It must be reachable from the platform (ie. not blocked by a firewall or in a private network).
  • It must implement the Route Service Protocol

This is how you define a User-Provided Route Service Instance and map it to the route of your app:

  1. From the command line, run:

cf create-user-provided-service SERVICE_INSTANCE -r ROUTE_SERVICE_URL

where SERVICE_INSTANCE is a unique, descriptive name for this Route Service Instance, and ROUTE_SERVICE_URL is the url of the route service endpoint; for example:

cf create-user-provided-service my-route-service -r https://route-service.example.org

the service will be immediately ready to be used, and you can query its status by running:

cf service my-route-service

  1. Bind this Route Service Instance to the application route

cf bind-route-service DOMAIN SERVICE_INSTANCE --hostname HOSTNAME

where DOMAIN is the domain used by your application, likely the default one cloudapps.digital, SERVICE_INSTANCE the name of the service that you have just created, and HOSTNAME the host or app name assigned to the app. For example:

cf bind-route-service cloudapps.digital my-route-service --hostname myapp

  1. You can list the routes of the current space, to see the applications and Route Services bound to them:

cf routes

If the Route Service endpoint is not responding correctly, you might get the following response when querying the route:

502 Bad Gateway: Registered endpoint failed to handle the request.

If you get this error, double check that the endpoint is working and reachable from the platform, that it is using a valid SSL certificate, that responds timely and that it implements the Route Service Protocol.

Implementing a Route Service

A Route Service can be implemented as a HTTPS application that will receive all requests for the associated route from the Cloud Foundry router. The request will contain the following additional headers:

  • X-CF-Forwarded-Url header contains the URL of the application route. The route service should forward the request to this URL.
  • X-CF-Proxy-Signature and X-CF-Proxy-Metadata: Signature and metadata used by the platform route itself to validate the request sent by the route service. The route service must always include these headers.

The Route Service must proxy back the request to the application route defined in header X-CF-Forwarded-Url within 60 seconds, otherwise the request will be refused. In addition, the total time to process the request and send a response back must be within 900 seconds.

Route Service request life cycle

You can refer to the Cloud Foundry documentation, example route services, and tutorial [external links] to learn more about how to implement route services.

Example: Route Service to add authentication

In the following example we will add a Route Service to provide basic HTTP authentication to the bound routes. This can be used to restrict access to a beta application.

An example of such a Route Service application code can be found in cf basic auth route service [external link]. Please note this is a proof-of-concept and is not intended to run in production.

We will deploy it as an application in the platform itself. Then we will bind this Route Service to a deployed application called myapp, accessible via https://myapp.cloudapps.digital.

  1. Deploy the route service as any other other application. Do not start it yet, as we need to configure it first.

    git clone https://github.com/alext/cf_basic_auth_route_service
    cd cf_basic_auth_route_service
    cf push my-basic-auth-service-app --no-start
    

Note: You might want to change my-basic-auth-service-app with a different name to avoid conflicts with other tenants’ apps.

  1. This example route service can only authenticate one user with fixed username and password. Choose any value for username and password then pass them to the application via environment variables AUTH_USERNAME and AUTH_PASSWORD. Finally the Route Service can be started:

    cf set-env my-basic-auth-service-app AUTH_USERNAME myuser
    cf set-env my-basic-auth-service-app AUTH_PASSWORD pass1234
    cf start my-basic-auth-service-app
    

The new service is serving in https://my-basic-auth-service-app.cloudapps.digital.

  1. Register the Route Service endpoint as a User-Provided Service Instance:

    cf create-user-provided-service my-basic-auth-service -r https://my-basic-auth-service-app.cloudapps.digital
    
  2. Finally, bind the Route Service to the application route:

    cf bind-route-service cloudpipelineapps.digital my-basic-auth-service --hostname myapp
    

The application in https://myapp.cloudapps.digital will now ask for basic HTTP authentication, with login myuser and password pass1234.

Using a custom domain

This section explains how to configure the domain name of your service for production applications.

When you use a custom domain you can also take advantage of the following:

  1. Fast delivery of content to your users via Content Distribution Network (CDN) caching (using AWS CloudFront).

  2. Encrypted user traffic via HTTPS support. This option comes with free TLS certificates with auto-renewal (using Let’s Encrypt).

How to create an instance of this service

To see the plans available you can run:

cf marketplace -s cdn-route

Before you begin, note that once you create a CDN service instance you can’t update or delete it until it has been successfully configured. This means if you make a mistake that prevents it from being successfully configured, you’ll need to ask support to manually delete the service instance.

First, target the space your application is running in:

cf target -o ORGNAME -s SPACENAME

Create a domain in your organisation (replace ORGNAME with your org name, and replace example.com with your domain):

cf create-domain ORGNAME example.com

Map the route to your app:

cf map-route APPNAME example.com

Create a cdn-route service instance by running the following command, replacing my-cdn-route with a name for your service instance, and example.com with your domain:

cf create-service cdn-route cdn-route my-cdn-route \
    -c '{"domain": "example.com"}'

This command includes cdn-route cdn-route because cdn-route is the name of the service and the name of the service plan.

If you have more than one domain you can pass a comma-delimited list to the domain parameter. Keep in mind that the broker will wait until all domains are CNAME’d, as explained in the next step:

cf create-service cdn-route cdn-route my-cdn-route \
    -c '{"domain": "example.com,www.example.com"}'

The maximum number of domains that can be associated with a single cdn-route service instance is 100.

How to set up DNS

Note: If you are creating a new site on GOV.UK PaaS, or you are migrating an existing site to GOV.UK PaaS that can tolerate a small amount of downtime during the migration, you can skip the first step and proceed directly to Create CNAME record(s).

Step 1: Create TXT record(s)

Once you’ve created the service instance you need to retrieve the instructions to set up your DNS. Run the following command, replacing my-cdn-route with the service instance name you used in the previous step.

$ cf service my-cdn-route

Last Operation
Status: create in progress
Message: Provisioning in progress [example.com => origin-my-paas-app.cloudapps.digital]; CNAME or ALIAS domain example.com to d3nrs0916m1mk2.cloudfront.net or create TXT record(s):
name: _acme-example.com., value: ngd2suc9gwUnH3btm7N6hSU7sBbNp-qYtSPYyny325E, ttl: 120

Create the TXT record(s) as instructed by the broker. The existence of these records will be validated by Let’s Encrypt when issuing your certificate and will not affect your site.

After the records have been created you will have to wait for up to an hour for the certificate to be provisioned. Your certificate has been provisioned when the cf service my-cdn-route command reports the status as create succeeded.

Last Operation
Status: create succeeded
Message: Service instance provisioned [example.com => origin-my-paas-app.cloudapps.digital]; CDN domain d3nrs0916m1mk2.cloudfront.net

It may take some time for CloudFront to serve your origin from all locations, because it is a globally distributed network. You can check in advance by sending requests to CloudFront and passing the custom domain in the Host header, using the example below. The CloudFront domain can be found by running cf service my-cdn-route.

curl -H "Host: <Custom domain>" https://<CloudFront domain>/

When this is working consistently, you can flip your DNS record to start serving content from the CDN. You may still have to wait for up to an hour to make sure the CloudFront distribution is updated everywhere.

Step 2: Create CNAME record(s)

Once the TXT records have been validated, or if you’ve decided to skip that step, you need to point your custom domain at the CDN. Run cf service my-cdn-route with the service instance name you choose.

Last Operation
Status: create succeeded
Message: Service instance provisioned [example.com => origin-my-paas-app.cloudapps.digital]; CDN domain d3nrs0916m1mk2.cloudfront.net

The output will include the CDN domain the broker has created for you. In this case you need to create a CNAME record in your DNS server pointing example.com to d3nrs0916m1mk2.cloudfront.net..

After the record is created wait for up to an hour for the CloudFront distribution to be provisioned and the DNS changes to propagate. Then visit your custom domain and see whether you have a valid certificate - in other words, that visiting your site in a modern browser doesn’t give you a certificate warning.

CDN Configuration Options

The following table shows the values you can provide when provisioning or updating a CDN. Further information on using the headers and cookies parameters can be found in the subsections below the table.

Options available
Name Required Description Default
domain Required Your custom domain (or domains separated by commas)
cookies Optional Forward cookies to the origin true
headers Optional A list of headers to forward to the origin ["Host"]
Cookies

Forwarding cookies to your origin can be disabled by setting the cookies parameter to false.

cf create-service cdn-route cdn-route my-cdn-route \
    -c '{"domain": "example.com", "cookies": false}'
Headers

By default our service broker configures CloudFront to forward the Host header to your app. Depending on how CloudFront handles certain headers [external link], you may want to whitelist extra headers:

cf create-service cdn-route cdn-route my-cdn-route \
    -c '{"domain": "example.com", "headers": ["Accept", "Authorization"]}'

You can supply up to nine headers. If you need to allow more headers you will have to forward all headers:

cf create-service cdn-route cdn-route my-cdn-route \
    -c '{"domain": "example.com", "headers": ["*"]}'

Note that forwarding headers has a negative impact on cacheability. See the More about how the CDN works section for details.

Troubleshooting

If your custom domain isn’t working:

  • make sure you’ve waited at least an hour
  • check your DNS setup to make sure you completed the CNAME record creation.

If you get the following error message when you try to update or delete a service instance:

Server error, status code: 409, error code: 60016, message: An operation for service instance [name] is in progress.

this happens because you can’t do anything to a service instance while it’s in a pending state. A CDN service instance stays pending until it detects the CNAME or ALIAS record. If this causes a problem for you ask support to manually delete the pending instance.

How to update a service instance

To update a service instance, run the following command (replace my-cdn-route with your service instance name, and replace example.com with your domain):

cf update-service my-cdn-route -c '{"domain": "example.com"}'

After the record is updated, wait up to an hour for the CloudFront distribution to be updated and the DNS changes to propagate. You may also have to clear your browser’s cache if it shows the old content after the DNS changes propagate.

When to update the DNS

You only need to add a CNAME entry when you update the domain field. If you do, follow How to set up DNS again.

Deleting a service instance

After you delete an existing service instance you may have to wait for up to an hour for the changes to complete and propagate.

More about how the CDN works

Caching

CloudFront uses your application’s Cache-Control or Expires HTTP headers to determine how long to cache content. If your application does not provide these headers, CloudFront will use a default timeout of 24 hours. This can be particularly confusing as different requests might be routed to different CloudFront Edge endpoints.

While there is no mechanism for GOV.UK PaaS users to trigger a cache clear, GOV.UK PaaS support can. Cache invalidation is not instantaneous; Amazon recommends expecting a lag time of 10-15 minutes (more if there are many distinct endpoints).

You can configure CloudFront to forward headers to your app, which causes CloudFront to cache multiple versions of an object based on the values in one or more request headers. See CloudFront’s documentation [external link] for more detail. This means the more headers you forward the less caching will take place. Forwarding all headers means no caching will happen.

Authentication

As noted above, cookies are passed through the CDN by default, meaning that cookie-based authentication will work as expected. Other headers, such as HTTP auth, are stripped by default. If you need a different configuration, see the above guidance on CDN configuration options.

Troubleshooting

Logs

If your app is failing to deploy, or crashing, and it’s not clear why from the command line client messages, you should consult the logs.

For log messages to be captured by Cloud Foundry, your application should be writing them to STDOUT/STDERR, rather than a log file.

The most direct way to view events related to your application through the deploy process is:

cf logs APPNAME

Used alone, cf logs will tail the combined stream of logs from each Cloud Foundry service involved in your application deploy. Running with the --recent flag will stream the entire Loggregator [external link] buffer for your app.

cf logs APPNAME --recent

cf events command

If you are trying to troubleshoot a problem and it’s hard to understand what’s happening from the logs, you can use the command:

cf events APPNAME

Running cf events shows you when an app starts, stops, restarts, or crashes (including error codes). The output is often easier to interpret than the output of cf logs.

More about logs and troubleshooting

For more information about logging and troubleshooting app deployment, see the following sections of the Cloud Foundry documentation:

Connecting with SSH

When you deploy an app to GOV.UK PaaS, it runs in a container, which is like a lightweight Linux virtual machine. Each app runs in its own isolated container.

Sometimes, it can be useful to connect directly to the container with SSH. You would usually only do this to get information for troubleshooting purposes, for example, if you can’t work out what is happening with your app using the cf logs and cf events commands described in the Logs section.

If you do run commands which will change the container temporarily, it’s a good idea to restart the app afterwards.

SSH is enabled by default. In most cases, you will find that you can SSH directly to your app’s container.

  1. Run:

    cf ssh APPNAME
    

    where APPNAME is the name of the app.

  2. For some tasks to work, you need to set up the interactive SSH session to match the buildpack environment. To do this, run:

    export HOME=/home/vcap/app
    [ -d ~/.profile.d ] && for f in ~/.profile.d/*; do source $f; done
    source /home/vcap/app/.profile
    

    You need to run this every time you start an SSH session.

    For more information, see the Cloud Foundry documentation on SSH Session Environment [external link].

  3. When you’re finished using SSH, use exit to end the session.

If you get an error like this:

FAILED
Error opening SSH connection: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain

go to the section below on Enabling SSH for an app.

Note that you do not need to generate any SSH keys. The cf CLI tool handles authentication for you.

Connecting with multiple instances

You may be running multiple instances of an app (created using cf scale or instances: in the manifest).

In this situation, each instance has a numerical instance index to distinguish it. You can see this in this example of the output of cf app exampleapp:

requested state: started
instances: 3/3
usage: 64M x 3 instances
urls: exampleapp.cloudapps.digital
last uploaded: Wed Dec 21 13:56:24 UTC 2016
stack: cflinuxfs2
buildpack: staticfile_buildpack

     state     since                    cpu    memory        disk         details
#0   running   2016-12-21 02:27:11 PM   3.0%   7.1M of 64M   6.8M of 1G
#1   running   2016-12-21 02:44:46 PM   1.0%   3.5M of 64M   6.8M of 1G
#2   running   2016-12-21 02:44:46 PM   1.0%   3.5M of 64M   6.8M of 1G

There are 3 instances, with instance indexes from 0 to 2.

If you have multiple instances like this and use cf-ssh, you will be connected to the app with an instance index of 0.

You can connect to a particular instance. For example, if you want to connect to instance 2, you can do this:

cf ssh --app-instance-index 2

Creating TCP tunnels with SSH

The cf ssh command supports local port forwarding, which allows you to create tunnels from your local system to the application instance container. This is useful when you want to connect from your local system to a backing service that is only accessible from an app running on GOV.UK PaaS.

Note: Versions 6.24 and 6.25 of the cf command line client have a bug that means it’s not possible to send more than 2GB of data through an SSH tunnel to a Cloud Foundry application. This bug has been fixed in version 6.26 of the client.

To enable local port forwarding, you can use the parameter -L:

cf ssh APPNAME -L LOCALPORT:REMOTEHOST:REMOTEPORT

This will forward the LOCALPORT port on the local system to the given REMOTEHOST host and REMOTEPORT port on the application container side.

Whenever a connection is made to this port, the connection is forwarded over the secure SSH channel, and a connection is made to the host and port REMOTEHOST:REMOTEPORT from the remote application container.

You can use the -L parameter multiple times to forward different ports.

The tunnel will be closed once the cf ssh command is stopped.

For example, you can connect directly to the PostgreSQL service bound to an application following these steps:

  1. Find the details of the service using cf env APPNAME. Here is some simplified and shortened example output:

    $ cf env myapp
    Getting env variables for app myapp in org myoth / space myorg as randomuser...
    OK
    
        System-Provided:
        {
        "VCAP_SERVICES": {
         "postgres": [
         {
         "credentials": {
         "host": "rdsbroker-01-ff-d2.cwm.eu-west-1.rds.amazonaws.com",
         "jdbcUrl": "jdbc:postgresql://rdsbroker-01-ff-d2.cwm.eu-west-1.rds.amazonaws.com:5432/rdsbroker_9f0_97_aa4?user=rdsbroker_9f0_97_aa4_owner\u0026password=xnYXthsgUFwPUOO",
         "name": "rdsbroker_9f0_97_aa4",
         "password": "xnYXthsgUFwPUOO",
         "port": 5432,
         "uri": "postgres://rdsbroker_9f0_97_aa4_owner:xnYXthsgUFwPUOO@rdsbroker-01-ff-d2.cwm.eu-west-1.rds.amazonaws.com:5432/rdsbroker_9f0_97_aa4",
         "username": "rdsbroker_9f0_97_aa4_owner"
        },
        ...
    

    You will need to know:

    • the remote host, displayed as "host":
    • the remote port, displayed as "port".
    • the PostgreSQL username, displayed as "username":
    • the PostgreSQL password, displayed as password:
    • the name of the database, displayed as name:
  2. Create a SSH tunnel using the local port 6666:

    cf ssh myapp -L 6666:HOST:PORT
    

    where HOST and PORT are the values you found in the previous step.

    This will open a shell in the remote container, and create a local tunnel using port 6666.

    Note: Be aware that this shell is in the remote application container, not the local system. You will need to open a new console if you want to work locally. The new port is open in the local system.

  3. In a different terminal, you can now connect to the local port in localhost:6666 using a postgres client:

    psql postgres://USERNAME:PASSWORD@localhost:6666/DATABASE_NAME
    

    replacing USERNAME, PASSWORD and DATABASE_NAME with the values from step 1

    You can also dump the database with pg_dump:

    pg_dump postgres://USERNAME:PASSWORD@localhost:6666/DATABASE_NAME > db.dump
    

SSH permissions

SSH can be either enabled or disabled independently for each space and app.

SSH must be enabled for both the space and the app before it will work. For example, if you have an app where SSH is enabled, but it is deployed to a space where SSH is disabled, SSH won’t work.

All new apps and spaces start out with SSH enabled.

Enabling SSH for an app

If you unexpectedly find that you can’t SSH to an app, the most likely cause is that SSH access is disabled for that app. This may be the case if your app was deployed before we enabled SSH access for tenants (prior to around 1600 GMT on 1st December 2016).

To check if SSH is enabled for an app, run:

cf ssh-enabled APPNAME

where APPNAME is the name of the app.

If you get a message like:

ssh support is disabled for APPNAME

you need to enable SSH for the app by running:

cf enable-ssh APPNAME

If you are running multiple instances of an app (created with cf scale or with instances: in the manifest), the enable-ssh command will affect all of them.

You do not need a special account permission level to enable SSH for an app. The default SpaceDeveloper role allows you to do this.

If SSH is already enabled, or enabling it still doesn’t make SSH work, go to Enabling SSH for a space below.

Enabling SSH for a space

If enabling SSH for an app doesn’t let you connect, check that SSH is enabled for the space it’s deployed in.

Check what space you’re working in with:

cf target

Then run:

cf space-ssh-allowed SPACENAME

where SPACENAME is the name of the space.

If you get a message like this:

ssh support is disabled in space 'sandbox'

you need to enable SSH for the space using:

cf allow-space-ssh SPACENAME

Your GOV.UK PaaS account needs the OrgManager or SpaceManager role to be able to enable SSH for a space. If the command above fails, ask someone with the correct role (probably a senior member of your team) to run it for you, or contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk.

Limiting SSH access

You should consider disabling SSH where it is not needed. For example, if you host the live versions of your apps in a production space, you may decide to disable SSH access there, but leave it enabled in your development and testing spaces.

More about using SSH

See the Cloud Foundry documentation on Accessing Apps with SSH [external link].

Failed login rate limit

If your Cloud Foundry account makes too many failed login attempts within a short period of time, your account will be locked for security reasons. While the account is locked, even the correct login details will not work.

Wait 5 minutes without trying to log in for the lock to expire.

Make sure that no automated services (such as Jenkins) are trying to log in using your account during this time.

PORT environment variable error

The PORT environment variable is system-provided, and you should not attempt to set it yourself.

If you are trying to set any environment variable and you get an error like this:

FAILED Server error, status code: 400, error code: 100001, message: The app is invalid: environment_variables cannot set PORT

the cause is that the value of PORT has been changed from the system-provided value.

Running these commands should fix the problem:

cf unset-env myapp PORT
cf restage myapp

where myapp is the name of the affected app instance.

Make sure that the app does not attempt to set the PORT variable.

PostgreSQL connection problems

If you have trouble connecting your app to a postgres backing service:

  1. Ensure your app is making a TLS connection to the PostgreSQL service.
  2. Use cf service SERVICE_INSTANCE to check that the service is bound to the app.
  3. Use cf env APPNAME to see the app’s environment variables and confirm that there is a VCAP_SERVICES={"postgres": section, indicating the app has correctly received the environment variables.
  4. Make sure your app writes database connection errors to STDOUT or STDERR, then look for recent errors with cf logs APPNAME --recent.
  5. If restarting the app doesn’t make it connect, try restaging the app with cf restage APPNAME.

Pushing apps fails on Windows (line endings issue)

We are aware of some cases where pushing an app can fail if you are pushing from a Windows machine that has saved project files with Windows-style (CRLF) line endings.

This happens because some buildpacks do not interpret the CRLF line endings correctly. This is an example of an error arising from an executable script with CRLF line endings when using the Ruby buildpack:

2016-12-02T06:03:25.95-0800 [APP/PROC/WEB/0]ERR /usr/bin/env: ruby
2016-12-02T06:03:25.95-0800 [APP/PROC/WEB/0]ERR : No such file or directory
2016-12-02T06:03:25.95-0800 [APP/PROC/WEB/0]OUT Exit status 127

Similar problems may happen in other buildpacks. If you find that pushing an app from Windows is failing, try making sure that files in your project are being saved with Unix-style LF line endings, not Windows-style CRLF line endings. Most text editors aimed at developers allow you to change the line ending style.

If you are working with a Git repository, disable Git’s autocrlf setting:

git config --global core.autocrlf false

then clone the repository again.

Managing apps

Scaling

The Cloud Foundry technology makes it easy to scale your application to meet increasing demand. Scaling does not happen automatically; you have to use the commands described below.

Note that the maximum resources you can use will be limited by your organization quotas. You can view them by running:

cf quotas

from the command line.

If you are anticipating a spike in demand for a service hosted on GOV.UK PaaS, please contact us well in advance at gov-uk-paas-support@digital.cabinet-office.gov.uk.

Increasing instances

You can change the number of instances of your app running in parallel. Incoming requests are automatically load-balanced across all instances.

For example, this command sets the number of running instances to five:

cf scale APPNAME -i 5

You can also use the manifest to set the number of instances that will start when you push the app:

---
  ...
  instances: 2

For a production app, you should always have at least two running instances.

Increasing memory and disk space

You can scale an application vertically by increasing the memory or disk space available to each instance of the app.

For example, this command increases the available memory for an app to 1 gigabyte:

cf scale APPNAME -m 1G

This command increases the disc space limit for an app to 512 megabytes:

cf scale myApp -k 512M

More about scaling

For more details, see Scaling an Application Using cf scale [external link] in the Cloud Foundry docs.

Quotas

Cloud Foundry capacity is managed by quotas. Quotas provide a reservation of application routes, memory, compute power, and service instances which your organization cannot exceed. You can set individual application quotas to control how much of your quota each of your applications can use.

Quota allocations

Your organisation will be assigned a quota based on your stated needs. This will cover the app instances you run. Backing services do not count towards this quota.

Your quota sets the following:

  • RAM: The amount of RAM available to your applications. The application also has a compute share derived from its memory limit.

  • Service instances: The number of service instances available to your organization.

  • Paid services: Whether or not paid services are available. postgres is a paid service.

  • Routes: The number of routes available to your applications (hostname and domain pairs where an application that exposes a listening port can be reached).

To see your organisation quota, run the command:

cf org YOURORG

where YOURORG is your organisation’s name. (If you don’t know the name, you can use cf orgs to find out).

Quota limits

If a new application push would exceed your organization’s quota, the request will fail with status code 400 and a message describing the limit that would be exceeded.

Example:

Creating app APPLICATION in org ORG / space SPACE as USER...
FAILED
Server error, status code: 400, error code: 100007, message: 
You have exceeded the memory limit for your organization's quota.

In this situation you have three options:

  1. Delete existing resources with cf delete, cf delete-service, cf delete-route or similar commands.
  2. Reconfigure existing application quotas and redeploy.
  3. Request a quota change: contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk.

Application quotas

As a PaaS tenant, you can divide your organization’s quota capacity amongst your applications as you see fit, by way of application quotas. Application limits are specified in your application manifest or as cf push command line options.

Use the following commands to set application quota options (in each pair below, the first is the version to use in the manifest, and the second is the command line version.)

  • memory: / -m

    Your application’s memory limit. An application’s compute limit is derived from its memory limit (see below).

  • disk_quota: / -k

    The maximum amount of disk space available to your app.

  • instances: / -i

    Sets the number of application instances to launch. Each additional instance receives the same memory and disk reservation. An application with a manifest specifying memory: 256M and instances: 4 would reserve 1GB (256M x 4) total.

    For a production application, you should always launch at least two instances.

Memory share and compute share

Your application’s compute limit is derived from its memory limit. Each application receives a guaranteed compute share equal to its relative share of memory.

Your application will be guaranteed to receive a portion of the vCPU compute power equal to its portion of memory allocation. In other words, it will receive at least this much vCPU time even if there are other applications competing for time.

Your application will be offered 100% of the vCPU compute power. If no other applications are running, the app can use all the computer power available. If there are other applications competing for time, each application’s guaranteed share determines how much time it will receive.

The application cannot access more than the specified amount of memory.

Application quota sizing

  • The environment default of 512MB memory is sufficient for most applications. Static sites and utility applications such as schedulers or loaders may require less. Use cf app APPNAME to check your application’s current memory and compute utilization.

    requested state: started
    instances: 1/1
    usage: 128M x 1 instances
    urls: 
    last uploaded: Wed Jul 22 20:09:56 UTC 2015
    
         state     since                    cpu    memory          disk          
    #0   running   2015-07-30 05:58:11 PM   0.0%   94.6M of 128M   80.4M of 128M      
    
  • Any application which exceeds its memory quota will be automatically restarted. Use cf events APPNAME to look for ‘out of memory’ crashes.

    ... description   
    ... index: 0, reason: CRASHED, exit_description: out of memory, exit_status: 255 
    

Redirecting all traffic

If a site moves to a different domain name, you can use a simple static site with a custom nginx.conf file to redirect all traffic from the old domain to the new domain. Example nginx.conf site for NEW_DOMAIN_NAME:

worker_processes 1;
daemon off;

error_log <%= ENV["APP_ROOT"] %>/nginx/logs/error.log;
events { worker_connections 1024; }

http {
  server {
    listen <%= ENV["PORT"] %>;
    server_name localhost;
    return 301 $scheme://NEW_DOMAIN_NAME$request_uri;
  }
}

Deploy your application to NEW_DOMAIN_NAME and then cf push a simple static site with that nginx.conf configuration to the old domain name. You can see a full working example here.

You can read more about nginx customization.

Managing users

User management overview

GOV.UK PaaS relies on the Cloud Foundry user management system to determine what each individual tenant user account can do.

Note that the “users” we’re talking about here are members of your team who use the command-line tool to manage apps on the PaaS, not the end users of your apps.

In this section, we’ll cover the most common roles and describe what they mean for how you use the PaaS.

For full details about Cloud Foundry roles, see the Cloud Foundry documentation [external link].

User roles

There are multiple roles that a user can have. These are always defined in relation to an organisation or a space. A user can have different roles in different orgs or spaces. A user can have more than one role within the same org or space.

To find out details of the users in an organisation and their roles, run:

cf org-users ORGNAME

where ORGNAME is the name of the organisation; use cf orgs to find out the names of organisations you can access.

Space Developer

This role applies within a particular space.

A Space Developer can push, start and stop apps, and create and bind services. This is the default role for a user who is not a manager.

For example, suppose you have testing and production spaces. You choose to grant a new junior developer the Space Developer role in the testing space, but no role in the production space. As a result, the developer can change apps in testing, but cannot do anything in production.

Space Manager

This role applies within a particular space.

A Space Manager can grant user roles within the space, and change properties of the space, such as the name. Being a Space Manager does not grant the ability to change apps or services, so you need the Space Developer role as well if you want to be able to do both things.

For example, suppose you have a senior developer who manages junior developers who work in your testing space, and also needs to change apps in both testing and production spaces. You choose to grant the senior developer the Space Manager role in the testing space, and the Space Developer role in both testing and production. As a result, the developer can add users to testing but not to production, and can change apps in both spaces.

Org Manager

This role applies to a particular organisation. Typically, each team only has one organisation, but you can have more than one.

An Org Manager can create/delete spaces and edit user roles. See the sections on Managing spaces and Granting access for details.

As an Org Manager, you have responsibilities related to adding and removing users: see the section on User lifecycle.

As part of our onboarding process, we will create an account with the Org Manager role. We recommend you have at least two people who are Org Managers, in case one is unavailable. The Org Managers would typically be senior staff: for example, you might choose to grant the role to your technical architect and a lead developer.

We will treat your Org Managers as our primary contacts for support issues. They will need to approve changes like adding new users, setting quotas and enabling paid services.

If you need the Org Manager role to be added to a user account, contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk.

User lifecycle

Users will need to be added or removed from GOV.UK PaaS as they join or leave your team.

The responsibility for managing the user lifecycle is split between GOV.UK PaaS support and the Org Managers on your team.

This section explains what needs to be done to add or delete users.

Adding users

User account creation is currently handled by our support team.

To add a new user to an existing project, have an Org Manager on your team contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk. We’ll add the new account and give it access to your sandbox space. An Org Manager will then need to grant access to additional spaces.

If you are looking to start a new project on the platform, please contact us at gov-uk-paas-support@digital.cabinet-office.gov.uk. We will create an Org Manager account first, so make sure to let us know who should have this responsibility.

Deleting users

When a team member leaves or stops working on a project, their access rights must be revoked.

One of your Org Managers must immediately remove all user roles the user has within your orgs and spaces.

The command to remove a space role is:

cf unset-space-role USERNAME ORG SPACE ROLE

For example, to remove all a user’s space roles you might need to run:

cf unset-space-role ana@example.com acme sandbox SpaceDeveloper

cf unset-space-role ana@example.com acme dev SpaceDeveloper

cf unset-space-role ana@example.com acme dev SpaceManager

If the user is an Org Manager, you will also need to remove that role from all your organisations:

cf unset-org-role USERNAME ORG ROLE

For example:

cf unset-org-role boss@example.com acme OrgManager

If the user is only leaving one project, but still needs access on the PaaS to work on other projects in other organisations, nothing more needs to be done.

If the user no longer needs access to GOV.UK PaaS, then the Org Manager should ask us to completely remove the user account.

Guidance

Buildpacks

If you’re relatively new to cloud or have a small development team, we recommend using the standard buildpacks to maximise the support you will receive from GOV.UK PaaS.

Using a custom buildpack will mean increased development and maintenance responsibilities and costs for you. If your language is not supported by standard buildpacks, we recommend that for a community supported custom buildpack that is maintained and updated by that community. We do not recommend that you create your own buildpack.

We are keen to help where we can, so please contact us if you are interested in this feature. If a custom buildpack is requested by multiple tenants, we will look to improving our support in this area.

Responsibilities

Your responsibilities differ depending on whether you use a standard buildpack, custom buildpack or docker image to deploy your app.

Standard Buildpack

You are only responsible for the app code and its dependencies.

Custom Buildpacks

You are responsible for managing this custom buildpack as well as the app code and its dependencies. This applies both in setting up the app, and running it in production.

When setting up the app:

  • Do your own due diligence on finding a maintained and supported custom buildpack so that you have a community to call upon in the case of problems
  • Test deploying with the buildpack early in your build process, not just before go-live

When running the app in production:

  • Ensure you have sufficient time to fix issues yourself as using them for deadline-driven development may be risky Regularly maintain your buildpack to update your runtime and dependencies as new security vulnerabilities are discovered and fixed
  • We may not be able to offer support for P1 or out-of-hours incidents; standard SLAs will not apply

Docker Images

You are responsible for your docker container and custom image. Learn about this experimental feature.

Custom Buildpacks compared to Docker Images

Custom buildpacks provide some advantages over choosing a Docker solution:

  • they are not dependent on Docker Hub availability a single way to specify app dependencies
  • large variety of open source buildpacks, meaning your deployment pipeline doesn’t need to include docker image creation
  • it is designed for 12-factor; some Docker images may contain databases or other forms of storage that should be provisioned as an external backing service

Docker Images also provide some advantages over custom buildpacks:

  • the same image can be run locally on developer machines, or on any Linux-based machine
  • exactly the same Docker image can be promoted between environments as an immutable asset, rather than being rebuilt each time (CF support for this is still experimental)