Working with Custom Buildpacks

Page last updated:

Buildpack Scripts

A buildpack repository contains three main scripts, situated in a folder named bin.

bin/detect

The detect script determines whether or not to apply the buildpack to an app. The script is called with one argument, the build directory for the app. The build directory contains the app files uploaded when a user performs a cf push.

The detect script returns an exit code of 0 if the buildpack is compatible with the app. In the case of system buildpacks, the script also prints the buildpack name, version, and other helpful information to STDOUT.

The following is an example detect script written in Ruby that checks for a Ruby app based on the existence of a Gemfile:

#!/usr/bin/env ruby

gemfile_path = File.join ARGV[0], "Gemfile"

if File.exist?(gemfile_path)
  puts "Ruby"
  exit 0
else
  exit 1
end

Optionally, the buildpack detect script can output additional details decided by the buildpack developer. These additional details include buildpack versioning information and a detailed list of configured frameworks and their associated versions.

The following is an example of the detailed information returned by the Java buildpack:

java-buildpack=v3.0-https://github.com/cloudfoundry/java-buildpack.git#3bd15e1 open-jdk-jre=1.8.0_45 spring-auto-reconfiguration=1.7.0_RELEASE tomcat-access-logging-support=2.4.0_RELEASE tomcat-instance=8.0.21 tomcat-lifecycle-support=2.4.0_RELEASE ...

bin/compile

The compile script builds a droplet by packaging the app dependencies, assuring that the app has all the necessary components needed to run.

The script is run with two arguments: the build directory for the app and the cache directory, which is a location the buildpack can use to store assets during the build process. During the execution of the compile script, all output sent to STDOUT is relayed through the cf CLI to the user.

The following is an example of a simple compile script:

#!/usr/bin/env ruby

#sync output

$stdout.sync = true

build_path = ARGV[0]
cache_path = ARGV[1]

install_ruby

private

def install_ruby
  puts "Installing Ruby"

  # !!! build tasks go here !!!
  # download ruby 
  # install ruby
end

bin/release

The release script provides feedback metadata to Cloud Foundry indicating how the app should be executed. The script is run with one argument, the build directory. The script must generate a YAML file in the following format:

default_process_types:
  web: start_command.filetype

default_process_types indicates the type of app being run and the command used to start it. This start command is used if a start command is not specified in the cf push or in a Procfile.

At this time, only web type of apps are supported.

Note: To define environment variables for your buildpack, add a bash script to the .profile.d directory in the root folder of your app.

The following example shows what a release script for a Rack app might return:

default_process_types:
  web: bundle exec rackup config.ru -p $PORT

Note: The web command runs as bash -c COMMAND when Cloud Foundry starts your app. Refer to the command attribute section for more information about custom start commands.

Droplet Filesystem

The buildpack staging process extracts the droplet into the /home/vcap directory inside the instance container, and creates the following filesystem tree:

app/ 
logs/
tmp/
staging_info.yml

The app directory contains BUILD_DIR contents, and staging_info.yml contains the staging metadata saved in the droplet.

Package Custom Buildpacks

Cloud Foundry buildpacks can work with limited or no Internet connectivity. The Buildpack Packager RubyGem gives the same flexibility to custom buildpacks, enabling them to work in partially or completely disconnected environments.

Use the Buildpack Packager

  1. Ensure that you have installed the Buildpack Packager RubyGem.
  2. Create a manifest.yml in your buildpack.
  3. Run the packager in cached mode:
    $ buildpack-packager --cached

The packager will add (almost) everything in your buildpack directory into a zip file. It will exclude anything marked for exclusion in your manifest.

In cached mode, the packager will download and add dependencies as described in the manifest.

The packager has the following option flags:

  • --force-download: By default, the packager stores the dependencies that it downloads while building a cached buildpack in a local cache at ~/.buildpack-packager. Storing dependencies enables the packager to avoid re-downloading them when repackaging similar buildpacks. Running buildpack-packager --cached with the --force-download option forces the packager to download dependencies from the S3 host and ignore the local cache. When packaging an uncached buildpack, --force-download does nothing.

  • --use-custom-manifest: To include a different manifest file in your packaged buildpack, you can call the packager with the --use-custom-manifest PATH/TO/MANIFEST.YML option. The packager generates a buildpack with the specified manifest. If you are building a cached buildpack, the packager vendors dependencies from the specified manifest as well.

For more information, see the documentation at the Buildpack Packager Github repository.

Use and Share the Packaged Buildpack

After you have packaged your buildpack using buildpack-packager you can use the resulting .zip file locally, or share it with others by uploading it to any network location that is accessible to the CLI. Users can then specify the buildpack with the -b option when they push apps. See Deploying Apps with a Custom Buildpack for details.

Note: Offline buildpack packages may contain proprietary dependencies that require distribution licensing or export control measures. For more information about offline buildpacks, refer to Packaging Dependencies for Offline Buildpacks.

Specify a Default Version

As of buildpack-packager version 2.3.0, you can specify the default version for a dependency by adding a default_versions object to the manifest.yml file. The default_versions object has two properties, name and version. For example:

default_versions:
- name: go
  version: 1.6.3
- name: other-dependency
  version: 1.1.1

To specify a default version:

  1. Add the default_version object to your manifest, following the rules below. You can find a complete example manifest in the Cloud Foundry go-buildpack repository.

  2. Run the default_version_for script from the compile-extensions repository, passing the path of your manifest.yml and the dependency name as arguments. The following command uses the example manifest from step 1:

    $ ./compile-extensions/bin/default_version_for manifest.yml go 1.6.3
    

Rules for Specifying a Default Version

The buildpack-packager script validates this object according to the following rules:

  • You can create at most one entry under default_versions for a single dependency. The following example causes buildpack-packager to fail with an error because the manifest specifies two default versions for the same go dependency.

    # Incorrect; will fail to package
    default_versions:
    - name: go
      version: 1.6.3
    - name: go
      version: 1.3.1
    
  • If you specify a default_version for a dependency, you must also list that dependency and version under the dependencies section of the manifest. The following example causes buildpack-packager to fail with an error because the manifest specifies version: 1.6.3 for the go dependency, but lists version: 1.5.4 under dependencies.

    # Incorrect; will fail to package
    default_versions:
    - name: go
      version: 1.6.3
    
    dependencies:
    - name: go
      version: 1.5.4
      uri: https://storage.googleapis.com/golang/go1.5.4.linux-amd64.tar.gz
      md5: 27b1c469797292064c65c995ffe30386
      cf_stacks:
      - cflinuxfs2
    

Deploy Apps with a Custom Buildpack

Once a custom buildpack has been created and pushed to a public git repository, the git URL can be passed via the cf CLI when pushing an app.

For example, for a buildpack that has been pushed to Github:

$ cf push my-new-app -b git://github.com/johndoe/my-buildpack.git

Alternatively, you can use a private git repository, with https and username/password authentication, as follows:

$ cf push my-new-app -b https://username:password@github.com/johndoe/my-buildpack.git

By default, Cloud Foundry uses the default branch of the buildpack’s git repository. You can specify a different branch using the git url as shown in the following example:

$ cf push my-new-app -b https://github.com/johndoe/my-buildpack.git#my-branch-name

Additionally, you can use tags or shas in a git repository, as follows:

$ cf push my-new-app -b https://github.com/johndoe/my-buildpack#v1.4.2
$ cf push my-new-app -b https://github.com/johndoe/my-buildpack#a2951e298bd22732fceb3968d541015120dbaf93

The app will then be deployed to Cloud Foundry, and the buildpack will be cloned from the repository and applied to the app.

Note: If a buildpack is specified using cf push -b the detect step will be skipped and as a result, no buildpack detect scripts will be run.

Note: A common development practice for custom buildpacks is to fork existing buildpacks and sync subsequent patches from upstream. To merge upstream patches to your custom buildpack, use the approach that Github recommends for syncing a fork.

Create a pull request or raise an issue on the source for this page in GitHub