Skip to content

module.mdx

Latest commit

 

History

History
249 lines (173 loc) · 8.2 KB

module.mdx

File metadata and controls

249 lines (173 loc) · 8.2 KB
Raw
title Module
description A module is a script executed during the build process of an image.

A module is a self-contained part of the build executed during the build process of an image. Most modules are bash and nushell scripts (blue-build/modules), but some default modules are implemented with Containerfile templating directly in blue-build/cli. Modules are configured in the recipe or an external configuration file.

Default configuration options

Modules themselves differ on what configuration options they use and require, and that information is available on module-specific reference pages. These options are always available.

type:

The name of the module to run. This corresponds to the name of the directory as well as the script's name in the module's directory. For example, using test would call the script in $MODULE_DIRECTORY/test/test.sh.

source: (optional)

:::caution A module can execute arbitrary code both in the image build and consequently on your booted system, so make sure you trust the source you specify. :::

The URL of the module repository (an OCI image) to pull the module from. If left unspecified, the source use is a hybrid the default module repository at ghcr.io/blue-build/modules and the custom modules in the local modules/ directory, where custom modules overwrite the default modules.

no-cache: (optional)

When set to true, the module is run regardless if previous layers in the build have been cached. This is useful in stages where the goal is to build the latest version of a program from a git repository. Otherwise, the module wouldn't run again unless previous layers were also re-ran.

env: (optional)

A list of environment variables to set when running the module. These variables are available to the module's script and can be used to customize its behavior. Set these as key-value pairs.

type: script
env:
  TEST: 'test'
snippets:
  - '[ -n "$TEST" ]'

secrets: (optional)

A list of secrets to mount when running the module. These secrets are available to the module's script and can be used only by it.

Creating a secret takes 2 parts:

  1. Specifying an environment variable, command, or file on the host containing the secret
  2. Defining the environment variable or file in the build to contain the secret

Host Side

In order to use a secret in the build, you will have to have it accessible on the host machine. This can come in the form of an environment variable, a file, the ssh socket, or a command that's ran to retrieve the secret. The type of the secret can be specified with the type: property.

Environment Variable

You can use type: env to pull the value of an environment variable as a secret.

modules:
  - type: script
    secrets:
      - type: env
        name: SECRET_ENV
        mount:
          type: env
          name: SECRET_ENV
    snippets:
      - echo $SECRET_ENV | login-script.sh
File

You can use type: file to read the contents of a file as a secret. Relative paths are relative to the root of the bluebuild project.

modules:
  - type: script
    secrets:
      - type: file
        source: ./secret.txt
        mount:
          type: env
          name: SECRET_ENV
    snippets:
      - echo $SECRET_ENV | login-script.sh
Command

You can use type: exec to execute a program/script that will retrieve the secret for you. The command must output the secret to stdout.

modules:
  - type: script
    secrets:
      - type: exec
        command: secret-manager-command
        args:
          - '-q'
          - some/arg/for/secret-command
        mount:
          type: env
          name: SECRET_ENV
    snippets:
      - echo $SECRET_ENV | login-script.sh
SSH

You can use type: ssh to mount the SSH socket into the build. This is useful for pulling from private git repos, or SSHing into a private server to retrieve files. You will need to make sure you have an SSH session loaded before running the build. You can do this by running:

# Start the ssh-agent and load env variables
eval "$(ssh-agent)"

# Add your ssh key to the current agent
ssh-add

Example:

modules:
  - type: script
    secrets:
      - type: ssh
    snippets:
      - git clone [email protected]:some/repo.git

Build side

When a secret is loaded from the host side, you will need to specify where to store the secret for that module run. Above we were using the type: env mount for demonstration purposes. You can combine most of the host side secrets (env, file, exec) to either build side mount (env, file). The type: ssh secret is mounted into the build for you, so a mount cannot be specified.

Environment variable

With type: env, a secret can be mounted into the build as an environment variable.

modules:
  - type: script
    secrets:
      - type: env
        name: SECRET_ENV
        mount:
          type: env
          name: SECRET_ENV
    snippets:
      - echo $SECRET_ENV | login-script.sh
File

With type: file, a secret can be mounted into the build as a file.

modules:
  - type: script
    secrets:
      - type: env
        name: SECRET_ENV
        mount:
          type: file
          destination: /tmp/secret_file
    snippets:
      - cat /tmp/secret_file | login-script.sh

How modules are launched

A module added into an image's configuration is turned into a RUN-statement that launches the module with a JSON version of its configuration in the generated Containerfile (or an equivalent dynamic bash command if using the legacy template).

For example, the following module configuration would be turned into the RUN-statement below:

# recipes/module.yml
type: rpm-ostree
install:
    - micro
uninstall:
    - firefox
    - firefox-langpacks
# the contents of this statement have been simplified slightly to better illustrate the topic on hand
RUN /tmp/modules/rpm-ostree/rpm-ostree.sh '{"type":"rpm-ostree,"from-file":"module.yml","repos":null,"install":["micro"],"remove":["firefox","firefox-langpacks"]}'

Module run environment

Every module is ran in an environment containing the following environment variables and functions.

CONFIG_DIRECTORY

Environment variable containing the path to the files for the build (/tmp/files or /tmp/config).

MODULE_DIRECTORY

Environment variable containing the path to the directory containing all the modules of the module repository the current module is from (/tmp/modules).

IMAGE_NAME

Environment variable containing the name of the image declared in recipe.yml.

IMAGE_REGISTRY

Environment variable containing the registry URL and namespace (usually ghcr.io/<username>). Can be used with IMAGE_NAME to get the full image URL.

BASE_IMAGE

Environment variable containing the URL of the OCI image used as the base.

OS_VERSION

Environment variable containing the major version of the OS image. The value is gathered from the VERSION_ID in /etc/os-release.

OS_ARCH

Environment variable containing the architecture codename of the OS image. For example: x86_64, aarch64, etc.

get_json_array

Bash function that helps with reading arrays from the module's configuration.

get_json_array OUTPUT_VAR_NAME 'try .key.to.array[]' "${1}"

or less readable, but safer command for extracting array values:

get_json_array OUTPUT_VAR_NAME 'try .["key"].["to"].["array"][]' "${1}"

module.yml

A module.yml is the metadata file for a public module, used on the website to generate module reference pages. May be used in future projects to showcase modules and supply some defaults for them.

name:

The name of the module, same as the name of the directory and script.

shortdesc:

A short description of the module, ideally not more than one sentence long. This is used in website metadata or anywhere a shorter module description is needed.

example:

A YAML string of example configuration showcasing the configuration options available with inline documentation to describe them.