Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
9b5a6f7
feat: Add Python-based subaction to resolve Nextflow version
MillironX May 29, 2026
db43658
feat: Add Python-based subation to install Nextflow to the Tool Cache
MillironX Jun 1, 2026
092c1aa
feat: Add Python based subaction to add secrets
MillironX Jun 1, 2026
d7abc70
fix: Nextflow version number in Java version switch
MillironX Jun 1, 2026
e06b013
refactor: Make main action use new Python subactions
MillironX Jun 1, 2026
2b5d2e3
ci: Update example usage with new version names and remove npm calls
MillironX Jun 1, 2026
20261a6
fix: Use bash execution of full script path
MillironX Jun 1, 2026
70c3e23
fix: Explicitly add GITHUB_TOKEN to environment
MillironX Jun 1, 2026
a756b92
fix: Move scope of GITHUB_TOKEN up
MillironX Jun 1, 2026
59552df
fix: Token syntax
MillironX Jun 1, 2026
170c608
fix: Logic for set Java environment variable
MillironX Jun 1, 2026
07c77f9
fix: Add executable bit to install-nextflow Python script
MillironX Jun 1, 2026
071009e
fix: Pass outer Java Install to inner Nextflow action
MillironX Jun 1, 2026
903146c
fix: Output the install path from install script
MillironX Jun 1, 2026
2073701
fix: Java version cutoff
MillironX Jun 1, 2026
c80d098
fix: Version number and release time not matching
MillironX Jun 1, 2026
7bf7023
Revert "fix: Pass outer Java Install to inner Nextflow action"
MillironX Jun 1, 2026
b3156ef
feat!: Remove all JavaScript
MillironX Jun 1, 2026
dbb6727
meta: Add nf-core dotfiles for Python
MillironX Jun 1, 2026
ce770ec
refactor: Move subactions to recommended directory
MillironX Jun 1, 2026
b2ff0dc
meta: Update gitignore for Python
MillironX Jun 1, 2026
1463093
feat: Add action outputs
MillironX Jun 1, 2026
b71d7e6
chore: Update CHANGELOG
MillironX Jun 1, 2026
0a49f9e
docs: Update documentation on breaking changes
MillironX Jun 1, 2026
2fee6c3
test: Add doctest for `split_version`
MillironX Jun 2, 2026
bddca21
refactor: Split API call and filter into separate functions
MillironX Jun 2, 2026
848b18e
refactor: Defer execution into `main` function
MillironX Jun 2, 2026
21c63db
test: Add doctest for `get_latest_version_string`
MillironX Jun 2, 2026
fe71c48
feat: Add Java 25 to Java version map
MillironX Jun 2, 2026
ac9ea59
test: Add doctests for `java_version`
MillironX Jun 2, 2026
70dbf90
meta: Add explanation for lack of tests to non-testable files
MillironX Jun 2, 2026
eb4afff
ci: Add unit testing back in
MillironX Jun 2, 2026
a5d720f
docs: Update Pull Request Template
MillironX Jun 2, 2026
a1f71b3
reafactor: Collapse subactions into single action
MillironX Jun 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"$schema": "https://raw.githubusercontent.com/devcontainers/spec/main/schemas/devContainer.schema.json",
"name": "nfcore",
"image": "nfcore/devcontainer:dev",

"remoteEnv": {
// Workspace path on the host for mounting with docker-outside-of-docker
"LOCAL_WORKSPACE_FOLDER": "${localWorkspaceFolder}"
},

"onCreateCommand": "./.devcontainer/setup.sh",

"remoteUser": "root",
"privileged": true,

"hostRequirements": {
"cpus": 4,
"memory": "16gb",
"storage": "32gb"
}
}
23 changes: 23 additions & 0 deletions .devcontainer/setup.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/usr/bin/env bash

# Customise the terminal command prompt
echo "export PROMPT_DIRTRIM=2" >> $HOME/.bashrc
echo "export PS1='\[\e[3;36m\]\w ->\[\e[0m\\] '" >> $HOME/.bashrc
export PROMPT_DIRTRIM=2
export PS1='\[\e[3;36m\]\w ->\[\e[0m\\] '

# Update Nextflow
nextflow self-update

# Install act
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/nektos/act/master/install.sh | bash

# Install the GitHub CLI (mostly for getting API tokens)
conda install gh --channel conda-forge

# Install pre-commit hooks
pip install prek
prek install --install-hooks

# Update welcome message
echo "Welcome to the nf-core/setup-nextflow devcontainer!" > /usr/local/etc/vscode-dev-containers/first-run-notice.txt
1 change: 0 additions & 1 deletion .gitattributes

This file was deleted.

161 changes: 84 additions & 77 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,27 +8,76 @@ Many thanks for taking an interest in improving nf-core/setup-nextflow.

## Project architecture

`setup-nextflow` is a [Composite GitHub Action](https://docs.github.com/en/actions/sharing-automations/creating-actions/creating-a-composite-action) that contains two steps:
`setup-nextflow` is a [Composite GitHub Action](https://docs.github.com/en/actions/sharing-automations/creating-actions/creating-a-composite-action) that performs four steps:

1. A reference to [actions/setup-java](https://github.com/actions/setup-java) to install an up-to-date JVM
2. A Node.js script (bundled in `dist/index.js`) to install Nextflow itself
1. It resolves an ambiguous "recent" tag like `stable` or `edge` to a concrete Nextflow version number with the correct version of Java to support it
2. It installs the correct version of Java using [actions/setup-java](https://github.com/actions/setup-java)
3. It installs and caches the version of Nextflow found in Step 1
4. It (optionally) sets up any provided [Nextflow secrets](https://docs.seqera.io/nextflow/secrets)

The TypeScript source code is compiled and bundled into `dist/index.js` using `@vercel/ncc`. The composite action runs this bundled JavaScript directly.
Steps 1, 3, and 4 are self-contained Python scripts. Steps 1, 2, and 3 are separated because they form a set of sequential dependencies, while Step 4 is separated as it is a completely separate functionality.

`setup-nextflow` depends on a JSON pseudo-API hosted on the nf-core website at <https://nf-co.re/nextflow_version>.
`nextflow_version` is updated twice a day via a [workflow within the nf-core/website repo](https://github.com/nf-core/website/blob/main/.github/workflows/build-json-files-and-md-cache.yml) to be up-to-date with Nextflow releases retrieved from the GitHub API.
Requests are proxied through the nf-core website in order to avoid hitting rate limits when accessing the GitHub API on every workflow run with this action.
Step 1 uses the GitHub API to retrieve a list of the 30 most recent Nextflow versions to determine which should be installed. However, it only accesses the API if and only if an ambiguous tag (`stable` or `edge`) is used, and only makes 1 call. This is to help reduce API calls and avoid hitting rate limits.

## Development tools

To run and test this project locally, you will need:

1. [Docker](https://docs.docker.com/engine/install/)
2. [Act](https://nektosact.com/installation/index.html)
3. [NodeJS](https://nodejs.org/en/download)
3. [Python](https://www.python.org/downloads/)
4. [Ruff](https://github.com/astral-sh/ruff)
5. [Prek](https://github.com/j178/prek)
6. [gh cli](https://cli.github.com/)

[Another engine that is compatible with the Docker Engine API may be used by act](https://nektosact.com/usage/custom_engine.html), but since feature sets and environment are very different between engines, please ensure that failures are reproducible using vendored Docker before opening an issue.

The entrypoint scripts are all written in Python, and designed to have no external dependencies, i.e. they can be run using any recent (>3.4) version of Python without installing packages.

In order to run the entrypoints, you will need to bootstrap environment variables found in the GitHub Actions runners. To do so, run

```bash
source boostrap.sh
```

from the repository root. This will set all Runner variables to temporary files or directories, and will populate a GitHub Token from the gh cli.

## Code formatting

### Ruff

All Python code in nf-core/setup-nextflow must be passed through the [Ruff code linter and formatter](https://github.com/astral-sh/ruff). This ensures a harmonised code formatting style throughout the codebase, from all contributors.

You can run Ruff on the command line (it's included in the dev dependencies) - eg. to run recursively on the whole repository:

```bash
ruff format .
```

Alternatively, Ruff has [integrations for most common editors](https://github.com/astral-sh/ruff-lsp) and VSCode(https://github.com/astral-sh/ruff-vscode) to automatically format code when you hit save.

There is an automated CI check that runs when you open a pull-request to nf-core/setup-nextflow that will fail if any code does not adhere to Ruff formatting.

Ruff has been adopted for linting and formatting in replacement of Black, isort (for imports) and pyupgrade. It also includes Flake8.

### pre-commit hooks

This repository comes with pre-commit hooks for ruff and Prettier, managed by [prek](https://github.com/j178/prek). Pre-commit hooks automatically run checks before a commit is committed into the git history. If all checks pass, the commit is made, if files are changed by the pre-commit hooks, the user is informed and has to stage the changes and attempt the commit again.

You can use the pre-commit hooks if you like, but you don't have to. The CI on Github will run the same checks as the tools installed with prek. If the pre-commit checks pass, then the same checks in the CI will pass, too.

You can install the pre-commit hooks into the development environment by running the following command in the root directory of the repository.

```bash
prek install --install-hooks
```

You can also run all pre-commit hooks without making a commit:

```bash
prek run --all-files
```

## Contribution workflow

If you'd like to write some code for nf-core/setup-nextflow, the standard workflow is as follows:
Expand All @@ -40,97 +89,55 @@ If you'd like to write some code for nf-core/setup-nextflow, the standard workfl

If you're not used to this workflow with git, you can start with some [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests) or even their [excellent `git` resources](https://try.github.io/).

If you are not familiar with a workflow with npm, know that you will need to run `npm ci` within the repository anytime you clone the repo, and after pulling any changes from GitHub.

## Unit Tests

Unit tests of JavaScript functions are written using the [ava test runner framework](https://github.com/avajs/ava).
`ava` will be setup for you upon running `npm ci`.
If you add new code, you should also add additional tests within the `test/` directory.
You should test your changes locally by running your code against these unit tests.
Execute all the tests with the following command:
Unit tests of deterministic Python functions are written using [Python doctests](https://docs.python.org/3/library/doctest.html). When adding new code, please try to consider if you can isolate the logic (`if` statements and flow) from the environment (API calls, file writes, etc.), and write doctests for the logic functions.

Execute doctests with the following command for each testable script file:

```bash
npm test
python -m doctest ./path/to/script/file.py -v
```

When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.
Some files do not contain unit tests, as we do not implement stubs/mocks/etc. for testing non-deterministic code.

When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests. Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.

## Integration Tests

A mock workflow is contained with the [workflows/example.yml](https://github.com/nf-core/setup-nextflow/tree/master/.github/workflows/example.yml) that can be used to test the end result of the GitHub Action within a GitHub Action-like environment.
`act` is configured via the [.actrc](https://github.com/nf-core/setup-nextflow/tree/master/.actrc) to pull the correct Docker images for bootstrapping a GitHub Actions Environment automatically.
A mock workflow is contained with the [workflows/example.yml](../.github/workflows/example.yml) that can be used to test the end result of the GitHub Action within a GitHub Action-like environment.
`act` is configured via the [.actrc](..//.actrc) file to pull the correct Docker images for bootstrapping a GitHub Actions Environment automatically.
You should test your changes locally by running your code through the integration tests via `act`.
Execute the integration tests with the following command:

```bash
act -j example-usage
act -j example-usage -s GITHUB_TOKEN="$(gh auth token)"
```

> [!NOTE] > `.actrc` is configured to work correctly on x86_64/amd64 Linux machines and Macs, and Apple Silicon Macs.
> The configuration is known to not function correctly on arm64 Linux machines (e.g. Asahi Linux).
> The configuration is known to not function correctly on arm64 Linux machines (e.g. Asahi Linux or Raspberry Pi).
> In order to run tests on an arm64 Linux machine, remove the `--container-architecture linux/amd64` line from `.actrc` temporarily.

## Lint tests
## GitHub Codespaces

`nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which all pipelines must adhere to.
As `setup-nextflow` is not a pipeline, those guidelines have limited applicability here, but we have opted to adapt the linting guidelines over to JavaScript by implementing both [ESLint](https://eslint.org/) and [Prettier](https://prettier.io/).
Linting should happen automatically before each git commit thanks to [Husky](https://typicode.github.io/husky/) pre-commit hooks that are installed when you `npm ci`.
If you are not able to commit your changes, try running
This repo includes a devcontainer configuration which will create a GitHub Codespaces for Nextflow development! This is an online developer environment that runs in your browser, complete with VSCode and a terminal.

```bash
npm run format
npm run lint:fix
```
To get started:

- Open the repo in [Codespaces](https://github.com/nf-core/tools/codespaces)
- Tools installed
- nf-core
- Nextflow
- Docker
- Act
- Python
- Prek
- gh

to have these tools fix the code formatting for you.

## Build process

GitHub Actions does not have an automated system for building and releasing actions written in TypeScript.[^1]
Technically speaking, a published GitHub action is not actually a package, but simply a git reference to a complete JavaScript execution tree with an `action.yml` file.
As such, the process for releasing a new version of `nf-core/setup-nextflow` is as follows:

1. Clone the `nf-core/setup-nextflow` repository locally - releases cannot be made from a fork
2. Bump the version number within `CHANGELOG.md`, `package.json`, and `package-lock.json`
3. Commit those changes to a branch of the name `release/vX.Y.Z`
4. Push that branch to GitHub (`git push -u origin release/vX.Y.Z`) and merge as a regular pull request
5. Back in the local repo, ensure you still have the `release/vX.Y.Z` branch checked out
6. Checkout an orphan branch of the name `build/vX.Y.Z` (`git checkout --orphan build/vX.Y.Z`). All changes will be untracked and unstaged.
7. Run `npm ci` and `npm run all`
8. Commit the following files/directories:
- dist
- docs
- subaction
- CHANGELOG.md
- LICENSE
- README.md
- action.yml
9. Delete all untracked files
10. Remove major and major.minor version tags for this version from the remote repository
```bash
git push origin :vX
git push origin :vX.Y
```
11. Tag the current commit with the major, major.minor, and major.minor.patch semantic versions. If git prompts you for a tag message, use the full major.minor.patch version string.
```bash
git tag vX
git tag vX.Y
git tag vX.Y.Z
```
12. Push the tags to GitHub (`git push --tags`)
13. (Optional) In the GitHub interface, create a [release](https://github.com/nf-core/setup-nextflow/releases) assigned to the tag with content copied from the CHANGELOG, and ensure it is published to the GitHub Marketplace
14. Delete the release and build branch within the local repo
```bash
git checkout master
git pull
git branch -d release/vX.Y.Z
git branch -D build/vX.Y.Z
```
Devcontainer specs:

- [DevContainer config](../.devcontainer/devcontainer.json)

## Getting help

For further information/help, please don't hesitate to get in touch on the nf-core Slack [#tools](https://nfcore.slack.com/channels/tools) channel ([join our Slack here](https://nf-co.re/join/slack)).

[^1]: [JasonEtco/build-and-tag-action](https://github.com/JasonEtco/build-and-tag-action) exists but does not function for TypeScript actions. See <https://github.com/JasonEtco/build-and-tag-action/issues/20>
7 changes: 3 additions & 4 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,8 @@ Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/setu

- [ ] This comment contains a description of changes (with reason)
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] Make sure your code lints (`npm run lint`)
- [ ] Make sure your code passes formatting checks (`npm run format:check`)
- [ ] Ensure the unit test suite passes (`npm run test`)
- [ ] Ensure the integration test suite passes (`act -j example-usage`)
- [ ] Make sure your code lints and passes formatting checks (`ruff check`)
- [ ] Ensure the unit test suite passes (`python -m doctest src/find-nextflow-version.py -v`)
- [ ] Ensure the integration test suite passes (`act -j example-usage -s GITHUB_TOKEN="$(gh auth token)"`)
- [ ] `CHANGELOG.md` is updated
- [ ] `README.md` is updated
14 changes: 3 additions & 11 deletions .github/renovate.json5
Original file line number Diff line number Diff line change
@@ -1,17 +1,9 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
$schema: "https://docs.renovatebot.com/renovate-schema.json",
extends: [
"github>nf-core/ops//.github/renovate/default.json5",
"config:recommended",
"schedule:weekly",
":automergeBranch"
":automergeBranch",
],
"packageRules": [
{
"matchManagers": ["npm"],
"automerge": true,
"matchUpdateTypes": ["minor", "patch", "pin", "digest"],
"schedule": ["every weekday before 5am"]
}
]
}
45 changes: 6 additions & 39 deletions .github/workflows/example.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,47 +19,26 @@ jobs:
fail-fast: false
matrix:
nextflow_version:
- "21.10.3"
- "22.04"
- "22.03.1-edge"
- "24.10.1"
- "latest"
- "latest-stable"
- "latest-edge"
- "latest-everything"
all_distribution:
- true
- false
- "stable"
- "edge"
- "18.10.1" # This is the oldest version we support
- "24.10.6" # This is on one side of the Java version switch
- "24.11.0-edge" # This is on the other side

steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: 24
cache: "npm"
- run: npm ci
- run: npm run build
- run: npm run package
- uses: ./
with:
version: ${{ matrix.nextflow_version }}
all: ${{ matrix.all_distribution }}
- run: nextflow -v

example-secrets:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: 24
cache: "npm"
- run: npm ci
- run: npm run build
- run: npm run package
- uses: ./
with:
version: "latest-stable"
version: "stable"
secrets: |
MY_TEST_SECRET=hello_world
ANOTHER_SECRET=foo_bar
Expand All @@ -73,17 +52,5 @@ jobs:
steps:
- uses: easimon/maximize-build-space@fc881a613ad2a34aca9c9624518214ebc21dfc0c # v10
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: 24
cache: "npm"
- run: npm ci
- run: npm run build
- run: npm run package
- uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5
if: ${{ env.ACT}}
with:
java-version: 21
distribution: temurin
- uses: ./
- run: nextflow -v
17 changes: 17 additions & 0 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: nf-core linting
# This workflow is triggered on pushes and PRs to the repository.
# It runs the pre-commit hooks server-side to ensure
# that the code meets the nf-core guidelines.
on:
pull_request:
release:
types: [published]

jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5

- name: Run prek
uses: j178/prek-action@v2
Loading
Loading