Repository navigation
DEV-1358: MATLAB update PR asks for a hardware check and says merging releases #22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # One-click release of the Java API: GitHub Packages, a GitHub Release, and the MATLAB jar. | |
| # | |
| # How to release | |
| # 0. On the bench, at the commit to release: tools/hil/run-java-hil.ps1 -Report (tools/hil/README.md). | |
| # It posts the commit status "java-hil"; the build job stops unless that passed, or hil_waiver | |
| # gives a Jira key (the waiver goes into the release notes). Dry runs only warn. | |
| # 1. Actions -> "Release Java API" -> Run workflow (on master). Leave `version` blank for the next | |
| # patch after the last API_v release (0.11.8_beta -> 0.11.9_beta), or type one. | |
| # 2. Run a MATLAB example on a Shimmer3 and a Shimmer3R with the ShimmerJavaClass.jar PR it opens in | |
| # Shimmer-MATLAB-ID, then merge it. Merging publishes a MATLAB API release (that repo's release.yml). | |
| # 3. Run "Release Android API" in ShimmerAndroidAPI, which builds against this version by default. | |
| # | |
| # The version is not kept in the source. The build files read it from -PreleaseVersion, which this | |
| # workflow passes, and say 'local' otherwise; the API_v tags record what was released. | |
| # | |
| # One run does, in order, and creates nothing public until the build and tests have passed: | |
| # build picks the version; fails if API_v<version> is already tagged; tests ShimmerDriver and builds | |
| # ShimmerDriverPC exactly as gradle.yml does. Read-only: it runs on PRs too. | |
| # publish real releases only, and the only job with packages:write. Publishes shimmerdriver, | |
| # shimmerbluetoothmanager and shimmerdriverpc together with one `gradlew publish` | |
| # (DEV-1115: publishing them one at a time by hand left shimmerbluetoothmanager | |
| # 0.11.8_beta unpublished), and keeps the jars it published for the release | |
| # matlab-jar builds ShimmerJavaClass.jar as Java 8 bytecode (`shadowJar -PmatlabJava8`), on a | |
| # separate runner so its Java 8 classes cannot end up in the published jars | |
| # release tags API_v<version> on the commit it built and creates the GitHub Release, with | |
| # notes from the previous API_v release and the jars attached. It is not marked | |
| # "Latest", which stays with the Consensys releases in this repo. | |
| # update-matlab opens a PR in Shimmer-MATLAB-ID replacing libs/ShimmerJavaClass.jar; merging it | |
| # publishes a MATLAB API release. Needs the MATLAB_REPO_TOKEN secret: a fine-grained | |
| # token for ShimmerResearch/Shimmer-MATLAB-ID with Contents and Pull requests read/write. | |
| # | |
| # A failed job can be re-run on its own with "Re-run failed jobs"; the jobs before it are not | |
| # repeated. A failed publish can leave some of the three versions in GitHub Packages: delete those | |
| # versions under the repo's Packages before re-running, or bump the version. | |
| # | |
| # dry_run runs build and matlab-jar without publishing, from any branch; a blank version still picks | |
| # the next one, so the run shows what a real release would be called. The workflow also | |
| # dry-runs on any PR that changes this file. | |
| name: Release Java API | |
| on: | |
| workflow_dispatch: | |
| inputs: | |
| version: | |
| description: 'Version, e.g. 0.12.0_beta. Blank = next patch after the last release' | |
| type: string | |
| required: false | |
| dry_run: | |
| description: 'Build and test only; publish nothing' | |
| type: boolean | |
| default: false | |
| hil_waiver: | |
| description: 'Jira key to release without a passing hardware test (HIL), e.g. DEV-1234. Blank = require one' | |
| type: string | |
| required: false | |
| pull_request: | |
| paths: | |
| - '.github/workflows/release.yml' | |
| concurrency: | |
| group: release-java-api | |
| cancel-in-progress: false | |
| jobs: | |
| build: | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 30 # a hung Gradle daemon otherwise holds the runner for 6 hours | |
| # Read-only on purpose: this job also runs on PRs, and any code in a job can recover that | |
| # job's token. Only the publish job, which runs on real releases alone, can write packages. | |
| permissions: | |
| contents: read | |
| statuses: read # the hardware test result the bench posts on the commit | |
| env: | |
| DRY_RUN: ${{ github.event_name != 'workflow_dispatch' || inputs.dry_run }} | |
| outputs: | |
| version: ${{ steps.version.outputs.version }} | |
| tag: ${{ steps.version.outputs.tag }} | |
| previous_tag: ${{ steps.version.outputs.previous_tag }} | |
| hil: ${{ steps.hil.outputs.hil }} | |
| steps: | |
| - name: Release from master only | |
| if: env.DRY_RUN != 'true' && github.ref != 'refs/heads/master' | |
| run: | | |
| echo "::error::A release must run on master, not ${GITHUB_REF_NAME}. Tick dry_run to try another branch." | |
| exit 1 | |
| - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 | |
| with: | |
| fetch-depth: 0 # tags, to refuse a version that is already released | |
| # Keep the token out of .git/config, where the Gradle build (and any plugin or dependency | |
| # it runs) could read it. Nothing here needs git credentials after checkout; gh uses GH_TOKEN. | |
| persist-credentials: false | |
| - name: Choose the version | |
| id: version | |
| env: | |
| GH_TOKEN: ${{ github.token }} | |
| REQUESTED: ${{ inputs.version }} | |
| run: | | |
| PREVIOUS_TAG=$(gh release list --repo "$GITHUB_REPOSITORY" --limit 200 --json tagName,createdAt \ | |
| --jq '[.[] | select(.tagName | startswith("API_v"))] | sort_by(.createdAt) | last | .tagName // ""') | |
| if [ -n "$REQUESTED" ]; then | |
| VERSION=$REQUESTED | |
| else | |
| # Next patch after the last release, keeping its suffix: 0.11.8_beta -> 0.11.9_beta. | |
| last=${PREVIOUS_TAG#API_v} | |
| if ! [[ "$last" =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)(.*)$ ]]; then | |
| echo "::error::Cannot work out the next version from '${PREVIOUS_TAG:-no previous release}'. Type one in the version box." | |
| exit 1 | |
| fi | |
| VERSION="${BASH_REMATCH[1]}.${BASH_REMATCH[2]}.$((BASH_REMATCH[3] + 1))${BASH_REMATCH[4]}" | |
| fi | |
| if ! [[ "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+[A-Za-z0-9_.-]*$ ]]; then | |
| echo "::error::'$VERSION' is not a version like 0.11.9_beta." | |
| exit 1 | |
| fi | |
| TAG="API_v$VERSION" | |
| if git rev-parse -q --verify "refs/tags/$TAG" >/dev/null; then | |
| if [ "$DRY_RUN" = true ]; then | |
| echo "::warning::$TAG already exists, so a real release would stop here. Dry run continues." | |
| else | |
| echo "::error::$TAG already exists. Leave the version blank for the next one, or type another." | |
| exit 1 | |
| fi | |
| fi | |
| { | |
| echo "version=$VERSION" | |
| echo "tag=$TAG" | |
| echo "previous_tag=$PREVIOUS_TAG" | |
| } >> "$GITHUB_OUTPUT" | |
| echo "Releasing $VERSION as $TAG (notes since ${PREVIOUS_TAG:-the first commit})" | |
| - name: Check the hardware test (HIL) result | |
| id: hil | |
| env: | |
| GH_TOKEN: ${{ github.token }} | |
| WAIVER: ${{ inputs.hil_waiver }} | |
| run: | | |
| context=java-hil | |
| if [ -n "$WAIVER" ] && ! [[ "$WAIVER" =~ ^[A-Z][A-Z0-9]+-[0-9]+$ ]]; then | |
| echo "::error::hil_waiver must be a Jira key such as DEV-1234, not '$WAIVER'." | |
| exit 1 | |
| fi | |
| # Newest first: the latest result for this context on this exact commit decides. | |
| line=$(gh api "repos/$GITHUB_REPOSITORY/commits/$GITHUB_SHA/statuses" --paginate \ | |
| --jq '.[] | [.context, .state, .description] | @tsv' | awk -F'\t' -v c="$context" '$1 == c { print; exit }') | |
| state=$(printf '%s' "$line" | cut -f2) | |
| description=$(printf '%s' "$line" | cut -f3) | |
| if [ "$state" = success ]; then | |
| echo "Hardware test passed on $GITHUB_SHA: $description" | |
| echo "hil=Tested on hardware before release: $description." >> "$GITHUB_OUTPUT" | |
| elif [ -n "$WAIVER" ]; then | |
| echo "::warning::No passing '$context' on $GITHUB_SHA (found: ${state:-none}); released under waiver $WAIVER." | |
| echo "hil=Released without a passing hardware test (HIL), waived under $WAIVER." >> "$GITHUB_OUTPUT" | |
| elif [ "$DRY_RUN" = true ]; then | |
| echo "::warning::No passing '$context' on $GITHUB_SHA (found: ${state:-none}). A real release would stop here." | |
| else | |
| echo "::error::No passing '$context' on $GITHUB_SHA (found: ${state:-none}${description:+: $description}). On the bench, at this commit, run: tools/hil/run-java-hil.ps1 -Report. Then run this workflow again, or give a Jira key in hil_waiver." | |
| exit 1 | |
| fi | |
| - name: Set up JDK 11 | |
| uses: actions/setup-java@cf277c60eb25467037889841efdb72551f06f6c3 # v4.9.1 | |
| with: | |
| java-version: '11' | |
| distribution: 'temurin' | |
| # Same pinned action as gradle.yml, and likewise no gradle-version: build through the | |
| # committed wrapper. | |
| - name: Setup Gradle | |
| uses: gradle/actions/setup-gradle@ec92e829475ac0c2315ea8f9eced72db85bb337a # v3.0.0 | |
| - name: Test ShimmerDriver | |
| working-directory: ShimmerDriver | |
| run: ./gradlew test | |
| # ShimmerDriverPC includes :ShimmerDriver and :ShimmerBluetoothManager as subprojects. | |
| - name: Build ShimmerDriverPC | |
| working-directory: ShimmerDriverPC | |
| run: ./gradlew build | |
| publish: | |
| needs: build | |
| if: github.event_name == 'workflow_dispatch' && !inputs.dry_run | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 30 # a hung Gradle daemon otherwise holds the runner for 6 hours | |
| permissions: | |
| contents: read | |
| packages: write | |
| env: | |
| VERSION: ${{ needs.build.outputs.version }} | |
| steps: | |
| - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 | |
| with: | |
| persist-credentials: false # publish gets its token explicitly, below | |
| - name: Set up JDK 11 | |
| uses: actions/setup-java@cf277c60eb25467037889841efdb72551f06f6c3 # v4.9.1 | |
| with: | |
| java-version: '11' | |
| distribution: 'temurin' | |
| - name: Setup Gradle | |
| uses: gradle/actions/setup-gradle@ec92e829475ac0c2315ea8f9eced72db85bb337a # v3.0.0 | |
| # `publish` runs in every project of the ShimmerDriverPC build, so all three artifacts go | |
| # out in one invocation, and -PreleaseVersion versions all three (and their POM | |
| # dependencies on each other). It builds the jars but does not re-run the tests, which the | |
| # build job ran on this same commit. The build files read the credentials from USERNAME and TOKEN. | |
| - name: Publish to GitHub Packages | |
| working-directory: ShimmerDriverPC | |
| env: | |
| USERNAME: ${{ github.actor }} | |
| TOKEN: ${{ github.token }} | |
| run: ./gradlew publish -PreleaseVersion="$VERSION" | |
| # The jars just published, so the GitHub Release carries the same files as GitHub Packages. | |
| - name: Collect the published jars | |
| run: | | |
| mkdir -p release-assets | |
| cp ShimmerDriver/build/libs/ShimmerDriver.jar "release-assets/shimmerdriver-$VERSION.jar" | |
| cp ShimmerBluetoothManager/build/libs/ShimmerBluetoothManager.jar "release-assets/shimmerbluetoothmanager-$VERSION.jar" | |
| cp ShimmerDriverPC/build/libs/ShimmerDriverPC.jar "release-assets/shimmerdriverpc-$VERSION.jar" | |
| ls -l release-assets | |
| - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 | |
| with: | |
| name: release-jars | |
| path: release-assets/ | |
| if-no-files-found: error | |
| matlab-jar: | |
| needs: build | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 30 # a hung Gradle daemon otherwise holds the runner for 6 hours | |
| permissions: | |
| contents: read | |
| steps: | |
| - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 | |
| with: | |
| persist-credentials: false # the Gradle build needs no git credentials | |
| - name: Set up JDK 11 | |
| uses: actions/setup-java@cf277c60eb25467037889841efdb72551f06f6c3 # v4.9.1 | |
| with: | |
| java-version: '11' | |
| distribution: 'temurin' | |
| - name: Setup Gradle | |
| uses: gradle/actions/setup-gradle@ec92e829475ac0c2315ea8f9eced72db85bb337a # v3.0.0 | |
| - name: Build ShimmerJavaClass.jar for MATLAB (Java 8 bytecode) | |
| working-directory: ShimmerDriverPC | |
| run: ./gradlew shadowJar -PmatlabJava8 | |
| # Every class in the jar, dependencies included, since a dependency built for a later Java | |
| # breaks MATLAB just as surely as our own code would (~32,500 classes, about 2 s). | |
| - name: Check every class in the jar is Java 8 bytecode | |
| run: | | |
| jar=ShimmerDriverPC/build/libs/ShimmerJavaClass.jar | |
| python3 - "$jar" <<'EOF' | |
| import struct, sys, zipfile | |
| MAIN = "com/shimmerresearch/tools/matlab/ShimmerJavaClass.class" | |
| jar = sys.argv[1] | |
| checked, bad = 0, [] | |
| with zipfile.ZipFile(jar) as z: | |
| names = z.namelist() | |
| if MAIN not in names: | |
| print(f"::error::{jar} does not contain {MAIN}.") | |
| sys.exit(1) | |
| for name in names: | |
| # Java 8 never loads module descriptors or multi-release entries for later Javas. | |
| if (not name.endswith(".class") or name.startswith("META-INF/versions/") | |
| or name.endswith("module-info.class")): | |
| continue | |
| with z.open(name) as f: | |
| head = f.read(8) | |
| checked += 1 | |
| if len(head) < 8 or head[:4] != b"\xca\xfe\xba\xbe": | |
| bad.append((name, "not a class file")) | |
| elif struct.unpack(">H", head[6:8])[0] > 52: | |
| bad.append((name, f"class version {struct.unpack('>H', head[6:8])[0]}")) | |
| print(f"Checked {checked} classes; {len(bad)} newer than Java 8 (class version 52).") | |
| for name, why in bad[:20]: | |
| print(f"::error::{name}: {why}. MATLAB on Java 8 cannot load it.") | |
| if len(bad) > 20: | |
| print(f"::error::...and {len(bad) - 20} more.") | |
| sys.exit(1 if bad else 0) | |
| EOF | |
| mkdir -p matlab-jar | |
| cp "$jar" matlab-jar/ShimmerJavaClass.jar | |
| ls -l matlab-jar | |
| - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 | |
| with: | |
| name: matlab-jar | |
| path: matlab-jar/ | |
| if-no-files-found: error | |
| release: | |
| needs: [build, publish, matlab-jar] | |
| if: github.event_name == 'workflow_dispatch' && !inputs.dry_run | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 10 | |
| permissions: | |
| contents: write # create the tag and the release | |
| env: | |
| GH_TOKEN: ${{ github.token }} | |
| VERSION: ${{ needs.build.outputs.version }} | |
| TAG: ${{ needs.build.outputs.tag }} | |
| PREVIOUS_TAG: ${{ needs.build.outputs.previous_tag }} | |
| HIL: ${{ needs.build.outputs.hil }} | |
| steps: | |
| - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 | |
| with: | |
| name: release-jars | |
| path: release-assets | |
| - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 | |
| with: | |
| name: matlab-jar | |
| path: matlab-jar | |
| - name: Create the tag and GitHub Release | |
| run: | | |
| cp matlab-jar/ShimmerJavaClass.jar "release-assets/ShimmerJavaClass-$VERSION.jar" | |
| # The generated notes, with the hardware test result (or its waiver) in front. | |
| args=(-f tag_name="$TAG" -f target_commitish="$GITHUB_SHA") | |
| if [ -n "$PREVIOUS_TAG" ]; then args+=(-f previous_tag_name="$PREVIOUS_TAG"); fi | |
| { | |
| if [ -n "$HIL" ]; then | |
| echo "$HIL" | |
| echo | |
| fi | |
| gh api "repos/$GITHUB_REPOSITORY/releases/generate-notes" "${args[@]}" --jq .body | |
| } > notes.md | |
| gh release create "$TAG" release-assets/* \ | |
| --repo "$GITHUB_REPOSITORY" \ | |
| --target "$GITHUB_SHA" \ | |
| --title "Shimmer Java API Rev $VERSION" \ | |
| --notes-file notes.md \ | |
| --latest=false | |
| { | |
| echo "### Released Java API $VERSION" | |
| echo "- Release: $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/releases/tag/$TAG" | |
| echo "- Packages: com.shimmerresearch:shimmerdriver, shimmerbluetoothmanager and shimmerdriverpc at $VERSION" | |
| echo "- ${HIL:-No hardware test result}" | |
| echo "- Next: run a MATLAB example on a Shimmer3 and a Shimmer3R and merge the MATLAB PR, which publishes a MATLAB API release; then run Release Android API in ShimmerAndroidAPI; it builds against $VERSION unless you pick another driver version." | |
| } >> "$GITHUB_STEP_SUMMARY" | |
| update-matlab: | |
| needs: [build, release] | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 10 | |
| permissions: {} # writes to Shimmer-MATLAB-ID with MATLAB_REPO_TOKEN, not this repo's token | |
| env: | |
| VERSION: ${{ needs.build.outputs.version }} | |
| TAG: ${{ needs.build.outputs.tag }} | |
| MATLAB_REPO: ShimmerResearch/Shimmer-MATLAB-ID | |
| BRANCH: ShimmerJavaClass_${{ needs.build.outputs.version }} | |
| HAS_TOKEN: ${{ secrets.MATLAB_REPO_TOKEN != '' }} | |
| steps: | |
| - name: Check for MATLAB_REPO_TOKEN | |
| if: env.HAS_TOKEN != 'true' | |
| run: | | |
| echo "::error::Secret MATLAB_REPO_TOKEN is not set, so the Shimmer-MATLAB-ID PR cannot be opened. Add a fine-grained token for $MATLAB_REPO (Contents and Pull requests: read/write) as a repo secret, then re-run this job." | |
| exit 1 | |
| # Credentials stay persisted here on purpose: the git push below uses them. Nothing else runs | |
| # in this job. | |
| - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 | |
| with: | |
| repository: ${{ env.MATLAB_REPO }} | |
| token: ${{ secrets.MATLAB_REPO_TOKEN }} | |
| path: matlab | |
| - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 | |
| with: | |
| name: matlab-jar | |
| path: matlab-jar | |
| - name: Open the PR updating libs/ShimmerJavaClass.jar | |
| env: | |
| GH_TOKEN: ${{ secrets.MATLAB_REPO_TOKEN }} | |
| working-directory: matlab | |
| run: | | |
| existing=$(gh pr list --repo "$MATLAB_REPO" --head "$BRANCH" --state open --json url --jq '.[0].url // ""') | |
| if [ -n "$existing" ]; then | |
| echo "PR already open: $existing" | |
| echo "- MATLAB PR: $existing" >> "$GITHUB_STEP_SUMMARY" | |
| exit 0 | |
| fi | |
| cp ../matlab-jar/ShimmerJavaClass.jar libs/ShimmerJavaClass.jar | |
| if git diff --quiet -- libs/ShimmerJavaClass.jar; then | |
| echo "libs/ShimmerJavaClass.jar is already this build; no PR needed." | |
| exit 0 | |
| fi | |
| git config user.name "github-actions[bot]" | |
| git config user.email "41898282+github-actions[bot]@users.noreply.github.com" | |
| git switch -c "$BRANCH" | |
| git add libs/ShimmerJavaClass.jar | |
| git commit -m "Update ShimmerJavaClass.jar to Shimmer Java API $VERSION" \ | |
| -m "Built by $GITHUB_REPOSITORY's Release workflow from $TAG ($GITHUB_SHA), as Java 8 bytecode." | |
| git push --force origin "HEAD:refs/heads/$BRANCH" | |
| url=$(gh pr create --repo "$MATLAB_REPO" --base master --head "$BRANCH" \ | |
| --title "Update ShimmerJavaClass.jar to Shimmer Java API $VERSION" \ | |
| --body "Replaces \`libs/ShimmerJavaClass.jar\` with the build from [$TAG]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/releases/tag/$TAG) (\`$GITHUB_SHA\`), compiled as Java 8 bytecode with \`gradlew shadowJar -PmatlabJava8\`. | |
| - Before merging, run an example such as \`plotandwriteexample.m\` on a Shimmer3 and a Shimmer3R. MATLAB cannot run in CI. | |
| - Merging publishes a MATLAB API release: the next patch version, with Java API $VERSION in its title. | |
| Opened by the [Release Java API run]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID).") | |
| echo "Opened $url" | |
| echo "- MATLAB PR: $url" >> "$GITHUB_STEP_SUMMARY" |