Repository navigation
[ci] Skip benchmark PR comments on fork PRs and show results in the job summary - #1943
Open
henryqdineen wants to merge 3 commits into
Open
henryqdineen wants to merge 3 commits into
henryqdineen wants to merge 3 commits into
Conversation
…ob summary Fork PRs get a read-only GITHUB_TOKEN under `pull_request`, so the `Post comment` step 403s and fails the perf and size jobs on every external PR even though the benchmarks themselves succeed. Guard the comment step to same-repo PRs and write the comparison table to the job summary so fork PRs still surface results. Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
@henryqdineen is attempting to deploy a commit to the Meta Open Source Team on Vercel. A member of the Team first needs to authorize it. |
yarn's own banner, "$ node ./compare.js …", and "Done in …" lines were captured along with the table into the PR comment and now the job summary. Dropping the `--` also removes yarn's deprecation warning from the step log. Co-Authored-By: Claude Opus 5.5 <[email protected]>
yarn 1 forwards script args without `--` and warns when it's present, so every benchmark step logged a deprecation warning. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed / motivation ?
The
perfandsizejobs inbenchmarks.ymlfail on every PR opened from a fork, so each push to a fork PR sends its author a "[facebook/stylex] PR run failed: benchmarks" email. The benchmarks run fine. The failure is the finalPost commentstep, which gets403 Resource not accessible by integration. For PRs from forks, GitHub givespull_requestworkflows a read-onlyGITHUB_TOKEN, so the step can't comment. A PR from a branch in this repo, like #1882, passes both jobs, while the PRs from my fork (#1911, #1914, #1920–#1923, #1928) all fail them.This change:
Post commentstep when the PR comes from a fork (github.event.pull_request.head.repo.full_name == github.repository).$GITHUB_STEP_SUMMARYin both jobs, so fork PRs still show results on the run's summary page.yarn -s … run -sso the table no longer includes yarn's banner,$ node ./compare.js …, andDone in …lines, and drops the unneeded--from the yarn calls so the steps stop logging yarn's deprecation warning.PRs from branches in this repo work as before. The jobs never failed on regressions, since
compare.jsonly flags them with**!!**, so this removes no gate. A crash in the benchmarks on the PR's code still fails the job.Linked PR/Issues
Follow-up to #1400. It added
permissions: pull-requests: write, which can't fix fork PRs: GitHub doesn't grant write access to fork PRs through thepermissionskey.Additional Context
Other repos use the same guard together with a step summary:
pacquet-micro-benchmark.yml: the samefind-commentandcreate-or-update-commentsetup, with the sameif:on both comment stepscode-size.ymlbundle-size.ymlIf we want the comment on fork PRs too, the safe next step is the two-workflow split that facebook/react's sizebot uses. The
pull_requestworkflow uploads the table as an artifact, and aworkflow_runworkflow, which gets a write token, posts it. The PR number has to come from theworkflow_runevent, not the artifact. I kept this PR small, but I'm happy to do that as a follow-up. Switching topull_request_targetisn't safe here, because these jobs runyarn installand scripts from the PR's code (GitHub Security Lab, and the same note in facebook/docusaurusbuild-perf.yml).This PR comes from a fork, so its own
perfandsizechecks will show whether the change works: they should pass, and the tables should appear on the run summary.Pre-flight checklist
Contribution Guidelines
🤖 Generated with Claude Code