From 2a1409fe88d416cc85fc96fb5bc473f83ed5a054 Mon Sep 17 00:00:00 2001 From: Kynan Ware <47394200+BagToad@users.noreply.github.com> Date: Thu, 30 Jul 2026 18:47:41 -0600 Subject: [PATCH 1/4] Merge commit from fork * Add terminal-safety mechanisms for untrusted content Introduce the building blocks for keeping untrusted external content (HTTP response bodies and the like) from reaching a terminal as live ANSI escape sequences, while leaving the application's own styled output untouched. - iostreams.Untrusted: a value type that wraps external content. The raw bytes are unexported; String sanitizes and is called automatically by fmt, so the default print path is safe. Raw and RawBytes are the explicit, greppable opt-out for non-terminal uses (disk, the API, hashing), and the type carries its label across JSON decoding so a decoded field stays marked. - iostreams.ContentOut and SetContentSanitization: a sink that sanitizes raw external streams by default and becomes a passthrough when a command opts out, so the sanitization decision is made at the moment of writing. - A CodeQL query (with help text, examples, and a per-category test suite) that flags HTTP response content reaching a terminal writer other than ContentOut without sanitization, treating ContentOut, Untrusted.String, the asciisanitizer wrap, and structured JSON decoding as the accepted resolutions. Wire it into the existing CodeQL workflow. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Migrate gist view to the terminal-safety mechanisms GetRawGistFile now returns iostreams.Untrusted, so the two callers are forced to declare intent. gist view carries the value to the render function and lets the sink decide: the raw dump goes through ContentOut (which honors --allow-escape-sequences) and the markdown path sanitizes its input with String before rendering to Out. gist edit takes Raw, because the content is opened in an editor and sent back to the API and must round-trip verbatim. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Migrate skills to the terminal-safety mechanisms FetchBlob base64-decodes blob content out of the JSON response, so it now returns iostreams.Untrusted and its callers split by intent: skills preview sanitizes for display, while the installer (writes the file to disk) and the frontmatter parsers take Raw to operate on the verbatim bytes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Migrate agent-task streaming logs to the terminal-safety mechanisms The agent-task chat completion chunk's Content and ReasoningText fields arrive over a streaming response, which is not a JSON content type, so the JSON transport sanitizer does not run on them. Type those fields as iostreams.Untrusted so provenance survives the per-line json.Unmarshal; printing them later sanitizes, and presence checks use Empty. This covers a path that flows resp.Body through json.Unmarshal to a printed field, which value dataflow cannot track but the type expresses directly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Migrate repo read-file to the terminal-safety mechanisms read-file already refuses to print terminal escape sequences by default (and opts in with --allow-escape-sequences), so it guards its own bytes. Route its two raw writes through ContentOut instead of Out, in passthrough mode, so it uses the one sink the terminal-safety query recognizes while keeping its refuse-by-default behavior. Passthrough is required here: sanitizing would corrupt binary files and strip the escapes that --allow-escape-sequences explicitly allows. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Migrate pr diff to the terminal-safety mechanisms pr diff neutralized escape sequences only when stdout was a terminal, leaving piped and redirected output raw. Neutralize by default in all modes instead: the plain and name-only paths write through ContentOut, and the colored path wraps the diff reader in the shared asciisanitizer, since colored output is always terminal-bound. Add --allow-escape-sequences to opt back into raw bytes for the non-colored paths, for example when piping a patch to another program. This also replaces the command's bespoke sanitizer with the shared asciisanitizer. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Migrate api response output to the terminal-safety mechanisms gh api copied non-JSON response bodies straight to stdout, so escape sequences in the response reached the terminal unneutralized (JSON bodies are already cleaned by the transport). Route the raw copy through ContentOut, which neutralizes escape sequences on an interactive terminal, stays raw when piped so binary payloads are not corrupted, and honors --allow-escape-sequences. Silent, verbose, and slurp writers are left in place. The colorized JSON path now sanitizes its input before jsoncolor adds color, keeping the color intact while removing the raw bytes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Migrate release download stdout to the terminal-safety mechanisms release download --output - copied asset bytes straight to stdout, so escape sequences in an asset reached the terminal unneutralized. Route the stdout write through ContentOut, which neutralizes escape sequences on an interactive terminal, stays raw when piped or redirected so binary assets are not corrupted, and honors --allow-escape-sequences. File downloads are unaffected. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Migrate codespace logs to the terminal-safety mechanisms gh codespace logs streams a remote log file over ssh by running cat or tail -f, and the shared ssh helper wires the command's stdout straight to os.Stdout, so the file's bytes reach the terminal without passing through this process and cannot be neutralized. Point the logs command's stdout at ContentOut instead: on a terminal this sanitizes escape sequences and forces the remote output through the process, while piped output stays raw so a follow stream is not buffered and saved logs keep their exact bytes. The shared helper is left untouched so the interactive ssh shell still passes control sequences through. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Build the CodeQL Go database from the shipped build The Go analysis previously relied on autobuild, which walks the whole tree and extracts the nested module under .github/codeql/queries' test fixtures as well as the main module. Both declare the same module path with their own pkg/iostreams, so the analyzer conflated the two and reported a flow through unrelated code. Switch the Go job to build-mode: manual and build with make, the same command the integration tests use. Extraction is then scoped to the packages the released gh binary compiles, which excludes the separate fixtures module, so the spurious finding is gone and the analyzed code matches what we ship. The actions job builds nothing, so it is marked build-mode: none. The query's own test suite is unchanged and still runs each fixture directory in isolation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Match guarded-content errors by kind in tests Replace the binary-error boolean with a `wantErrAs` field so the table matches typed errors with errors.As and sentinels with errors.Is, mirroring how callers detect them. Any future typed error slots in without a new field. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 53357989-fb84-4835-9eee-258d59e755e5 * Clarify that Untrusted wraps string content Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Convert Untrusted tests to testify assertions Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Collapse guarded content cases into the table test Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Match binary terminal errors with errors.AsType Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Reuse the raw gist content when writing it out Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Cover the non-truncated gist file path Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Document the refusal guarantee on CopyGuardedContent Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Clear stale content when unmarshaling JSON null Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 53357989-fb84-4835-9eee-258d59e755e5 --- .github/codeql/codeql-config.yml | 19 ++ .github/codeql/codeql-pack.lock.yml | 24 ++ .github/codeql/qlpack.yml | 9 + .../UnsanitizedResponseToTerminalBad.go | 34 +++ .../UnsanitizedResponseToTerminalGood.go | 52 ++++ .../unsanitized-response-to-terminal.md | 118 +++++++++ .../unsanitized-response-to-terminal.qhelp | 84 ++++++ .../unsanitized-response-to-terminal.ql | 239 ++++++++++++++++++ .github/codeql/tests/.gitignore | 3 + .github/codeql/tests/codeql-pack.lock.yml | 24 ++ .../unsanitized-response-to-terminal/go.mod | 8 + .../unsanitized-response-to-terminal/go.sum | 4 + .../hits_base64_after_json.go | 34 +++ .../hits_iocopy_crossfunc.go | 28 ++ .../hits_readall_intraproc.go | 19 ++ .../hits_scanner_crossfunc.go | 33 +++ .../hits_untrusted_raw.go | 22 ++ .../misses_contentout.go | 34 +++ .../misses_decoder_decode.go | 29 +++ .../misses_disk_roundtrip.go | 43 ++++ .../misses_field_after_unmarshal.go | 41 +++ .../misses_file_sink.go | 23 ++ .../misses_sanitized.go | 40 +++ .../misses_untrusted_string.go | 21 ++ .../pkg/iostreams/iostreams.go | 29 +++ .../test.expected | 77 ++++++ .../test.qlref | 1 + .../go-gh/v2/pkg/asciisanitizer/sanitizer.go | 12 + .../golang.org/x/text/transform/reader.go | 18 ++ .../vendor/modules.txt | 6 + .github/workflows/codeql.yml | 24 +- .gitignore | 5 + internal/skills/discovery/discovery.go | 18 +- internal/skills/discovery/discovery_test.go | 2 +- internal/skills/installer/installer.go | 6 +- pkg/cmd/agent-task/shared/log.go | 32 +-- pkg/cmd/api/api.go | 26 +- pkg/cmd/api/api_test.go | 82 +++++- pkg/cmd/codespace/logs.go | 10 + pkg/cmd/gist/edit/edit.go | 4 +- pkg/cmd/gist/shared/shared.go | 15 +- pkg/cmd/gist/shared/shared_test.go | 2 +- pkg/cmd/gist/view/view.go | 39 ++- pkg/cmd/gist/view/view_test.go | 100 ++++++++ pkg/cmd/pr/diff/diff.go | 90 +++---- pkg/cmd/pr/diff/diff_test.go | 74 +++++- pkg/cmd/release/download/download.go | 26 +- pkg/cmd/release/download/download_test.go | 123 +++++++++ pkg/cmd/repo/read-file/read_file.go | 39 +-- pkg/cmd/repo/read-file/read_file_test.go | 39 --- pkg/cmd/skills/preview/preview.go | 11 +- pkg/cmd/skills/search/search.go | 2 +- pkg/iostreams/content.go | 92 +++++++ pkg/iostreams/content_test.go | 152 +++++++++++ pkg/iostreams/iostreams.go | 52 +++- pkg/iostreams/untrusted.go | 95 +++++++ pkg/iostreams/untrusted_test.go | 79 ++++++ 57 files changed, 2177 insertions(+), 190 deletions(-) create mode 100644 .github/codeql/codeql-config.yml create mode 100644 .github/codeql/codeql-pack.lock.yml create mode 100644 .github/codeql/qlpack.yml create mode 100644 .github/codeql/queries/examples/UnsanitizedResponseToTerminalBad.go create mode 100644 .github/codeql/queries/examples/UnsanitizedResponseToTerminalGood.go create mode 100644 .github/codeql/queries/unsanitized-response-to-terminal.md create mode 100644 .github/codeql/queries/unsanitized-response-to-terminal.qhelp create mode 100644 .github/codeql/queries/unsanitized-response-to-terminal.ql create mode 100644 .github/codeql/tests/.gitignore create mode 100644 .github/codeql/tests/codeql-pack.lock.yml create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/go.mod create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/go.sum create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/hits_base64_after_json.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/hits_iocopy_crossfunc.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/hits_readall_intraproc.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/hits_scanner_crossfunc.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/hits_untrusted_raw.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/misses_contentout.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/misses_decoder_decode.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/misses_disk_roundtrip.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/misses_field_after_unmarshal.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/misses_file_sink.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/misses_sanitized.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/misses_untrusted_string.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/pkg/iostreams/iostreams.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/test.expected create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/test.qlref create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/vendor/github.com/cli/go-gh/v2/pkg/asciisanitizer/sanitizer.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/vendor/golang.org/x/text/transform/reader.go create mode 100644 .github/codeql/tests/unsanitized-response-to-terminal/vendor/modules.txt create mode 100644 pkg/iostreams/content.go create mode 100644 pkg/iostreams/content_test.go create mode 100644 pkg/iostreams/untrusted.go create mode 100644 pkg/iostreams/untrusted_test.go diff --git a/.github/codeql/codeql-config.yml b/.github/codeql/codeql-config.yml new file mode 100644 index 00000000000..3047ab29349 --- /dev/null +++ b/.github/codeql/codeql-config.yml @@ -0,0 +1,19 @@ +name: "cli/cli CodeQL config" + +# This config extends the default `security-and-quality` suite with the +# custom queries in `.github/codeql/queries/`. The custom queries enforce +# project-specific invariants that are not covered by the stock packs: +# +# - unsanitized-response-to-terminal.ql: HTTP response content that +# reaches a terminal writer (`os.Stdout` / `os.Stderr` / +# `iostreams.IOStreams.Out` / `ErrOut`) other than `ContentOut` +# without being sanitized. Writing to `ContentOut`, calling +# `iostreams.Untrusted.String`, wrapping with `asciisanitizer`, or +# decoding as structured JSON are accepted, so untrusted response +# content is sanitized before it can reach a terminal. +# +# This config is only meaningful for the Go matrix entry; the Actions +# matrix entry ignores it. +queries: + - uses: security-and-quality + - uses: ./.github/codeql/queries diff --git a/.github/codeql/codeql-pack.lock.yml b/.github/codeql/codeql-pack.lock.yml new file mode 100644 index 00000000000..357ee5dab5c --- /dev/null +++ b/.github/codeql/codeql-pack.lock.yml @@ -0,0 +1,24 @@ +--- +lockVersion: 1.0.0 +dependencies: + codeql/concepts: + version: 0.0.24 + codeql/controlflow: + version: 2.0.34 + codeql/dataflow: + version: 2.1.6 + codeql/go-all: + version: 7.1.1 + codeql/mad: + version: 1.0.50 + codeql/ssa: + version: 2.0.26 + codeql/threat-models: + version: 1.0.50 + codeql/tutorial: + version: 1.0.50 + codeql/typetracking: + version: 2.0.34 + codeql/util: + version: 2.0.37 +compiled: false diff --git a/.github/codeql/qlpack.yml b/.github/codeql/qlpack.yml new file mode 100644 index 00000000000..cdcdfcc7d69 --- /dev/null +++ b/.github/codeql/qlpack.yml @@ -0,0 +1,9 @@ +name: cli/cli-custom-security +version: 0.0.1 +library: false +extractor: go +tests: tests +dependencies: + codeql/go-all: ^7.1.1 +default-suite: + - queries: queries diff --git a/.github/codeql/queries/examples/UnsanitizedResponseToTerminalBad.go b/.github/codeql/queries/examples/UnsanitizedResponseToTerminalBad.go new file mode 100644 index 00000000000..eeb4698ba66 --- /dev/null +++ b/.github/codeql/queries/examples/UnsanitizedResponseToTerminalBad.go @@ -0,0 +1,34 @@ +package example + +import ( + "fmt" + "io" + "net/http" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +type Options struct { + IO *iostreams.IOStreams + HTTPClient *http.Client + URL string +} + +func run(opts *Options) error { + resp, err := opts.HTTPClient.Get(opts.URL) + if err != nil { + return err + } + defer resp.Body.Close() + + body, err := io.ReadAll(resp.Body) + if err != nil { + return err + } + + // BAD: server-controlled bytes are written to IO.Out, which does not + // sanitize. Any ANSI escape sequences in the response will be rendered + // by the user's terminal. + fmt.Fprint(opts.IO.Out, string(body)) + return nil +} diff --git a/.github/codeql/queries/examples/UnsanitizedResponseToTerminalGood.go b/.github/codeql/queries/examples/UnsanitizedResponseToTerminalGood.go new file mode 100644 index 00000000000..0f907196f29 --- /dev/null +++ b/.github/codeql/queries/examples/UnsanitizedResponseToTerminalGood.go @@ -0,0 +1,52 @@ +package example + +import ( + "fmt" + "io" + "net/http" + + "github.com/cli/cli/v2/pkg/iostreams" + "github.com/spf13/cobra" +) + +type Options struct { + IO *iostreams.IOStreams + HTTPClient *http.Client + URL string + + AllowEscapeSequences bool +} + +func newCmd() *cobra.Command { + opts := &Options{} + cmd := &cobra.Command{ + Use: "fetch", + RunE: func(*cobra.Command, []string) error { return run(opts) }, + } + cmd.Flags().BoolVar(&opts.AllowEscapeSequences, "allow-escape-sequences", false, + "Allow printing terminal escape sequences") + return cmd +} + +func run(opts *Options) error { + if opts.AllowEscapeSequences { + opts.IO.SetContentSanitization(false) + } + + resp, err := opts.HTTPClient.Get(opts.URL) + if err != nil { + return err + } + defer resp.Body.Close() + + body, err := io.ReadAll(resp.Body) + if err != nil { + return err + } + + // GOOD: external bytes flow through ContentOut, which sanitizes ANSI + // escape sequences by default. The --allow-escape-sequences flag is the + // documented opt-out for trusted content. + fmt.Fprint(opts.IO.ContentOut, string(body)) + return nil +} diff --git a/.github/codeql/queries/unsanitized-response-to-terminal.md b/.github/codeql/queries/unsanitized-response-to-terminal.md new file mode 100644 index 00000000000..90b5b8d8ae6 --- /dev/null +++ b/.github/codeql/queries/unsanitized-response-to-terminal.md @@ -0,0 +1,118 @@ + +# HTTP response content reaches a terminal without ContentOut or sanitization +Bytes consumed from an HTTP response body are server-controlled and may contain ANSI escape sequences. When those bytes reach a terminal writer without sanitization, a remote attacker can move the cursor, repaint the screen, fake a shell prompt, write to the clipboard via OSC sequences, or otherwise manipulate the user's terminal session. + +This query flags HTTP response content, including bytes reintroduced by base64 decoding, that reaches a terminal writer (`IOStreams.Out`, `IOStreams.ErrOut`, `os.Stdout`, or `os.Stderr`) without first being written to `IOStreams.ContentOut`, sanitized, or decoded as a structured format (e.g. `encoding/json`). + + +## Recommendation +Choose the writer based on the kind of content you are printing: + +* `IOStreams.Out` is for application output the developer authored: tables, prompts, formatted messages, color-coded status. It does not sanitize and never should, because the developer controls every byte that reaches it. +* `IOStreams.ContentOut` is for external content the developer did not author: HTTP response bodies, file contents fetched from a remote, anything where a third party chose the bytes. It sanitizes ANSI escape sequences by default. +These patterns satisfy the query: + +1. Label the content at its source as `iostreams.Untrusted` and print it with `String()` (or any `fmt` verb, which calls `String()`); the value sanitizes itself. Its `Raw()` method is the explicit opt-out and is still flagged if it reaches a terminal. +1. Write external bytes to `IOStreams.ContentOut`. +1. Decode the bytes into a structured value first (`json.Unmarshal`, `(*json.Decoder).Decode`); the fields you print afterwards are no longer raw external content. +1. For commands where the user has opted into raw output, add a per-command `--allow-escape-sequences` flag and call `opts.IO.SetContentSanitization(false)` before writing. The bytes still go through `ContentOut`, but ContentOut becomes a passthrough for that invocation. + +## Example +In the following BAD example, the response body is written directly to `IOStreams.Out`. A server can embed ANSI escape sequences in the response and they will be rendered by the user's terminal: + + +```go +package example + +import ( + "fmt" + "io" + "net/http" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +type Options struct { + IO *iostreams.IOStreams + HTTPClient *http.Client + URL string +} + +func run(opts *Options) error { + resp, err := opts.HTTPClient.Get(opts.URL) + if err != nil { + return err + } + defer resp.Body.Close() + + body, err := io.ReadAll(resp.Body) + if err != nil { + return err + } + + // BAD: server-controlled bytes are written to IO.Out, which does not + // sanitize. Any ANSI escape sequences in the response will be rendered + // by the user's terminal. + fmt.Fprint(opts.IO.Out, string(body)) + return nil +} + +``` +In the following GOOD example, the same body is written to `IOStreams.ContentOut`, which sanitizes ANSI escape sequences. An `--allow-escape-sequences` flag is provided for users who explicitly want raw output for trusted content: + + +```go +package example + +import ( + "fmt" + "io" + "net/http" + + "github.com/cli/cli/v2/pkg/iostreams" + "github.com/spf13/cobra" +) + +type Options struct { + IO *iostreams.IOStreams + HTTPClient *http.Client + URL string + + AllowEscapeSequences bool +} + +func newCmd() *cobra.Command { + opts := &Options{} + cmd := &cobra.Command{ + Use: "fetch", + RunE: func(*cobra.Command, []string) error { return run(opts) }, + } + cmd.Flags().BoolVar(&opts.AllowEscapeSequences, "allow-escape-sequences", false, + "Allow printing terminal escape sequences") + return cmd +} + +func run(opts *Options) error { + if opts.AllowEscapeSequences { + opts.IO.SetContentSanitization(false) + } + + resp, err := opts.HTTPClient.Get(opts.URL) + if err != nil { + return err + } + defer resp.Body.Close() + + body, err := io.ReadAll(resp.Body) + if err != nil { + return err + } + + // GOOD: external bytes flow through ContentOut, which sanitizes ANSI + // escape sequences by default. The --allow-escape-sequences flag is the + // documented opt-out for trusted content. + fmt.Fprint(opts.IO.ContentOut, string(body)) + return nil +} + +``` diff --git a/.github/codeql/queries/unsanitized-response-to-terminal.qhelp b/.github/codeql/queries/unsanitized-response-to-terminal.qhelp new file mode 100644 index 00000000000..af7bed1bf19 --- /dev/null +++ b/.github/codeql/queries/unsanitized-response-to-terminal.qhelp @@ -0,0 +1,84 @@ + + + +

+ Bytes consumed from an HTTP response body are server-controlled and may + contain ANSI escape sequences. When those bytes reach a terminal writer + without sanitization, a remote attacker can move the cursor, repaint the + screen, fake a shell prompt, write to the clipboard via OSC sequences, + or otherwise manipulate the user's terminal session. +

+

+ This query flags HTTP response content, including bytes reintroduced by + base64 decoding, that reaches a terminal writer + (IOStreams.Out, IOStreams.ErrOut, + os.Stdout, or os.Stderr) without first being + written to IOStreams.ContentOut, sanitized, or decoded as a + structured format (e.g. encoding/json). +

+ + + +

+ Choose the writer based on the kind of content you are printing: +

+ +

+ These patterns satisfy the query: +

+
    +
  1. + Label the content at its source as iostreams.Untrusted and + print it with String() (or any fmt verb, which + calls String()); the value sanitizes itself. Its + Raw() method is the explicit opt-out and is still flagged if + it reaches a terminal. +
    2. +
  2. + Write external bytes to IOStreams.ContentOut. +
    3. +
  3. + Decode the bytes into a structured value first + (json.Unmarshal, (*json.Decoder).Decode); + the fields you print afterwards are no longer raw external content. +
    4. +
  4. + For commands where the user has opted into raw output, add a + per-command --allow-escape-sequences flag and call + opts.IO.SetContentSanitization(false) before writing. + The bytes still go through ContentOut, but ContentOut + becomes a passthrough for that invocation. +
    5. +
+ + + +

+ In the following BAD example, the response body is written directly to + IOStreams.Out. A server can embed ANSI escape sequences in + the response and they will be rendered by the user's terminal: +

+ + +

+ In the following GOOD example, the same body is written to + IOStreams.ContentOut, which sanitizes ANSI escape + sequences. An --allow-escape-sequences flag is provided for users who + explicitly want raw output for trusted content: +

+ + + diff --git a/.github/codeql/queries/unsanitized-response-to-terminal.ql b/.github/codeql/queries/unsanitized-response-to-terminal.ql new file mode 100644 index 00000000000..6a1cf6c392f --- /dev/null +++ b/.github/codeql/queries/unsanitized-response-to-terminal.ql @@ -0,0 +1,239 @@ +/** + * @name HTTP response content reaches a terminal without ContentOut or sanitization + * @description Raw bytes consumed from an HTTP response body, or reintroduced by + * decoding base64, must either be written to `IOStreams.ContentOut` + * or wrapped with the asciisanitizer before reaching a terminal + * writer. The body is tracked across function boundaries, so a body + * returned from a fetch helper and consumed by its caller is still + * covered. Values produced by structured decoding (encoding/json) + * are trusted, since cli/cli's REST clients sanitize JSON bodies at + * the transport layer before decoding. + * @kind path-problem + * @problem.severity error + * @precision medium + * @id cli-cli/unsanitized-response-to-terminal + * @tags security + */ + +import go +import semmle.go.dataflow.TaintTracking + +// ContentOut is the blessed sanitizing writer. Writing raw content there is the +// safe choice, so it is excluded from the terminal sink set below. +predicate isContentOutRead(DataFlow::Node n) { + exists(Field f | + f.hasQualifiedName("github.com/cli/cli/v2/pkg/iostreams", "IOStreams", "ContentOut") and + n = f.getARead() + ) +} + +// Value flow from a ContentOut read so a ContentOut writer stored in a local +// variable is still recognised as the blessed sink. Plain DataFlow (not taint) +// is used so it does not leak across sibling fields of a shared IOStreams. +module ContentOutWriterConfig implements DataFlow::ConfigSig { + predicate isSource(DataFlow::Node n) { isContentOutRead(n) } + + predicate isSink(DataFlow::Node n) { exists(n) } +} + +module ContentOutWriterFlow = DataFlow::Global; + +predicate isContentOutWriter(DataFlow::Node n) { + isContentOutRead(n) or + exists(DataFlow::Node src | isContentOutRead(src) and ContentOutWriterFlow::flow(src, n)) +} + +// ANSI injection requires bytes to reach a terminal. The terminal-bound writers +// are os.Stdout / os.Stderr and the IOStreams.Out / IOStreams.ErrOut fields. +// File, socket, and buffer writers are not terminals and are intentionally out +// of scope. +predicate isTerminalWriterRead(DataFlow::Node n) { + ( + exists(Variable v | + v.hasQualifiedName("os", "Stdout") or v.hasQualifiedName("os", "Stderr") + | + n = v.getARead() + ) + or + exists(Field f | + f.hasQualifiedName("github.com/cli/cli/v2/pkg/iostreams", "IOStreams", "Out") or + f.hasQualifiedName("github.com/cli/cli/v2/pkg/iostreams", "IOStreams", "ErrOut") + | + n = f.getARead() + ) + ) and + not isContentOutRead(n) +} + +// Value flow from a terminal-writer read so aliased writers +// (`w := opts.IO.Out; fmt.Fprint(w, ...)`) still count as terminal sinks. Plain +// DataFlow (not taint) is used so it does not leak across sibling fields of a +// shared IOStreams (which would otherwise mark ContentOut as terminal-bound). +module TerminalWriterConfig implements DataFlow::ConfigSig { + predicate isSource(DataFlow::Node n) { isTerminalWriterRead(n) } + + predicate isSink(DataFlow::Node n) { exists(n) } +} + +module TerminalWriterFlow = DataFlow::Global; + +predicate isTerminalBoundWriter(DataFlow::Node n) { + isTerminalWriterRead(n) or + exists(DataFlow::Node src | isTerminalWriterRead(src) and TerminalWriterFlow::flow(src, n)) +} + +// Raw HTTP body reader. Sourcing at the field read (rather than a local +// consumption call) lets global taint carry the reader across returns and +// parameters before anything reads it. +predicate isResponseBodyReader(DataFlow::Node n) { + exists(Field bodyField | + bodyField.hasQualifiedName("net/http", "Response", "Body") and + n = bodyField.getARead() + ) +} + +// Base64 decoding reintroduces raw bytes that any text sanitization applied to +// the encoded form never saw, so the decoded stream is its own source. +predicate isBase64DecodeSource(DataFlow::Node n) { + exists(DataFlow::CallNode c | + c.getTarget().hasQualifiedName("encoding/base64", "NewDecoder") and n = c + ) + or + exists(DataFlow::MethodCallNode c | + c.getTarget().hasQualifiedName("encoding/base64", "Encoding", "DecodeString") and n = c + ) +} + +// Carry taint from a reader value into the bytes a read produces, so the flow +// continues from the reader to wherever those bytes are written. +predicate isReaderConsumptionStep(DataFlow::Node pred, DataFlow::Node succ) { + exists(DataFlow::CallNode call | + ( + call.getTarget().hasQualifiedName("io", "ReadAll") or + call.getTarget().hasQualifiedName("io/ioutil", "ReadAll") + ) and + pred = call.getArgument(0) and + // ReadAll returns (bytes, error). Track taint into result 0, the bytes, only. + // The error result is an I/O or decode failure string, never response data, + // so tracking it would flag code that merely prints that error. + succ = call.getResult(0) + ) + or + exists(DataFlow::CallNode call | + call.getTarget().hasQualifiedName("bufio", "NewScanner") and + pred = call.getArgument(0) and + succ = call + ) + or + exists(DataFlow::MethodCallNode read | + ( + read.getTarget().hasQualifiedName("bufio", "Scanner", "Text") or + read.getTarget().hasQualifiedName("bufio", "Scanner", "Bytes") + ) and + pred = read.getReceiver() and + succ = read + ) +} + +// The asciisanitizer wrap is the blessed barrier. Both Sanitizer{} and +// &Sanitizer{} forms are accepted, regardless of which fields are set. +predicate isSanitizerBarrier(DataFlow::Node n) { + exists(DataFlow::CallNode c, Type argTy | + c.getTarget().hasQualifiedName("golang.org/x/text/transform", "NewReader") and + argTy = c.getArgument(1).getType() and + ( + argTy.hasQualifiedName("github.com/cli/go-gh/v2/pkg/asciisanitizer", "Sanitizer") or + argTy + .(PointerType) + .getBaseType() + .hasQualifiedName("github.com/cli/go-gh/v2/pkg/asciisanitizer", "Sanitizer") + ) and + n = c + ) +} + +// Structured decoding is trusted. Both `json.Unmarshal` and +// `(*json.Decoder).Decode` go through cli/cli's REST clients, which are built on +// go-gh's sanitizing transport: for JSON content types the body is sanitized +// before any decode. A decoded value is therefore not raw external content. +predicate isStructuredDecodeBarrier(DataFlow::Node n) { + exists(DataFlow::CallNode c | + c.getTarget().hasQualifiedName("encoding/json", "Unmarshal") and + n = c.getArgument(0) + ) + or + exists(DataFlow::MethodCallNode c | + c.getTarget().hasQualifiedName("encoding/json", "Decoder", "Decode") and + n = c.getReceiver() + ) +} + +// iostreams.Untrusted.String() returns content with ANSI escapes neutralized, so +// its result is sanitized. Raw and RawBytes are the deliberate opt-out and are +// intentionally NOT barriers, so a value taken out through them stays tracked to +// the terminal (unless the destination is ContentOut, which sanitizes itself). +predicate isUntrustedStringBarrier(DataFlow::Node n) { + exists(DataFlow::MethodCallNode c | + c.getTarget().hasQualifiedName("github.com/cli/cli/v2/pkg/iostreams", "Untrusted", "String") and + n = c + ) +} + +module UnsanitizedResponseConfig implements DataFlow::ConfigSig { + predicate isSource(DataFlow::Node n) { + isResponseBodyReader(n) or isBase64DecodeSource(n) + } + + predicate isSink(DataFlow::Node n) { + exists(DataFlow::CallNode call, int i | + ( + call.getTarget().hasQualifiedName("fmt", "Fprint") or + call.getTarget().hasQualifiedName("fmt", "Fprintln") or + call.getTarget().hasQualifiedName("fmt", "Fprintf") + ) and + isTerminalBoundWriter(call.getArgument(0)) and + not isContentOutWriter(call.getArgument(0)) and + i >= 1 and + n = call.getArgument(i) + ) + or + exists(DataFlow::CallNode call | + ( + call.getTarget().hasQualifiedName("io", "Copy") or + call.getTarget().hasQualifiedName("io", "CopyBuffer") + ) and + isTerminalBoundWriter(call.getArgument(0)) and + not isContentOutWriter(call.getArgument(0)) and + n = call.getArgument(1) + ) + or + exists(DataFlow::MethodCallNode call | + call.getTarget().getName() = "Write" and + isTerminalBoundWriter(call.getReceiver()) and + not isContentOutWriter(call.getReceiver()) and + n = call.getArgument(0) + ) + } + + predicate isBarrier(DataFlow::Node n) { + isSanitizerBarrier(n) or isStructuredDecodeBarrier(n) or isUntrustedStringBarrier(n) + } + + predicate isAdditionalFlowStep(DataFlow::Node pred, DataFlow::Node succ) { + isReaderConsumptionStep(pred, succ) + } +} + +module UnsanitizedResponseFlow = TaintTracking::Global; + +import UnsanitizedResponseFlow::PathGraph + +from UnsanitizedResponseFlow::PathNode source, UnsanitizedResponseFlow::PathNode sink +where + UnsanitizedResponseFlow::flowPath(source, sink) and + not sink.getNode().getFile().getRelativePath().regexpMatch(".*_test\\.go") and + not sink.getNode().getFile().getRelativePath().regexpMatch("internal/fake_vuln/.*") and + not sink.getNode().getFile().getRelativePath().regexpMatch("\\.github/codeql/tests/.*") +select sink.getNode(), source, sink, + "HTTP response content reaches a terminal writer that is not IOStreams.ContentOut. " + + "Write external content to opts.IO.ContentOut, or wrap it with the asciisanitizer." diff --git a/.github/codeql/tests/.gitignore b/.github/codeql/tests/.gitignore new file mode 100644 index 00000000000..e24a007a083 --- /dev/null +++ b/.github/codeql/tests/.gitignore @@ -0,0 +1,3 @@ +# CodeQL test runner outputs (regenerated each run) +*.testproj/ +*.actual diff --git a/.github/codeql/tests/codeql-pack.lock.yml b/.github/codeql/tests/codeql-pack.lock.yml new file mode 100644 index 00000000000..357ee5dab5c --- /dev/null +++ b/.github/codeql/tests/codeql-pack.lock.yml @@ -0,0 +1,24 @@ +--- +lockVersion: 1.0.0 +dependencies: + codeql/concepts: + version: 0.0.24 + codeql/controlflow: + version: 2.0.34 + codeql/dataflow: + version: 2.1.6 + codeql/go-all: + version: 7.1.1 + codeql/mad: + version: 1.0.50 + codeql/ssa: + version: 2.0.26 + codeql/threat-models: + version: 1.0.50 + codeql/tutorial: + version: 1.0.50 + codeql/typetracking: + version: 2.0.34 + codeql/util: + version: 2.0.37 +compiled: false diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/go.mod b/.github/codeql/tests/unsanitized-response-to-terminal/go.mod new file mode 100644 index 00000000000..6ed3c2fec7c --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/go.mod @@ -0,0 +1,8 @@ +module github.com/cli/cli/v2 + +go 1.25.0 + +require ( + github.com/cli/go-gh/v2 v2.13.0 + golang.org/x/text v0.37.0 +) diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/go.sum b/.github/codeql/tests/unsanitized-response-to-terminal/go.sum new file mode 100644 index 00000000000..249ff20e6a1 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/go.sum @@ -0,0 +1,4 @@ +github.com/cli/go-gh/v2 v2.13.0 h1:jEHZu/VPVoIJkciK3pzZd3rbT8J90swsK5Ui4ewH1ys= +github.com/cli/go-gh/v2 v2.13.0/go.mod h1:Us/NbQ8VNM0fdaILgoXSz6PKkV5PWaEzkJdc9vR2geM= +golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc= +golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38= diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/hits_base64_after_json.go b/.github/codeql/tests/unsanitized-response-to-terminal/hits_base64_after_json.go new file mode 100644 index 00000000000..112683a9cef --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/hits_base64_after_json.go @@ -0,0 +1,34 @@ +package fixtures + +import ( + "encoding/base64" + "fmt" + "io" + "strings" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +// A base64 field decoded back into raw bytes, then printed to Out. The decode +// reintroduces content that escaped any sanitization of the encoded text. Must +// be flagged. +type blobResponse struct { + Content string +} + +func fetchBlob(resp blobResponse) (string, error) { + decoded, err := io.ReadAll(base64.NewDecoder(base64.StdEncoding, strings.NewReader(resp.Content))) + if err != nil { + return "", err + } + return string(decoded), nil +} + +func PreviewBlob(resp blobResponse, ios *iostreams.IOStreams) error { + content, err := fetchBlob(resp) + if err != nil { + return err + } + fmt.Fprint(ios.Out, content) + return nil +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/hits_iocopy_crossfunc.go b/.github/codeql/tests/unsanitized-response-to-terminal/hits_iocopy_crossfunc.go new file mode 100644 index 00000000000..1b652eece62 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/hits_iocopy_crossfunc.go @@ -0,0 +1,28 @@ +package fixtures + +import ( + "io" + "net/http" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +// A helper returns the raw body; the caller streams it to Out from a different +// function. Must be flagged. +func fetchBodyForCopy(url string) (io.ReadCloser, error) { + resp, err := http.Get(url) + if err != nil { + return nil, err + } + return resp.Body, nil +} + +func CopyBodyToOut(url string, ios *iostreams.IOStreams) error { + r, err := fetchBodyForCopy(url) + if err != nil { + return err + } + defer r.Close() + _, err = io.Copy(ios.Out, r) + return err +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/hits_readall_intraproc.go b/.github/codeql/tests/unsanitized-response-to-terminal/hits_readall_intraproc.go new file mode 100644 index 00000000000..5dd63daff39 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/hits_readall_intraproc.go @@ -0,0 +1,19 @@ +package fixtures + +import ( + "fmt" + "io" + "net/http" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +// io.ReadAll of the body, printed to Out in the same function. Must be flagged. +func ReadAllToOut(resp *http.Response, ios *iostreams.IOStreams) error { + body, err := io.ReadAll(resp.Body) + if err != nil { + return err + } + fmt.Fprintln(ios.Out, string(body)) + return nil +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/hits_scanner_crossfunc.go b/.github/codeql/tests/unsanitized-response-to-terminal/hits_scanner_crossfunc.go new file mode 100644 index 00000000000..e7867873467 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/hits_scanner_crossfunc.go @@ -0,0 +1,33 @@ +package fixtures + +import ( + "bufio" + "fmt" + "io" + "net/http" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +// A helper returns the raw body; the caller scans it line by line and prints to +// Out. Must be flagged. +func fetchLog(url string) (io.ReadCloser, error) { + resp, err := http.Get(url) + if err != nil { + return nil, err + } + return resp.Body, nil +} + +func ScanLogToOut(url string, ios *iostreams.IOStreams) error { + rc, err := fetchLog(url) + if err != nil { + return err + } + defer rc.Close() + scanner := bufio.NewScanner(rc) + for scanner.Scan() { + fmt.Fprintf(ios.Out, "%s\n", scanner.Text()) + } + return nil +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/hits_untrusted_raw.go b/.github/codeql/tests/unsanitized-response-to-terminal/hits_untrusted_raw.go new file mode 100644 index 00000000000..e51ff8193fc --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/hits_untrusted_raw.go @@ -0,0 +1,22 @@ +package fixtures + +import ( + "fmt" + "io" + "net/http" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +// A body minted as Untrusted but taken out through Raw and printed. Raw is the +// deliberate opt-out and is not a barrier, so the content reaches the terminal +// raw and must be flagged. +func RawEscapeHatchToOut(resp *http.Response, ios *iostreams.IOStreams) error { + body, err := io.ReadAll(resp.Body) + if err != nil { + return err + } + u := iostreams.NewUntrustedBytes(body) + fmt.Fprintln(ios.Out, u.Raw()) + return nil +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/misses_contentout.go b/.github/codeql/tests/unsanitized-response-to-terminal/misses_contentout.go new file mode 100644 index 00000000000..72edc981ca4 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/misses_contentout.go @@ -0,0 +1,34 @@ +package fixtures + +import ( + "fmt" + "io" + "net/http" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +// Raw body written to the blessed ContentOut writer. Must NOT be flagged. +func ReadAllToContentOut(url string, ios *iostreams.IOStreams) error { + resp, err := http.Get(url) + if err != nil { + return err + } + defer resp.Body.Close() + body, err := io.ReadAll(resp.Body) + if err != nil { + return err + } + fmt.Fprintln(ios.ContentOut, string(body)) + return nil +} + +func CopyToContentOut(url string, ios *iostreams.IOStreams) error { + resp, err := http.Get(url) + if err != nil { + return err + } + defer resp.Body.Close() + _, err = io.Copy(ios.ContentOut, resp.Body) + return err +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/misses_decoder_decode.go b/.github/codeql/tests/unsanitized-response-to-terminal/misses_decoder_decode.go new file mode 100644 index 00000000000..ae5fb2656ae --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/misses_decoder_decode.go @@ -0,0 +1,29 @@ +package fixtures + +import ( + "encoding/json" + "fmt" + "net/http" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +// json.NewDecoder on the body. The transport JSON sanitizer feeds this path, so +// the Decode barrier keeps the query silent. Must NOT be flagged. +type issueView struct { + Title string +} + +func ViewIssueTitle(url string, ios *iostreams.IOStreams) error { + resp, err := http.Get(url) + if err != nil { + return err + } + defer resp.Body.Close() + var iss issueView + if err := json.NewDecoder(resp.Body).Decode(&iss); err != nil { + return err + } + fmt.Fprintln(ios.Out, iss.Title) + return nil +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/misses_disk_roundtrip.go b/.github/codeql/tests/unsanitized-response-to-terminal/misses_disk_roundtrip.go new file mode 100644 index 00000000000..af476bd8c51 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/misses_disk_roundtrip.go @@ -0,0 +1,43 @@ +package fixtures + +import ( + "bufio" + "fmt" + "io" + "net/http" + "os" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +// The body is written to a disk cache, then reopened and printed. Static taint +// cannot bridge the filesystem, so the query is silent here by necessity. The +// runtime ContentOut writer is what covers this case. Documented as a known +// limitation; must NOT be flagged. +func cacheBody(url, path string) error { + resp, err := http.Get(url) + if err != nil { + return err + } + defer resp.Body.Close() + f, err := os.Create(path) + if err != nil { + return err + } + defer f.Close() + _, err = io.Copy(f, resp.Body) + return err +} + +func PrintCachedFile(path string, ios *iostreams.IOStreams) error { + f, err := os.Open(path) + if err != nil { + return err + } + defer f.Close() + scanner := bufio.NewScanner(f) + for scanner.Scan() { + fmt.Fprintf(ios.Out, "%s\n", scanner.Text()) + } + return nil +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/misses_field_after_unmarshal.go b/.github/codeql/tests/unsanitized-response-to-terminal/misses_field_after_unmarshal.go new file mode 100644 index 00000000000..69207533d50 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/misses_field_after_unmarshal.go @@ -0,0 +1,41 @@ +package fixtures + +import ( + "encoding/json" + "fmt" + "io" + "net/http" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +// A helper reads the raw body into bytes; the caller json.Unmarshals it and +// prints a decoded field. Statically this is identical to the safe case where an +// already-sanitized JSON response is decoded and a field printed, so the query +// stays silent on purpose. The runtime ContentOut writer is the mitigation. Must +// NOT be flagged. +type logEntry struct { + Content string +} + +func fetchLogBytes(url string) ([]byte, error) { + resp, err := http.Get(url) + if err != nil { + return nil, err + } + defer resp.Body.Close() + return io.ReadAll(resp.Body) +} + +func RenderLogField(url string, ios *iostreams.IOStreams) error { + raw, err := fetchLogBytes(url) + if err != nil { + return err + } + var entry logEntry + if err := json.Unmarshal(raw, &entry); err != nil { + return err + } + fmt.Fprintln(ios.Out, entry.Content) + return nil +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/misses_file_sink.go b/.github/codeql/tests/unsanitized-response-to-terminal/misses_file_sink.go new file mode 100644 index 00000000000..e9c9b8eee37 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/misses_file_sink.go @@ -0,0 +1,23 @@ +package fixtures + +import ( + "io" + "net/http" + "os" +) + +// The body is copied to a file on disk, not a terminal. Must NOT be flagged. +func DownloadToFile(url, path string) error { + resp, err := http.Get(url) + if err != nil { + return err + } + defer resp.Body.Close() + f, err := os.Create(path) + if err != nil { + return err + } + defer f.Close() + _, err = io.Copy(f, resp.Body) + return err +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/misses_sanitized.go b/.github/codeql/tests/unsanitized-response-to-terminal/misses_sanitized.go new file mode 100644 index 00000000000..83f505cf1fb --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/misses_sanitized.go @@ -0,0 +1,40 @@ +package fixtures + +import ( + "bufio" + "fmt" + "io" + "net/http" + + "github.com/cli/go-gh/v2/pkg/asciisanitizer" + "golang.org/x/text/transform" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +// The body is wrapped with the asciisanitizer transform before printing. Must +// NOT be flagged. +func SanitizedScanToOut(url string, ios *iostreams.IOStreams) error { + resp, err := http.Get(url) + if err != nil { + return err + } + defer resp.Body.Close() + sanitized := transform.NewReader(resp.Body, &asciisanitizer.Sanitizer{}) + scanner := bufio.NewScanner(sanitized) + for scanner.Scan() { + fmt.Fprintf(ios.Out, "%s\n", scanner.Text()) + } + return nil +} + +func SanitizedCopyToOut(url string, ios *iostreams.IOStreams) error { + resp, err := http.Get(url) + if err != nil { + return err + } + defer resp.Body.Close() + sanitized := transform.NewReader(resp.Body, &asciisanitizer.Sanitizer{}) + _, err = io.Copy(ios.Out, sanitized) + return err +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/misses_untrusted_string.go b/.github/codeql/tests/unsanitized-response-to-terminal/misses_untrusted_string.go new file mode 100644 index 00000000000..cb2b5b2a434 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/misses_untrusted_string.go @@ -0,0 +1,21 @@ +package fixtures + +import ( + "fmt" + "io" + "net/http" + + "github.com/cli/cli/v2/pkg/iostreams" +) + +// A body minted as Untrusted and printed through String(), which sanitizes. Must +// NOT be flagged. +func UntrustedStringToOut(resp *http.Response, ios *iostreams.IOStreams) error { + body, err := io.ReadAll(resp.Body) + if err != nil { + return err + } + u := iostreams.NewUntrustedBytes(body) + fmt.Fprintln(ios.Out, u.String()) + return nil +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/pkg/iostreams/iostreams.go b/.github/codeql/tests/unsanitized-response-to-terminal/pkg/iostreams/iostreams.go new file mode 100644 index 00000000000..7a46dc7fe8b --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/pkg/iostreams/iostreams.go @@ -0,0 +1,29 @@ +package iostreams + +import "io" + +// IOStreams is a minimal stub mirroring the real package's writer fields so the +// query can match Out / ErrOut / ContentOut by qualified name in tests. +type IOStreams struct { + Out io.Writer + ErrOut io.Writer + ContentOut io.Writer +} + +// Untrusted is a minimal stub of the real provenance type so fixtures can mint +// and unwrap external content and the query can match String / Raw by qualified +// name. +type Untrusted struct { + raw string +} + +func NewUntrusted(s string) Untrusted { return Untrusted{raw: s} } + +func NewUntrustedBytes(b []byte) Untrusted { return Untrusted{raw: string(b)} } + +func (u Untrusted) String() string { return sanitizeStub(u.raw) } + +func (u Untrusted) Raw() string { return u.raw } + +func sanitizeStub(s string) string { return s } + diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/test.expected b/.github/codeql/tests/unsanitized-response-to-terminal/test.expected new file mode 100644 index 00000000000..ad9a2c7bc32 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/test.expected @@ -0,0 +1,77 @@ +edges +| hits_base64_after_json.go:20:2:20:99 | ... := ...[0] | hits_base64_after_json.go:24:9:24:23 | type conversion | provenance | | +| hits_base64_after_json.go:20:29:20:98 | call to NewDecoder | hits_base64_after_json.go:20:2:20:99 | ... := ...[0] | provenance | Config | +| hits_base64_after_json.go:20:29:20:98 | call to NewDecoder | hits_base64_after_json.go:20:2:20:99 | ... := ...[0] | provenance | MaD:1773 | +| hits_base64_after_json.go:24:9:24:23 | type conversion | hits_base64_after_json.go:28:2:28:32 | ... := ...[0] | provenance | | +| hits_base64_after_json.go:28:2:28:32 | ... := ...[0] | hits_base64_after_json.go:32:22:32:28 | content | provenance | | +| hits_base64_after_json.go:32:22:32:28 | content | hits_base64_after_json.go:32:2:32:29 | []type{args} | provenance | | +| hits_iocopy_crossfunc.go:17:9:17:17 | selection of Body | hits_iocopy_crossfunc.go:21:2:21:32 | ... := ...[0] | provenance | | +| hits_iocopy_crossfunc.go:21:2:21:32 | ... := ...[0] | hits_iocopy_crossfunc.go:26:28:26:28 | r | provenance | | +| hits_readall_intraproc.go:13:2:13:35 | ... := ...[0] | hits_readall_intraproc.go:17:24:17:35 | type conversion | provenance | | +| hits_readall_intraproc.go:13:26:13:34 | selection of Body | hits_readall_intraproc.go:13:2:13:35 | ... := ...[0] | provenance | Config | +| hits_readall_intraproc.go:13:26:13:34 | selection of Body | hits_readall_intraproc.go:13:2:13:35 | ... := ...[0] | provenance | MaD:1773 | +| hits_readall_intraproc.go:17:24:17:35 | type conversion | hits_readall_intraproc.go:17:2:17:36 | []type{args} | provenance | | +| hits_scanner_crossfunc.go:19:9:19:17 | selection of Body | hits_scanner_crossfunc.go:23:2:23:25 | ... := ...[0] | provenance | | +| hits_scanner_crossfunc.go:23:2:23:25 | ... := ...[0] | hits_scanner_crossfunc.go:28:30:28:31 | rc | provenance | | +| hits_scanner_crossfunc.go:28:13:28:32 | call to NewScanner | hits_scanner_crossfunc.go:30:32:30:38 | scanner | provenance | | +| hits_scanner_crossfunc.go:28:30:28:31 | rc | hits_scanner_crossfunc.go:28:13:28:32 | call to NewScanner | provenance | Config | +| hits_scanner_crossfunc.go:28:30:28:31 | rc | hits_scanner_crossfunc.go:28:13:28:32 | call to NewScanner | provenance | MaD:14 | +| hits_scanner_crossfunc.go:30:32:30:38 | scanner | hits_scanner_crossfunc.go:30:32:30:45 | call to Text | provenance | Config | +| hits_scanner_crossfunc.go:30:32:30:38 | scanner | hits_scanner_crossfunc.go:30:32:30:45 | call to Text | provenance | MaD:26 | +| hits_scanner_crossfunc.go:30:32:30:45 | call to Text | hits_scanner_crossfunc.go:30:3:30:46 | []type{args} | provenance | | +| hits_untrusted_raw.go:15:2:15:35 | ... := ...[0] | hits_untrusted_raw.go:19:35:19:38 | body | provenance | | +| hits_untrusted_raw.go:15:26:15:34 | selection of Body | hits_untrusted_raw.go:15:2:15:35 | ... := ...[0] | provenance | Config | +| hits_untrusted_raw.go:15:26:15:34 | selection of Body | hits_untrusted_raw.go:15:2:15:35 | ... := ...[0] | provenance | MaD:1773 | +| hits_untrusted_raw.go:19:7:19:39 | call to NewUntrustedBytes [raw] | hits_untrusted_raw.go:20:24:20:24 | u [raw] | provenance | | +| hits_untrusted_raw.go:19:35:19:38 | body | hits_untrusted_raw.go:19:7:19:39 | call to NewUntrustedBytes [raw] | provenance | | +| hits_untrusted_raw.go:19:35:19:38 | body | pkg/iostreams/iostreams.go:22:24:22:24 | definition of b | provenance | | +| hits_untrusted_raw.go:20:24:20:24 | u [raw] | hits_untrusted_raw.go:20:24:20:30 | call to Raw | provenance | | +| hits_untrusted_raw.go:20:24:20:24 | u [raw] | pkg/iostreams/iostreams.go:26:7:26:7 | definition of u [raw] | provenance | | +| hits_untrusted_raw.go:20:24:20:30 | call to Raw | hits_untrusted_raw.go:20:2:20:31 | []type{args} | provenance | | +| pkg/iostreams/iostreams.go:22:24:22:24 | definition of b | pkg/iostreams/iostreams.go:22:68:22:76 | type conversion | provenance | | +| pkg/iostreams/iostreams.go:22:68:22:76 | type conversion | pkg/iostreams/iostreams.go:22:53:22:77 | struct literal [raw] | provenance | | +| pkg/iostreams/iostreams.go:26:7:26:7 | definition of u [raw] | pkg/iostreams/iostreams.go:26:42:26:42 | u [raw] | provenance | | +| pkg/iostreams/iostreams.go:26:42:26:42 | u [raw] | pkg/iostreams/iostreams.go:26:42:26:46 | selection of raw | provenance | | +nodes +| hits_base64_after_json.go:20:2:20:99 | ... := ...[0] | semmle.label | ... := ...[0] | +| hits_base64_after_json.go:20:29:20:98 | call to NewDecoder | semmle.label | call to NewDecoder | +| hits_base64_after_json.go:24:9:24:23 | type conversion | semmle.label | type conversion | +| hits_base64_after_json.go:28:2:28:32 | ... := ...[0] | semmle.label | ... := ...[0] | +| hits_base64_after_json.go:32:2:32:29 | []type{args} | semmle.label | []type{args} | +| hits_base64_after_json.go:32:22:32:28 | content | semmle.label | content | +| hits_iocopy_crossfunc.go:17:9:17:17 | selection of Body | semmle.label | selection of Body | +| hits_iocopy_crossfunc.go:21:2:21:32 | ... := ...[0] | semmle.label | ... := ...[0] | +| hits_iocopy_crossfunc.go:26:28:26:28 | r | semmle.label | r | +| hits_readall_intraproc.go:13:2:13:35 | ... := ...[0] | semmle.label | ... := ...[0] | +| hits_readall_intraproc.go:13:26:13:34 | selection of Body | semmle.label | selection of Body | +| hits_readall_intraproc.go:17:2:17:36 | []type{args} | semmle.label | []type{args} | +| hits_readall_intraproc.go:17:24:17:35 | type conversion | semmle.label | type conversion | +| hits_scanner_crossfunc.go:19:9:19:17 | selection of Body | semmle.label | selection of Body | +| hits_scanner_crossfunc.go:23:2:23:25 | ... := ...[0] | semmle.label | ... := ...[0] | +| hits_scanner_crossfunc.go:28:13:28:32 | call to NewScanner | semmle.label | call to NewScanner | +| hits_scanner_crossfunc.go:28:30:28:31 | rc | semmle.label | rc | +| hits_scanner_crossfunc.go:30:3:30:46 | []type{args} | semmle.label | []type{args} | +| hits_scanner_crossfunc.go:30:32:30:38 | scanner | semmle.label | scanner | +| hits_scanner_crossfunc.go:30:32:30:45 | call to Text | semmle.label | call to Text | +| hits_untrusted_raw.go:15:2:15:35 | ... := ...[0] | semmle.label | ... := ...[0] | +| hits_untrusted_raw.go:15:26:15:34 | selection of Body | semmle.label | selection of Body | +| hits_untrusted_raw.go:19:7:19:39 | call to NewUntrustedBytes [raw] | semmle.label | call to NewUntrustedBytes [raw] | +| hits_untrusted_raw.go:19:35:19:38 | body | semmle.label | body | +| hits_untrusted_raw.go:20:2:20:31 | []type{args} | semmle.label | []type{args} | +| hits_untrusted_raw.go:20:24:20:24 | u [raw] | semmle.label | u [raw] | +| hits_untrusted_raw.go:20:24:20:30 | call to Raw | semmle.label | call to Raw | +| pkg/iostreams/iostreams.go:22:24:22:24 | definition of b | semmle.label | definition of b | +| pkg/iostreams/iostreams.go:22:53:22:77 | struct literal [raw] | semmle.label | struct literal [raw] | +| pkg/iostreams/iostreams.go:22:68:22:76 | type conversion | semmle.label | type conversion | +| pkg/iostreams/iostreams.go:26:7:26:7 | definition of u [raw] | semmle.label | definition of u [raw] | +| pkg/iostreams/iostreams.go:26:42:26:42 | u [raw] | semmle.label | u [raw] | +| pkg/iostreams/iostreams.go:26:42:26:46 | selection of raw | semmle.label | selection of raw | +subpaths +| hits_untrusted_raw.go:19:35:19:38 | body | pkg/iostreams/iostreams.go:22:24:22:24 | definition of b | pkg/iostreams/iostreams.go:22:53:22:77 | struct literal [raw] | hits_untrusted_raw.go:19:7:19:39 | call to NewUntrustedBytes [raw] | +| hits_untrusted_raw.go:20:24:20:24 | u [raw] | pkg/iostreams/iostreams.go:26:7:26:7 | definition of u [raw] | pkg/iostreams/iostreams.go:26:42:26:46 | selection of raw | hits_untrusted_raw.go:20:24:20:30 | call to Raw | +#select +| hits_base64_after_json.go:32:2:32:29 | []type{args} | hits_base64_after_json.go:20:29:20:98 | call to NewDecoder | hits_base64_after_json.go:32:2:32:29 | []type{args} | HTTP response content reaches a terminal writer that is not IOStreams.ContentOut. Write external content to opts.IO.ContentOut, or wrap it with the asciisanitizer. | +| hits_iocopy_crossfunc.go:26:28:26:28 | r | hits_iocopy_crossfunc.go:17:9:17:17 | selection of Body | hits_iocopy_crossfunc.go:26:28:26:28 | r | HTTP response content reaches a terminal writer that is not IOStreams.ContentOut. Write external content to opts.IO.ContentOut, or wrap it with the asciisanitizer. | +| hits_readall_intraproc.go:17:2:17:36 | []type{args} | hits_readall_intraproc.go:13:26:13:34 | selection of Body | hits_readall_intraproc.go:17:2:17:36 | []type{args} | HTTP response content reaches a terminal writer that is not IOStreams.ContentOut. Write external content to opts.IO.ContentOut, or wrap it with the asciisanitizer. | +| hits_scanner_crossfunc.go:30:3:30:46 | []type{args} | hits_scanner_crossfunc.go:19:9:19:17 | selection of Body | hits_scanner_crossfunc.go:30:3:30:46 | []type{args} | HTTP response content reaches a terminal writer that is not IOStreams.ContentOut. Write external content to opts.IO.ContentOut, or wrap it with the asciisanitizer. | +| hits_untrusted_raw.go:20:2:20:31 | []type{args} | hits_untrusted_raw.go:15:26:15:34 | selection of Body | hits_untrusted_raw.go:20:2:20:31 | []type{args} | HTTP response content reaches a terminal writer that is not IOStreams.ContentOut. Write external content to opts.IO.ContentOut, or wrap it with the asciisanitizer. | diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/test.qlref b/.github/codeql/tests/unsanitized-response-to-terminal/test.qlref new file mode 100644 index 00000000000..843fbea4062 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/test.qlref @@ -0,0 +1 @@ +queries/unsanitized-response-to-terminal.ql diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/vendor/github.com/cli/go-gh/v2/pkg/asciisanitizer/sanitizer.go b/.github/codeql/tests/unsanitized-response-to-terminal/vendor/github.com/cli/go-gh/v2/pkg/asciisanitizer/sanitizer.go new file mode 100644 index 00000000000..9a02658b178 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/vendor/github.com/cli/go-gh/v2/pkg/asciisanitizer/sanitizer.go @@ -0,0 +1,12 @@ +// Minimal stub of github.com/cli/go-gh/v2/pkg/asciisanitizer for CodeQL test +// extraction. Only needs the Sanitizer type to exist under the expected +// qualified name so the barrier predicate's hasQualifiedName check matches. +package asciisanitizer + +type Sanitizer struct{} + +func (s *Sanitizer) Reset() {} + +func (s *Sanitizer) Transform(dst, src []byte, atEOF bool) (int, int, error) { + return 0, 0, nil +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/vendor/golang.org/x/text/transform/reader.go b/.github/codeql/tests/unsanitized-response-to-terminal/vendor/golang.org/x/text/transform/reader.go new file mode 100644 index 00000000000..7a54f60b4d1 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/vendor/golang.org/x/text/transform/reader.go @@ -0,0 +1,18 @@ +// Minimal stub of golang.org/x/text/transform for CodeQL test extraction. +// Only needs NewReader to exist under the expected qualified name. +package transform + +import "io" + +type Transformer interface { + Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) + Reset() +} + +type Reader struct{ r io.Reader } + +func (r *Reader) Read(p []byte) (int, error) { return r.r.Read(p) } + +func NewReader(r io.Reader, t Transformer) *Reader { + return &Reader{r: r} +} diff --git a/.github/codeql/tests/unsanitized-response-to-terminal/vendor/modules.txt b/.github/codeql/tests/unsanitized-response-to-terminal/vendor/modules.txt new file mode 100644 index 00000000000..44bdf242178 --- /dev/null +++ b/.github/codeql/tests/unsanitized-response-to-terminal/vendor/modules.txt @@ -0,0 +1,6 @@ +# github.com/cli/go-gh/v2 v2.13.0 +## explicit; go 1.21 +github.com/cli/go-gh/v2/pkg/asciisanitizer +# golang.org/x/text v0.37.0 +## explicit; go 1.21 +golang.org/x/text/transform diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index ab456c1c577..dfee2a817ce 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -9,6 +9,7 @@ on: - '**/*.md' schedule: - cron: "0 0 * * 0" + workflow_dispatch: permissions: actions: read # for github/codeql-action/init to get workflow details @@ -21,7 +22,18 @@ jobs: strategy: fail-fast: false matrix: - language: ['go', 'actions'] + include: + # Go uses our custom config, which extends `security-and-quality` + # with the project-specific queries under `.github/codeql/queries/`. + # `build-mode: manual` runs our own build below so extraction is scoped + # to the main module and never co-extracts the nested query-test module. + - language: go + build-mode: manual + config-file: ./.github/codeql/codeql-config.yml + # Actions uses the stock `security-and-quality` suite. + - language: actions + build-mode: none + queries: security-and-quality steps: - name: Check out code @@ -37,7 +49,15 @@ jobs: uses: github/codeql-action/init@e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4.37.3 with: languages: ${{ matrix.language }} - queries: security-and-quality + build-mode: ${{ matrix.build-mode }} + config-file: ${{ matrix.config-file }} + queries: ${{ matrix.queries }} + + # Mirror the shipped build (see go.yml integration-tests) so the analyzed + # code matches what we release. + - name: Build Go + if: matrix.language == 'go' + run: make - name: Perform CodeQL Analysis uses: github/codeql-action/analyze@e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4.37.3 diff --git a/.gitignore b/.gitignore index 25549846a52..2d65ee64682 100644 --- a/.gitignore +++ b/.gitignore @@ -39,8 +39,13 @@ *~ vendor/ +!.github/codeql/tests/**/vendor/ gh # Test coverage artifacts coverage.out lcov.info + +# CodeQL scratch database (regenerated locally) +codeql-db/ +*.sarif diff --git a/internal/skills/discovery/discovery.go b/internal/skills/discovery/discovery.go index ff6f1286e6b..ead18a0fbfb 100644 --- a/internal/skills/discovery/discovery.go +++ b/internal/skills/discovery/discovery.go @@ -18,6 +18,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/skills/frontmatter" + "github.com/cli/cli/v2/pkg/iostreams" ) // specNamePattern matches the strict agentskills.io name spec: @@ -627,7 +628,7 @@ func fetchDescription(client *api.Client, host, owner, repo string, skill *Skill if err != nil { return "" } - result, err := frontmatter.Parse(content) + result, err := frontmatter.Parse(content.Raw()) if err != nil { return "" } @@ -867,8 +868,11 @@ func walkTree(client *api.Client, host, owner, repo, sha, prefix string, depth i return files, nil } -// FetchBlob retrieves the content of a blob by SHA. -func FetchBlob(client *api.Client, host, owner, repo, sha string) (string, error) { +// FetchBlob retrieves the content of a blob by SHA. The blob is base64-encoded +// inside the JSON response and decoded here, so it is returned as +// iostreams.Untrusted and callers must choose sanitized display or raw +// round-tripping. +func FetchBlob(client *api.Client, host, owner, repo, sha string) (iostreams.Untrusted, error) { apiPath := fmt.Sprintf("repos/%s/%s/git/blobs/%s", url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(sha)) var resp struct { SHA string `json:"sha"` @@ -876,21 +880,21 @@ func FetchBlob(client *api.Client, host, owner, repo, sha string) (string, error Encoding string `json:"encoding"` } if err := client.REST(host, "GET", apiPath, nil, &resp); err != nil { - return "", fmt.Errorf("could not fetch blob: %w", err) + return iostreams.Untrusted{}, fmt.Errorf("could not fetch blob: %w", err) } if resp.Encoding != "base64" { - return "", fmt.Errorf("unexpected blob encoding: %s", resp.Encoding) + return iostreams.Untrusted{}, fmt.Errorf("unexpected blob encoding: %s", resp.Encoding) } // GitHub API returns base64 with embedded newlines; use the StdEncoding // decoder via a reader to handle them transparently. decoded, err := io.ReadAll(base64.NewDecoder(base64.StdEncoding, strings.NewReader(resp.Content))) if err != nil { - return "", fmt.Errorf("could not decode blob content: %w", err) + return iostreams.Untrusted{}, fmt.Errorf("could not decode blob content: %w", err) } - return string(decoded), nil + return iostreams.NewUntrustedBytes(decoded), nil } // DiscoverLocalSkills finds non-hidden-dir skills in a local directory using diff --git a/internal/skills/discovery/discovery_test.go b/internal/skills/discovery/discovery_test.go index 8d1cff8c93e..0ecb8aa5708 100644 --- a/internal/skills/discovery/discovery_test.go +++ b/internal/skills/discovery/discovery_test.go @@ -772,7 +772,7 @@ func TestFetchBlob(t *testing.T) { return } require.NoError(t, err) - assert.Equal(t, tt.want, got) + assert.Equal(t, tt.want, got.Raw()) }) } } diff --git a/internal/skills/installer/installer.go b/internal/skills/installer/installer.go index 005681cac54..a0b0bfb708f 100644 --- a/internal/skills/installer/installer.go +++ b/internal/skills/installer/installer.go @@ -266,11 +266,15 @@ func installSkill(opts *Options, skill discovery.Skill, baseDir string) error { } for _, file := range files { - content, err := discovery.FetchBlob(opts.Client, opts.Host, opts.Owner, opts.Repo, file.SHA) + fetchedContent, err := discovery.FetchBlob(opts.Client, opts.Host, opts.Owner, opts.Repo, file.SHA) if err != nil { return fmt.Errorf("could not fetch %s: %w", file.Path, err) } + // Install path: the blob is written to disk verbatim, so the raw bytes + // must be preserved. + content := fetchedContent.Raw() + relPath := strings.TrimPrefix(file.Path, skill.Path+"/") safeDest, err := safeSkillDir.Join(relPath) diff --git a/pkg/cmd/agent-task/shared/log.go b/pkg/cmd/agent-task/shared/log.go index c94f5e603de..57cb5dc4b36 100644 --- a/pkg/cmd/agent-task/shared/log.go +++ b/pkg/cmd/agent-task/shared/log.go @@ -97,9 +97,9 @@ func renderLogEntry(entry chatCompletionChunkEntry, w io.Writer, io *iostreams.I } if len(choice.Delta.ToolCalls) == 0 { - if choice.Delta.Content != "" && choice.Delta.Role == "assistant" { + if !choice.Delta.Content.Empty() && choice.Delta.Role == "assistant" { // Copilot message and we should display. - renderRawMarkdown(choice.Delta.Content, w, io) + renderRawMarkdown(choice.Delta.Content.String(), w, io) } continue } @@ -107,14 +107,14 @@ func renderLogEntry(entry chatCompletionChunkEntry, w io.Writer, io *iostreams.I // Since we don't want to clear-and-reprint live progress of events, we // need to only process entries that correspond to a finished tool call. // Such entries have a non-empty Content field. - if choice.Delta.Content == "" { + if choice.Delta.Content.Empty() { continue } - if choice.Delta.ReasoningText != "" { + if !choice.Delta.ReasoningText.Empty() { // Note that this should be formatted as a normal "thought" message, // without the heading. - renderRawMarkdown(choice.Delta.ReasoningText, w, io) + renderRawMarkdown(choice.Delta.ReasoningText.String(), w, io) } for _, tc := range choice.Delta.ToolCalls { @@ -139,7 +139,7 @@ func renderLogEntry(entry chatCompletionChunkEntry, w io.Writer, io *iostreams.I } renderToolCallTitle(w, cs, fmt.Sprintf("View %s", cs.Bold(relativeFilePath(args.Path))), "") - content := stripDiffFormat(choice.Delta.Content) + content := stripDiffFormat(choice.Delta.Content.String()) if err := renderFileContentAsMarkdown(args.Path, content, w, io); err != nil { fmt.Fprintf(io.ErrOut, "\nfailed to render viewed file content: %v\n\n", err) @@ -153,9 +153,9 @@ func renderLogEntry(entry chatCompletionChunkEntry, w io.Writer, io *iostreams.I renderToolCallTitle(w, cs, "Run Bash command", "") } - contentWithCommand := choice.Delta.Content + contentWithCommand := choice.Delta.Content.String() if v.Command != "" { - contentWithCommand = fmt.Sprintf("$ %s\n%s", v.Command, choice.Delta.Content) + contentWithCommand = fmt.Sprintf("$ %s\n%s", v.Command, choice.Delta.Content.String()) } if err := renderFileContentAsMarkdown("commands.sh", contentWithCommand, w, io); err != nil { fmt.Fprintf(io.ErrOut, "\nfailed to render bash command output: %v\n\n", err) @@ -220,9 +220,9 @@ func renderLogEntry(entry chatCompletionChunkEntry, w io.Writer, io *iostreams.I } // TODO: KW I wasn't able to get this case to populate ever. - if choice.Delta.Content != "" { + if !choice.Delta.Content.Empty() { // Try to treat this as JSON - if err := renderContentAsJSONMarkdown("", choice.Delta.Content, w, io); err != nil { + if err := renderContentAsJSONMarkdown("", choice.Delta.Content.String(), w, io); err != nil { fmt.Fprintf(io.ErrOut, "\nfailed to render progress update content: %v\n", err) } } @@ -247,9 +247,9 @@ func renderLogEntry(entry chatCompletionChunkEntry, w io.Writer, io *iostreams.I } renderToolCallTitle(w, cs, "Edit", cs.Bold(relativeFilePath(args.Path))) - if err := renderFileContentAsMarkdown("output.diff", choice.Delta.Content, w, io); err != nil { + if err := renderFileContentAsMarkdown("output.diff", choice.Delta.Content.String(), w, io); err != nil { fmt.Fprintf(io.ErrOut, "\nfailed to render str_replace diff: %v\n\n", err) - fmt.Fprintln(io.ErrOut, choice.Delta.Content) + fmt.Fprintln(io.ErrOut, choice.Delta.Content.String()) } default: // Unknown tool call. For example for "codeql_checker": @@ -257,7 +257,7 @@ func renderLogEntry(entry chatCompletionChunkEntry, w io.Writer, io *iostreams.I renderGenericToolCall(w, cs, name) // If it's JSON, treat it as such, otherwise we skip whatever the content is. - _ = renderContentAsJSONMarkdown("Output:", choice.Delta.Content, w, io) + _ = renderContentAsJSONMarkdown("Output:", choice.Delta.Content.String(), w, io) // The entirety of the args can be treated as "input" to the tool call. // We try to render it as JSON, but if that fails, just skip it. @@ -500,9 +500,9 @@ type chatCompletionChunkEntry struct { Object string `json:"object"` Choices []struct { Delta struct { - ReasoningText string `json:"reasoning_text"` - Content string `json:"content"` - Role string `json:"role"` + ReasoningText iostreams.Untrusted `json:"reasoning_text"` + Content iostreams.Untrusted `json:"content"` + Role string `json:"role"` ToolCalls []struct { Function struct { Name string `json:"name"` diff --git a/pkg/cmd/api/api.go b/pkg/cmd/api/api.go index 4a87e0f8cb9..5b85f987ca4 100644 --- a/pkg/cmd/api/api.go +++ b/pkg/cmd/api/api.go @@ -59,6 +59,8 @@ type ApiOptions struct { CacheTTL time.Duration FilterOutput string Verbose bool + + AllowEscapeSequences bool } func NewCmdApi(f *cmdutil.Factory, runF func(*ApiOptions) error) *cobra.Command { @@ -298,6 +300,7 @@ func NewCmdApi(f *cmdutil.Factory, runF func(*ApiOptions) error) *cobra.Command cmd.Flags().StringVarP(&opts.FilterOutput, "jq", "q", "", "Query to select values from the response using jq syntax") cmd.Flags().DurationVar(&opts.CacheTTL, "cache", 0, "Cache the response, e.g. \"3600s\", \"60m\", \"1h\"") cmd.Flags().BoolVar(&opts.Verbose, "verbose", false, "Include full HTTP request and response in the output") + cmd.Flags().BoolVar(&opts.AllowEscapeSequences, "allow-escape-sequences", false, "Allow printing terminal escape sequences") return cmd } @@ -331,7 +334,13 @@ func apiRun(opts *ApiOptions) error { } } - var bodyWriter io.Writer = opts.IO.Out + // Response content funnels through ContentOut. It stays in passthrough here: + // JSON is sanitized by the transport and the jq/template/jsoncolor paths emit + // our own formatting, so only a raw non-JSON body needs neutralizing, done at + // its copy below. + opts.IO.SetContentSanitization(false) + + var bodyWriter io.Writer = opts.IO.ContentOut var headersWriter io.Writer = opts.IO.Out if opts.Silent { bodyWriter = io.Discard @@ -518,7 +527,20 @@ func processResponse(resp *http.Response, opts *ApiOptions, bodyWriter, headersW isLastPage: isLastPage, } } - _, err = io.Copy(bodyWriter, responseBody) + // A raw non-JSON body is the only response the transport does not sanitize. + // It is faithful byte output, so binary bound for a terminal and text + // carrying escape sequences are refused; the opt-out flag and discarded + // output stream verbatim. + if !isJSON && !opts.AllowEscapeSequences && bodyWriter != io.Discard { + err = iostreams.CopyGuardedContent(bodyWriter, responseBody, opts.IO.IsStdoutTTY()) + if binErr, ok := errors.AsType[iostreams.BinaryTerminalError](err); ok { + err = fmt.Errorf("%w; redirect or pipe stdout to save it, or pass --allow-escape-sequences to output it anyway", binErr) + } else if errors.Is(err, iostreams.ErrEscapeSequence) { + err = errors.New("the response contains terminal escape sequences; pass --allow-escape-sequences to output it anyway") + } + } else { + _, err = io.Copy(bodyWriter, responseBody) + } } if err != nil { return diff --git a/pkg/cmd/api/api_test.go b/pkg/cmd/api/api_test.go index bc29b1eb73c..33a45543579 100644 --- a/pkg/cmd/api/api_test.go +++ b/pkg/cmd/api/api_test.go @@ -428,6 +428,7 @@ func Test_apiRun(t *testing.T) { options ApiOptions httpResponse *http.Response err error + errMsg string stdout string stderr string isatty bool @@ -656,6 +657,81 @@ func Test_apiRun(t *testing.T) { stderr: ``, isatty: true, }, + { + name: "refuses escape sequences in non-JSON body on a TTY", + httpResponse: &http.Response{ + StatusCode: 200, + Body: io.NopCloser(bytes.NewBufferString("\x1b[31mred\x1b[m")), + Header: http.Header{"Content-Type": []string{"text/plain"}}, + }, + errMsg: "the response contains terminal escape sequences; pass --allow-escape-sequences to output it anyway", + stdout: ``, + stderr: ``, + isatty: true, + }, + { + name: "passes escape sequences through with --allow-escape-sequences on a TTY", + options: ApiOptions{ + AllowEscapeSequences: true, + }, + httpResponse: &http.Response{ + StatusCode: 200, + Body: io.NopCloser(bytes.NewBufferString("\x1b[31mred\x1b[m")), + Header: http.Header{"Content-Type": []string{"text/plain"}}, + }, + err: nil, + stdout: "\x1b[31mred\x1b[m", + stderr: ``, + isatty: true, + }, + { + name: "refuses escape sequences in non-JSON body when piped", + httpResponse: &http.Response{ + StatusCode: 200, + Body: io.NopCloser(bytes.NewBufferString("\x1b[31mred\x1b[m")), + Header: http.Header{"Content-Type": []string{"text/plain"}}, + }, + errMsg: "the response contains terminal escape sequences; pass --allow-escape-sequences to output it anyway", + stdout: ``, + stderr: ``, + isatty: false, + }, + { + name: "outputs clean non-JSON text on a TTY", + httpResponse: &http.Response{ + StatusCode: 200, + Body: io.NopCloser(bytes.NewBufferString("plain readme text\n")), + Header: http.Header{"Content-Type": []string{"text/plain"}}, + }, + err: nil, + stdout: "plain readme text\n", + stderr: ``, + isatty: true, + }, + { + name: "streams binary non-JSON body when piped", + httpResponse: &http.Response{ + StatusCode: 200, + Body: io.NopCloser(bytes.NewReader(append([]byte("\x89PNG\r\n\x1a\n"), make([]byte, 16)...))), + Header: http.Header{"Content-Type": []string{"application/octet-stream"}}, + }, + err: nil, + stdout: string(append([]byte("\x89PNG\r\n\x1a\n"), make([]byte, 16)...)), + stderr: ``, + isatty: false, + }, + { + name: "refuses binary non-JSON body on a TTY", + httpResponse: &http.Response{ + StatusCode: 200, + Body: io.NopCloser(bytes.NewReader(append([]byte("\x89PNG\r\n\x1a\n"), make([]byte, 16)...))), + Header: http.Header{"Content-Type": []string{"application/octet-stream"}}, + }, + errMsg: "refusing to output binary content (image/png) to the terminal; redirect or pipe stdout to save it, or pass --allow-escape-sequences to output it anyway", + stdout: ``, + stderr: ``, + isatty: true, + }, } for _, tt := range tests { @@ -675,7 +751,11 @@ func Test_apiRun(t *testing.T) { } err := apiRun(&tt.options) - if err != tt.err { + if tt.errMsg != "" { + if err == nil || err.Error() != tt.errMsg { + t.Errorf("expected error %q, got %v", tt.errMsg, err) + } + } else if err != tt.err { t.Errorf("expected error %v, got %v", tt.err, err) } diff --git a/pkg/cmd/codespace/logs.go b/pkg/cmd/codespace/logs.go index 37b30121760..4f8a420c7dd 100644 --- a/pkg/cmd/codespace/logs.go +++ b/pkg/cmd/codespace/logs.go @@ -88,6 +88,16 @@ func (a *App) Logs(ctx context.Context, selector *CodespaceSelector, follow bool return fmt.Errorf("remote command: %w", err) } + // The log file is external content. On a terminal, route it through + // ContentOut to neutralize escape sequences (assigning a writer other than an + // *os.File also forces the remote output through this process so the sanitizer + // runs). When piped, pass the bytes through unchanged: there is no live + // terminal to manipulate, and a follow stream cannot be buffered to fail closed. + if !a.io.IsStdoutTTY() { + a.io.SetContentSanitization(false) + } + cmd.Stdout = a.io.ContentOut + tunnelClosed := make(chan error, 1) go func() { opts := portforwarder.ForwardPortOpts{ diff --git a/pkg/cmd/gist/edit/edit.go b/pkg/cmd/gist/edit/edit.go index 6f00c906cbd..d52aaac1848 100644 --- a/pkg/cmd/gist/edit/edit.go +++ b/pkg/cmd/gist/edit/edit.go @@ -292,7 +292,9 @@ func editRun(opts *EditOptions) error { return err } - gistFile.Content = fullContent + // Round-trip path: the content is opened in an editor and sent + // back to the API, so the raw bytes must be preserved verbatim. + gistFile.Content = fullContent.Raw() } } diff --git a/pkg/cmd/gist/shared/shared.go b/pkg/cmd/gist/shared/shared.go index 305e2f6426b..61f09af7c57 100644 --- a/pkg/cmd/gist/shared/shared.go +++ b/pkg/cmd/gist/shared/shared.go @@ -248,28 +248,31 @@ func PromptGists(prompter prompter.Prompter, client *http.Client, host string, c return &gists[result], nil } -func GetRawGistFile(httpClient *http.Client, rawURL string) (string, error) { +// GetRawGistFile fetches the full content of a gist file from its raw URL. The +// bytes are external content, so they are returned as iostreams.Untrusted to +// force callers to choose between sanitized display and raw round-tripping. +func GetRawGistFile(httpClient *http.Client, rawURL string) (iostreams.Untrusted, error) { req, err := http.NewRequest("GET", rawURL, nil) if err != nil { - return "", err + return iostreams.Untrusted{}, err } resp, err := httpClient.Do(req) if err != nil { - return "", err + return iostreams.Untrusted{}, err } defer resp.Body.Close() if resp.StatusCode != http.StatusOK { - return "", api.HandleHTTPError(resp) + return iostreams.Untrusted{}, api.HandleHTTPError(resp) } body, err := io.ReadAll(resp.Body) if err != nil { - return "", err + return iostreams.Untrusted{}, err } - return string(body), nil + return iostreams.NewUntrustedBytes(body), nil } diff --git a/pkg/cmd/gist/shared/shared_test.go b/pkg/cmd/gist/shared/shared_test.go index d75ebc2b72d..28587c1409b 100644 --- a/pkg/cmd/gist/shared/shared_test.go +++ b/pkg/cmd/gist/shared/shared_test.go @@ -307,7 +307,7 @@ func TestGetRawGistFile(t *testing.T) { } } else { assert.NoError(t, err) - assert.Equal(t, tt.want, result) + assert.Equal(t, tt.want, result.Raw()) } reg.Verify(t) diff --git a/pkg/cmd/gist/view/view.go b/pkg/cmd/gist/view/view.go index 9eb906cde8f..0cff1b5d5b1 100644 --- a/pkg/cmd/gist/view/view.go +++ b/pkg/cmd/gist/view/view.go @@ -1,6 +1,7 @@ package view import ( + "errors" "fmt" "net/http" "sort" @@ -33,6 +34,8 @@ type ViewOptions struct { Raw bool Web bool ListFiles bool + + AllowEscapeSequences bool } func NewCmdView(f *cmdutil.Factory, runF func(*ViewOptions) error) *cobra.Command { @@ -69,12 +72,18 @@ func NewCmdView(f *cmdutil.Factory, runF func(*ViewOptions) error) *cobra.Comman cmd.Flags().BoolVarP(&opts.Web, "web", "w", false, "Open gist in the browser") cmd.Flags().BoolVar(&opts.ListFiles, "files", false, "List file names from the gist") cmd.Flags().StringVarP(&opts.Filename, "filename", "f", "", "Display a single file from the gist") + cmd.Flags().BoolVar(&opts.AllowEscapeSequences, "allow-escape-sequences", false, "Allow printing terminal escape sequences") return cmd } func viewRun(opts *ViewOptions) error { gistID := opts.Selector + + if opts.AllowEscapeSequences { + opts.IO.SetContentSanitization(false) + } + client, err := opts.HttpClient() if err != nil { return err @@ -136,17 +145,19 @@ func viewRun(opts *ViewOptions) error { defer opts.IO.StopPager() render := func(gf *shared.GistFile) error { + // Treat the file content as untrusted external bytes. The truncated + // path fetches the full content from the raw URL. + content := iostreams.NewUntrusted(gf.Content) if gf.Truncated { fullContent, err := shared.GetRawGistFile(client, gf.RawURL) - if err != nil { return err } - gf.Content = fullContent + content = fullContent } - if shared.IsBinaryContents([]byte(gf.Content)) { + if shared.IsBinaryContents(content.RawBytes()) { if len(gist.Files) == 1 || opts.Filename != "" { return fmt.Errorf("error: file is binary") } @@ -155,7 +166,10 @@ func viewRun(opts *ViewOptions) error { } if strings.Contains(gf.Type, "markdown") && !opts.Raw { - rendered, err := markdown.Render(gf.Content, + // Markdown rendering emits application-styled output to Out, so its + // input is sanitized here; --allow-escape-sequences applies to the + // raw dump below. + rendered, err := markdown.Render(content.String(), markdown.WithTheme(opts.IO.TerminalTheme()), markdown.WithWrap(opts.IO.TerminalWidth())) if err != nil { @@ -165,11 +179,22 @@ func viewRun(opts *ViewOptions) error { return err } - if _, err := fmt.Fprint(opts.IO.Out, gf.Content); err != nil { + // Raw dump. On a terminal, ContentOut renders escape sequences inert. + // When the output is piped, refuse content carrying escape sequences + // rather than silently rewriting the bytes; --allow-escape-sequences + // forces raw. + if !opts.AllowEscapeSequences && !opts.IO.IsStdoutTTY() { + if iostreams.ContainsEscapeSequence(content.RawBytes()) { + return errors.New("gist file contains terminal escape sequences; pass --allow-escape-sequences to view it anyway") + } + opts.IO.SetContentSanitization(false) + } + raw := content.Raw() + if _, err := fmt.Fprint(opts.IO.ContentOut, raw); err != nil { return err } - if !strings.HasSuffix(gf.Content, "\n") { - _, err := fmt.Fprint(opts.IO.Out, "\n") + if !strings.HasSuffix(raw, "\n") { + _, err := fmt.Fprint(opts.IO.ContentOut, "\n") return err } diff --git a/pkg/cmd/gist/view/view_test.go b/pkg/cmd/gist/view/view_test.go index 85c2b7ad86b..dcfe561ef66 100644 --- a/pkg/cmd/gist/view/view_test.go +++ b/pkg/cmd/gist/view/view_test.go @@ -147,6 +147,100 @@ func Test_viewRun(t *testing.T) { }, wantOut: "bwhiizzzbwhuiiizzzz\n", }, + { + name: "truncated raw file with escape sequences is sanitized on a terminal", + isTTY: true, + opts: &ViewOptions{ + Selector: "1234", + ListFiles: false, + }, + mockGist: &shared.Gist{ + Files: map[string]*shared.GistFile{ + "escaped.txt": { + Type: "text/plain", + Content: "", + Truncated: true, + RawURL: "https://gist.githubusercontent.com/user/1234/raw/escaped.txt", + }, + }, + }, + wantOut: "danger^[[31m\n", + }, + { + name: "piped truncated raw file with escape sequences is refused", + isTTY: false, + opts: &ViewOptions{ + Selector: "1234", + ListFiles: false, + }, + mockGist: &shared.Gist{ + Files: map[string]*shared.GistFile{ + "escaped.txt": { + Type: "text/plain", + Content: "", + Truncated: true, + RawURL: "https://gist.githubusercontent.com/user/1234/raw/escaped.txt", + }, + }, + }, + wantErr: "gist file contains terminal escape sequences; pass --allow-escape-sequences to view it anyway", + }, + { + name: "piped truncated clean file passes through raw", + isTTY: false, + opts: &ViewOptions{ + Selector: "1234", + ListFiles: false, + }, + mockGist: &shared.Gist{ + Files: map[string]*shared.GistFile{ + "clean-truncated.txt": { + Type: "text/plain", + Content: "", + Truncated: true, + RawURL: "https://gist.githubusercontent.com/user/1234/raw/clean-truncated.txt", + }, + }, + }, + wantOut: "clean text\n", + }, + { + name: "piped truncated file with escape sequences passes through with --allow-escape-sequences", + isTTY: false, + opts: &ViewOptions{ + Selector: "1234", + ListFiles: false, + AllowEscapeSequences: true, + }, + mockGist: &shared.Gist{ + Files: map[string]*shared.GistFile{ + "escaped.txt": { + Type: "text/plain", + Content: "", + Truncated: true, + RawURL: "https://gist.githubusercontent.com/user/1234/raw/escaped.txt", + }, + }, + }, + wantOut: "danger\x1b[31m\n", + }, + { + name: "piped inline file escapes are already neutralized by the JSON transport", + isTTY: false, + opts: &ViewOptions{ + Selector: "1234", + ListFiles: false, + }, + mockGist: &shared.Gist{ + Files: map[string]*shared.GistFile{ + "inline-escaped.txt": { + Type: "text/plain", + Content: "danger\x1b[31m", + }, + }, + }, + wantOut: "danger^[[31m\n", + }, { name: "one file, no ID supplied", isTTY: true, @@ -453,6 +547,12 @@ func Test_viewRun(t *testing.T) { } else if filename == "also-truncated.txt" { reg.Register(httpmock.REST("GET", "user/1234/raw/also-truncated.txt"), httpmock.StringResponse("This is the full content of the also-truncated file retrieved from raw URL")) + } else if filename == "escaped.txt" { + reg.Register(httpmock.REST("GET", "user/1234/raw/escaped.txt"), + httpmock.StringResponse("danger\x1b[31m")) + } else if filename == "clean-truncated.txt" { + reg.Register(httpmock.REST("GET", "user/1234/raw/clean-truncated.txt"), + httpmock.StringResponse("clean text")) } } } diff --git a/pkg/cmd/pr/diff/diff.go b/pkg/cmd/pr/diff/diff.go index 91dc14ef472..7555442a8ed 100644 --- a/pkg/cmd/pr/diff/diff.go +++ b/pkg/cmd/pr/diff/diff.go @@ -10,8 +10,6 @@ import ( "path" "regexp" "strings" - "unicode" - "unicode/utf8" "github.com/MakeNowJust/heredoc" "github.com/cli/cli/v2/api" @@ -22,6 +20,7 @@ import ( "github.com/cli/cli/v2/pkg/cmd/pr/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" + "github.com/cli/go-gh/v2/pkg/asciisanitizer" "github.com/spf13/cobra" "golang.org/x/text/transform" ) @@ -39,6 +38,8 @@ type DiffOptions struct { NameOnly bool BrowserMode bool Exclude []string + + AllowEscapeSequences bool } func NewCmdDiff(f *cmdutil.Factory, runF func(*DiffOptions) error) *cobra.Command { @@ -64,6 +65,10 @@ func NewCmdDiff(f *cmdutil.Factory, runF func(*DiffOptions) error) *cobra.Comman Use %[1]s--exclude%[1]s to filter out files matching a glob pattern. The pattern uses forward slashes as path separators on all platforms. You can repeat the flag to exclude multiple patterns. + + By default, terminal escape sequences in the diff are neutralized, since + they could manipulate your terminal. Pass %[1]s--allow-escape-sequences%[1]s to + print the diff verbatim, for example when piping a patch to another program. `, "`"), Example: heredoc.Doc(` # See diff for current branch @@ -113,6 +118,7 @@ func NewCmdDiff(f *cmdutil.Factory, runF func(*DiffOptions) error) *cobra.Comman cmd.Flags().BoolVar(&opts.NameOnly, "name-only", false, "Display only names of changed files") cmd.Flags().BoolVarP(&opts.BrowserMode, "web", "w", false, "Open the pull request diff in the browser") cmd.Flags().StringSliceVarP(&opts.Exclude, "exclude", "e", nil, "Exclude files matching glob `patterns` from the diff") + cmd.Flags().BoolVar(&opts.AllowEscapeSequences, "allow-escape-sequences", false, "Allow printing terminal escape sequences") return cmd } @@ -163,8 +169,12 @@ func diffRun(opts *DiffOptions) error { } diff = filtered } - if opts.IO.IsStdoutTTY() { - diff = sanitizedReader(diff) + // A terminal shows escape sequences inert through ContentOut; piped output is + // faithful, so a diff carrying escape sequences is refused rather than silently + // altered. --allow-escape-sequences streams raw on both. The colored path + // always neutralizes, since it is terminal-bound. + if opts.AllowEscapeSequences { + opts.IO.SetContentSanitization(false) } if err := opts.IO.StartPager(); err == nil { @@ -174,15 +184,28 @@ func diffRun(opts *DiffOptions) error { } if opts.NameOnly { - return changedFilesNames(opts.IO.Out, diff) + return changedFilesNames(opts.IO.ContentOut, diff) + } + + if opts.UseColor { + return colorDiffLines(opts.IO.Out, sanitizedReader(diff)) } - if !opts.UseColor { - _, err = io.Copy(opts.IO.Out, diff) + if !opts.AllowEscapeSequences && !opts.IO.IsStdoutTTY() { + data, err := io.ReadAll(diff) + if err != nil { + return err + } + if iostreams.ContainsEscapeSequence(data) { + return errors.New("the diff contains terminal escape sequences; pass --allow-escape-sequences to output it anyway") + } + opts.IO.SetContentSanitization(false) + _, err = opts.IO.ContentOut.Write(data) return err } - return colorDiffLines(opts.IO.Out, diff) + _, err = io.Copy(opts.IO.ContentOut, diff) + return err } func fetchDiff(httpClient *http.Client, baseRepo ghrepo.Interface, prNumber int, asPatch bool) (io.ReadCloser, error) { @@ -333,56 +356,7 @@ func changedFilesNames(w io.Writer, r io.Reader) error { } func sanitizedReader(r io.Reader) io.Reader { - return transform.NewReader(r, sanitizer{}) -} - -// sanitizer replaces non-printable characters with their printable representations -type sanitizer struct{ transform.NopResetter } - -// Transform implements transform.Transformer. -func (t sanitizer) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) { - for r, size := rune(0), 0; nSrc < len(src); { - if r = rune(src[nSrc]); r < utf8.RuneSelf { - size = 1 - } else if r, size = utf8.DecodeRune(src[nSrc:]); size == 1 && !atEOF && !utf8.FullRune(src[nSrc:]) { - // Invalid rune. - err = transform.ErrShortSrc - break - } - - if isPrint(r) { - if nDst+size > len(dst) { - err = transform.ErrShortDst - break - } - for i := 0; i < size; i++ { - dst[nDst] = src[nSrc] - nDst++ - nSrc++ - } - continue - } else { - nSrc += size - } - - replacement := fmt.Sprintf("\\u{%02x}", r) - - if nDst+len(replacement) > len(dst) { - err = transform.ErrShortDst - break - } - - for _, c := range replacement { - dst[nDst] = byte(c) - nDst++ - } - } - return -} - -// isPrint reports if a rune is safe to be printed to a terminal -func isPrint(r rune) bool { - return r == '\n' || r == '\r' || r == '\t' || unicode.IsPrint(r) + return transform.NewReader(r, &asciisanitizer.Sanitizer{}) } var diffHeaderRegexp = regexp.MustCompile(`(?:^|\n)diff\s--git.*\s("?)b/(.*)`) diff --git a/pkg/cmd/pr/diff/diff_test.go b/pkg/cmd/pr/diff/diff_test.go index 6a91ba9ca96..b95ac8a8311 100644 --- a/pkg/cmd/pr/diff/diff_test.go +++ b/pkg/cmd/pr/diff/diff_test.go @@ -123,6 +123,16 @@ func Test_NewCmdDiff(t *testing.T) { BrowserMode: true, }, }, + { + name: "allow escape sequences", + args: "--allow-escape-sequences", + isTTY: true, + want: DiffOptions{ + SelectorArg: "", + UseColor: true, + AllowEscapeSequences: true, + }, + }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { @@ -163,6 +173,7 @@ func Test_NewCmdDiff(t *testing.T) { assert.Equal(t, tt.want.UseColor, opts.UseColor) assert.Equal(t, tt.want.BrowserMode, opts.BrowserMode) assert.Equal(t, tt.want.Exclude, opts.Exclude) + assert.Equal(t, tt.want.AllowEscapeSequences, opts.AllowEscapeSequences) }) } } @@ -173,9 +184,11 @@ func Test_diffRun(t *testing.T) { tests := []struct { name string opts DiffOptions + notTTY bool wantFields []string wantStdout string wantStderr string + wantErr string wantBrowsedURL string httpStubs func(*httpmock.Registry) }{ @@ -284,6 +297,57 @@ index f2b4805c..3d7bd0f9 100644 wantStderr: "Opening https://github.com/OWNER/REPO/pull/123/files in your browser.\n", wantBrowsedURL: "https://github.com/OWNER/REPO/pull/123/files", }, + { + name: "neutralizes escape sequences by default", + opts: DiffOptions{ + SelectorArg: "123", + UseColor: false, + }, + wantFields: []string{"number"}, + wantStdout: "diff --git a/f b/f\n+ hello ^[[m world\n", + httpStubs: func(reg *httpmock.Registry) { + stubDiffRequest(reg, "application/vnd.github.v3.diff", "diff --git a/f b/f\n+ hello \x1b[m world\n") + }, + }, + { + name: "passes escape sequences through with --allow-escape-sequences", + opts: DiffOptions{ + SelectorArg: "123", + UseColor: false, + AllowEscapeSequences: true, + }, + wantFields: []string{"number"}, + wantStdout: "diff --git a/f b/f\n+ hello \x1b[m world\n", + httpStubs: func(reg *httpmock.Registry) { + stubDiffRequest(reg, "application/vnd.github.v3.diff", "diff --git a/f b/f\n+ hello \x1b[m world\n") + }, + }, + { + name: "piped diff with escape sequences is refused", + opts: DiffOptions{ + SelectorArg: "123", + UseColor: false, + }, + notTTY: true, + wantFields: []string{"number"}, + wantErr: "the diff contains terminal escape sequences; pass --allow-escape-sequences to output it anyway", + httpStubs: func(reg *httpmock.Registry) { + stubDiffRequest(reg, "application/vnd.github.v3.diff", "diff --git a/f b/f\n+ hello \x1b[m world\n") + }, + }, + { + name: "piped clean diff passes through raw", + opts: DiffOptions{ + SelectorArg: "123", + UseColor: false, + }, + notTTY: true, + wantFields: []string{"number"}, + wantStdout: "diff --git a/f b/f\n+ hello world\n", + httpStubs: func(reg *httpmock.Registry) { + stubDiffRequest(reg, "application/vnd.github.v3.diff", "diff --git a/f b/f\n+ hello world\n") + }, + }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { @@ -300,7 +364,7 @@ index f2b4805c..3d7bd0f9 100644 tt.opts.Browser = browser ios, _, stdout, stderr := iostreams.Test() - ios.SetStdoutTTY(true) + ios.SetStdoutTTY(!tt.notTTY) tt.opts.IO = ios finder := shared.NewMockFinder("123", pr, ghrepo.New("OWNER", "REPO")) @@ -308,7 +372,11 @@ index f2b4805c..3d7bd0f9 100644 tt.opts.Finder = finder err := diffRun(&tt.opts) - assert.NoError(t, err) + if tt.wantErr != "" { + assert.EqualError(t, err, tt.wantErr) + } else { + assert.NoError(t, err) + } assert.Equal(t, tt.wantStdout, stdout.String()) assert.Equal(t, tt.wantStderr, stderr.String()) @@ -569,7 +637,7 @@ func Test_matchesAny(t *testing.T) { func Test_sanitizedReader(t *testing.T) { input := strings.NewReader("\t hello \x1B[m world! ăѣ𝔠ծề\r\n") - expected := "\t hello \\u{1b}[m world! ăѣ𝔠ծề\r\n" + expected := "\t hello ^[[m world! ăѣ𝔠ծề\r\n" err := iotest.TestReader(sanitizedReader(input), []byte(expected)) if err != nil { diff --git a/pkg/cmd/release/download/download.go b/pkg/cmd/release/download/download.go index e86b36a41d7..d132de5b631 100644 --- a/pkg/cmd/release/download/download.go +++ b/pkg/cmd/release/download/download.go @@ -38,6 +38,8 @@ type DownloadOptions struct { Concurrency int ArchiveType string + + AllowEscapeSequences bool } func NewCmdDownload(f *cmdutil.Factory, runF func(*DownloadOptions) error) *cobra.Command { @@ -110,6 +112,7 @@ func NewCmdDownload(f *cmdutil.Factory, runF func(*DownloadOptions) error) *cobr cmd.Flags().StringVarP(&opts.ArchiveType, "archive", "A", "", "Download the source code archive in the specified `format` (zip or tar.gz)") cmd.Flags().BoolVar(&opts.OverwriteExisting, "clobber", false, "Overwrite existing files of the same name") cmd.Flags().BoolVar(&opts.SkipExisting, "skip-existing", false, "Skip downloading when files of the same name exist") + cmd.Flags().BoolVar(&opts.AllowEscapeSequences, "allow-escape-sequences", false, "Allow printing terminal escape sequences when writing an asset to standard output") cmdutil.DisableAuthCheck(cmd) @@ -214,12 +217,20 @@ func downloadRun(opts *DownloadOptions) error { return fmt.Errorf("unable to write more than one asset with `--output`, got %d assets", len(toDownload)) } + // An asset written to standard output is external content. It funnels through + // ContentOut so the sink is auditable; the safety decision (refuse binary bound + // for a terminal or escape sequences in text, unless --allow-escape-sequences) + // is made per copy below. Writing to a file keeps the raw bytes. + opts.IO.SetContentSanitization(false) + dest := destinationWriter{ file: opts.OutputFile, dir: opts.Destination, skipExisting: opts.SkipExisting, overwrite: opts.OverwriteExisting, - stdout: opts.IO.Out, + stdout: opts.IO.ContentOut, + allowEscapes: opts.AllowEscapeSequences, + isTTY: opts.IO.IsStdoutTTY(), } return downloadAssets(&dest, httpClient, toDownload, opts.Concurrency, isArchive, opts.IO) @@ -347,6 +358,8 @@ type destinationWriter struct { skipExisting bool overwrite bool stdout io.Writer + allowEscapes bool + isTTY bool } func (w destinationWriter) makePath(name string) string { @@ -389,7 +402,16 @@ func (w destinationWriter) check(fp string) error { func (w destinationWriter) Copy(name string, r io.Reader) (copyErr error) { fp := w.makePath(name) if fp == "-" { - _, copyErr = io.Copy(w.stdout, r) + if w.allowEscapes { + _, copyErr = io.Copy(w.stdout, r) + return + } + copyErr = iostreams.CopyGuardedContent(w.stdout, r, w.isTTY) + if binErr, ok := errors.AsType[iostreams.BinaryTerminalError](copyErr); ok { + copyErr = fmt.Errorf("%w; use `--output` to save it to a file, or pass --allow-escape-sequences to output it anyway", binErr) + } else if errors.Is(copyErr, iostreams.ErrEscapeSequence) { + copyErr = errors.New("the asset contains terminal escape sequences; use `--output` to save it to a file, or pass --allow-escape-sequences to output it anyway") + } return } if copyErr = w.check(fp); copyErr != nil { diff --git a/pkg/cmd/release/download/download_test.go b/pkg/cmd/release/download/download_test.go index 549ae62d59b..855380c3856 100644 --- a/pkg/cmd/release/download/download_test.go +++ b/pkg/cmd/release/download/download_test.go @@ -488,6 +488,129 @@ func Test_downloadRun(t *testing.T) { wantStdout: `1234`, wantStderr: ``, }, + { + name: "download single asset to standard output refuses escape sequences on a TTY", + isTTY: true, + opts: DownloadOptions{ + OutputFile: "-", + TagName: "v1.2.3", + Destination: "", + Concurrency: 2, + FilePatterns: []string{"*windows-32bit.zip"}, + }, + httpStubs: func(reg *httpmock.Registry) { + shared.StubFetchRelease(t, reg, "OWNER", "REPO", "v1.2.3", `{ + "assets": [ + { "name": "windows-32bit.zip", "size": 12, + "url": "https://api.github.com/assets/1234" } + ], + "tarball_url": "https://api.github.com/repos/OWNER/REPO/tarball/v1.2.3", + "zipball_url": "https://api.github.com/repos/OWNER/REPO/zipball/v1.2.3" + }`) + + reg.Register(httpmock.REST("GET", "assets/1234"), httpmock.StringResponse("\x1b[31mred\x1b[m")) + }, + wantErr: "the asset contains terminal escape sequences; use `--output` to save it to a file, or pass --allow-escape-sequences to output it anyway", + }, + { + name: "download single asset to standard output passes escape sequences through with --allow-escape-sequences", + isTTY: true, + opts: DownloadOptions{ + OutputFile: "-", + TagName: "v1.2.3", + Destination: "", + Concurrency: 2, + FilePatterns: []string{"*windows-32bit.zip"}, + AllowEscapeSequences: true, + }, + httpStubs: func(reg *httpmock.Registry) { + shared.StubFetchRelease(t, reg, "OWNER", "REPO", "v1.2.3", `{ + "assets": [ + { "name": "windows-32bit.zip", "size": 12, + "url": "https://api.github.com/assets/1234" } + ], + "tarball_url": "https://api.github.com/repos/OWNER/REPO/tarball/v1.2.3", + "zipball_url": "https://api.github.com/repos/OWNER/REPO/zipball/v1.2.3" + }`) + + reg.Register(httpmock.REST("GET", "assets/1234"), httpmock.StringResponse("\x1b[31mred\x1b[m")) + }, + wantStdout: "\x1b[31mred\x1b[m", + wantStderr: ``, + }, + { + name: "download single asset to standard output refuses escape sequences when piped", + isTTY: false, + opts: DownloadOptions{ + OutputFile: "-", + TagName: "v1.2.3", + Destination: "", + Concurrency: 2, + FilePatterns: []string{"*windows-32bit.zip"}, + }, + httpStubs: func(reg *httpmock.Registry) { + shared.StubFetchRelease(t, reg, "OWNER", "REPO", "v1.2.3", `{ + "assets": [ + { "name": "windows-32bit.zip", "size": 12, + "url": "https://api.github.com/assets/1234" } + ], + "tarball_url": "https://api.github.com/repos/OWNER/REPO/tarball/v1.2.3", + "zipball_url": "https://api.github.com/repos/OWNER/REPO/zipball/v1.2.3" + }`) + + reg.Register(httpmock.REST("GET", "assets/1234"), httpmock.StringResponse("\x1b[31mred\x1b[m")) + }, + wantErr: "the asset contains terminal escape sequences; use `--output` to save it to a file, or pass --allow-escape-sequences to output it anyway", + }, + { + name: "download single binary asset to standard output streams raw when piped", + isTTY: false, + opts: DownloadOptions{ + OutputFile: "-", + TagName: "v1.2.3", + Destination: "", + Concurrency: 2, + FilePatterns: []string{"*windows-32bit.zip"}, + }, + httpStubs: func(reg *httpmock.Registry) { + shared.StubFetchRelease(t, reg, "OWNER", "REPO", "v1.2.3", `{ + "assets": [ + { "name": "windows-32bit.zip", "size": 24, + "url": "https://api.github.com/assets/1234" } + ], + "tarball_url": "https://api.github.com/repos/OWNER/REPO/tarball/v1.2.3", + "zipball_url": "https://api.github.com/repos/OWNER/REPO/zipball/v1.2.3" + }`) + + reg.Register(httpmock.REST("GET", "assets/1234"), httpmock.StringResponse(string(append([]byte("\x89PNG\r\n\x1a\n"), make([]byte, 16)...)))) + }, + wantStdout: string(append([]byte("\x89PNG\r\n\x1a\n"), make([]byte, 16)...)), + wantStderr: ``, + }, + { + name: "download single binary asset to standard output is refused on a TTY", + isTTY: true, + opts: DownloadOptions{ + OutputFile: "-", + TagName: "v1.2.3", + Destination: "", + Concurrency: 2, + FilePatterns: []string{"*windows-32bit.zip"}, + }, + httpStubs: func(reg *httpmock.Registry) { + shared.StubFetchRelease(t, reg, "OWNER", "REPO", "v1.2.3", `{ + "assets": [ + { "name": "windows-32bit.zip", "size": 24, + "url": "https://api.github.com/assets/1234" } + ], + "tarball_url": "https://api.github.com/repos/OWNER/REPO/tarball/v1.2.3", + "zipball_url": "https://api.github.com/repos/OWNER/REPO/zipball/v1.2.3" + }`) + + reg.Register(httpmock.REST("GET", "assets/1234"), httpmock.StringResponse(string(append([]byte("\x89PNG\r\n\x1a\n"), make([]byte, 16)...)))) + }, + wantErr: "refusing to output binary content (image/png) to the terminal; use `--output` to save it to a file, or pass --allow-escape-sequences to output it anyway", + }, { name: "draft release with null tarball_url and zipball_url", isTTY: true, diff --git a/pkg/cmd/repo/read-file/read_file.go b/pkg/cmd/repo/read-file/read_file.go index faf0a95a90c..5940236cf72 100644 --- a/pkg/cmd/repo/read-file/read_file.go +++ b/pkg/cmd/repo/read-file/read_file.go @@ -1,7 +1,6 @@ package readfile import ( - "bytes" "errors" "fmt" "net/http" @@ -179,18 +178,24 @@ func readFileRun(opts *ReadFileOptions) error { return nil } - if mime, ok := binaryContentType(file.Content); ok { + // read-file does its own escape-sequence guarding below, so it writes raw + // bytes through ContentOut in passthrough mode. Leaving sanitization on + // would corrupt binary files and strip the escapes that + // --allow-escape-sequences explicitly allows. + opts.IO.SetContentSanitization(false) + + if mime, ok := iostreams.BinaryContentType(file.Content); ok { if opts.IO.IsStdoutTTY() { return fmt.Errorf("binary file (%s, %s); use --output to save to a file or pipe stdout", mime, text.FormatSize(int64(file.Size))) } - _, err = opts.IO.Out.Write(file.Content) + _, err = opts.IO.ContentOut.Write(file.Content) return err } // Refuse terminal escape sequences unless --allow-escape-sequences, in both TTY and non-TTY modes, // so a malicious file cannot manipulate a downstream terminal. - if !opts.AllowEscapeSequences && containsEscapeSequence(file.Content) { + if !opts.AllowEscapeSequences && iostreams.ContainsEscapeSequence(file.Content) { return errors.New("file contains terminal escape sequences; use --allow-escape-sequences to read anyway") } @@ -201,7 +206,7 @@ func readFileRun(opts *ReadFileOptions) error { defer opts.IO.StopPager() } - _, err = opts.IO.Out.Write(file.Content) + _, err = opts.IO.ContentOut.Write(file.Content) return err } @@ -297,27 +302,3 @@ func writeToOutput(file *repoFile, output string, clobber bool) (string, error) return dest, nil } - -// binaryContentType reports whether content appears to be binary and, if so, returns -// its detected MIME type. Textual content returns ("", false). -func binaryContentType(content []byte) (string, bool) { - if len(content) == 0 { - return "", false - } - - ct := http.DetectContentType(content) - if i := strings.IndexByte(ct, ';'); i >= 0 { - ct = strings.TrimSpace(ct[:i]) - } - - if strings.HasPrefix(ct, "text/") { - return "", false - } - return ct, true -} - -// containsEscapeSequence reports whether content contains an ANSI escape byte (0x1B), -// which could be used to manipulate the terminal when printed. -func containsEscapeSequence(content []byte) bool { - return bytes.IndexByte(content, 0x1B) >= 0 -} diff --git a/pkg/cmd/repo/read-file/read_file_test.go b/pkg/cmd/repo/read-file/read_file_test.go index 659d1b936e8..aa90d15fd58 100644 --- a/pkg/cmd/repo/read-file/read_file_test.go +++ b/pkg/cmd/repo/read-file/read_file_test.go @@ -756,42 +756,3 @@ func Test_contentsAPIPath(t *testing.T) { }) } } - -func Test_binaryContentType(t *testing.T) { - tests := []struct { - name string - content []byte - wantMIME string - wantBinary bool - }{ - { - name: "empty content is not binary", - content: []byte{}, - wantBinary: false, - }, - { - name: "plain text is not binary", - content: []byte("hello world\n"), - wantBinary: false, - }, - { - name: "png is binary", - content: append([]byte("\x89PNG\r\n\x1a\n"), make([]byte, 16)...), - wantMIME: "image/png", - wantBinary: true, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - mime, ok := binaryContentType(tt.content) - assert.Equal(t, tt.wantBinary, ok) - assert.Equal(t, tt.wantMIME, mime) - }) - } -} - -func Test_containsEscapeSequence(t *testing.T) { - assert.False(t, containsEscapeSequence([]byte("plain text"))) - assert.True(t, containsEscapeSequence([]byte("danger\x1b[31m"))) -} diff --git a/pkg/cmd/skills/preview/preview.go b/pkg/cmd/skills/preview/preview.go index 06d50154c91..500af05924e 100644 --- a/pkg/cmd/skills/preview/preview.go +++ b/pkg/cmd/skills/preview/preview.go @@ -209,7 +209,7 @@ func previewRun(opts *PreviewOptions) error { return err } - rendered := opts.renderFile("SKILL.md", content) + rendered := opts.renderFile("SKILL.md", content.String()) // Collect extra files (everything that isn't SKILL.md) var extraFiles []discovery.SkillFile @@ -304,10 +304,11 @@ func renderAllFiles(opts *PreviewOptions, cs *iostreams.ColorScheme, skill disco continue } fetched++ - totalBytes += len(fileContent) + sanitized := fileContent.String() + totalBytes += len(sanitized) fmt.Fprintf(out, "\n%s\n\n", cs.Bold("── "+f.Path+" ──")) - fmt.Fprint(out, fileContent) - if !strings.HasSuffix(fileContent, "\n") { + fmt.Fprint(out, sanitized) + if !strings.HasSuffix(sanitized, "\n") { fmt.Fprintln(out) } } @@ -358,7 +359,7 @@ func renderInteractive(opts *PreviewOptions, cs *iostreams.ColorScheme, skill di fmt.Fprintf(opts.IO.ErrOut, "%s could not fetch %s: %v\n", cs.Red("!"), selectedFile.Path, fetchErr) continue } - content = renderSelectedFilePreview(opts, selectedFile.Path, fileContent) + content = renderSelectedFilePreview(opts, selectedFile.Path, fileContent.String()) if !strings.HasSuffix(content, "\n") { content += "\n" } diff --git a/pkg/cmd/skills/search/search.go b/pkg/cmd/skills/search/search.go index 5f510aae2d2..074e338ae72 100644 --- a/pkg/cmd/skills/search/search.go +++ b/pkg/cmd/skills/search/search.go @@ -855,7 +855,7 @@ func fetchDescriptions(client *api.Client, host string, skills []skillResult) ma if err != nil { return } - result, err := frontmatter.Parse(content) + result, err := frontmatter.Parse(content.Raw()) if err != nil { return } diff --git a/pkg/iostreams/content.go b/pkg/iostreams/content.go new file mode 100644 index 00000000000..b12389ad3d3 --- /dev/null +++ b/pkg/iostreams/content.go @@ -0,0 +1,92 @@ +package iostreams + +import ( + "bytes" + "errors" + "fmt" + "io" + "net/http" + "strings" +) + +// contentSniffLen is how many leading bytes are inspected to classify content as +// binary or textual, matching the sample size used by [http.DetectContentType]. +const contentSniffLen = 512 + +// ContainsEscapeSequence reports whether b contains an ANSI escape byte (0x1B), +// which can manipulate a terminal when printed. +func ContainsEscapeSequence(b []byte) bool { + return bytes.IndexByte(b, 0x1B) >= 0 +} + +// BinaryContentType reports whether content appears to be binary and, if so, +// returns its detected MIME type. Textual content returns ("", false). +func BinaryContentType(content []byte) (string, bool) { + if len(content) == 0 { + return "", false + } + ct := http.DetectContentType(content) + if i := strings.IndexByte(ct, ';'); i >= 0 { + ct = strings.TrimSpace(ct[:i]) + } + if strings.HasPrefix(ct, "text/") { + return "", false + } + return ct, true +} + +// BinaryTerminalError reports that binary content was about to be written to a +// terminal, where it is unreadable and may carry control bytes. +type BinaryTerminalError struct { + MIME string +} + +func (e BinaryTerminalError) Error() string { + return fmt.Sprintf("refusing to output binary content (%s) to the terminal", e.MIME) +} + +// ErrEscapeSequence reports that textual content carried terminal escape +// sequences and was refused. +var ErrEscapeSequence = errors.New("content contains terminal escape sequences") + +// CopyGuardedContent writes external content from r to w under the safety model +// used by byte-moving commands: binary content is refused when w targets a +// terminal and streamed verbatim otherwise, while textual content is refused when +// it carries terminal escape sequences. Binary content streams without buffering; +// only textual content is buffered, so its escapes are caught before any byte is +// written. On refusal the output stream is left untouched; otherwise it receives +// the full content. isTTY reports whether w targets the user's terminal. +// +// It returns [BinaryTerminalError] or [ErrEscapeSequence] so callers can add +// command-specific guidance. Callers that must stream verbatim (an explicit +// opt-out, or output bound for a file) should copy directly instead. +func CopyGuardedContent(w io.Writer, r io.Reader, isTTY bool) error { + head := make([]byte, contentSniffLen) + n, err := io.ReadFull(r, head) + if err != nil && !errors.Is(err, io.EOF) && !errors.Is(err, io.ErrUnexpectedEOF) { + return err + } + head = head[:n] + + if mime, ok := BinaryContentType(head); ok { + if isTTY { + return BinaryTerminalError{MIME: mime} + } + if _, err := w.Write(head); err != nil { + return err + } + _, err := io.Copy(w, r) + return err + } + + rest, err := io.ReadAll(r) + if err != nil { + return err + } + content := append(head, rest...) + if ContainsEscapeSequence(content) { + return ErrEscapeSequence + } + _, err = w.Write(content) + return err +} diff --git a/pkg/iostreams/content_test.go b/pkg/iostreams/content_test.go new file mode 100644 index 00000000000..7343d73b3df --- /dev/null +++ b/pkg/iostreams/content_test.go @@ -0,0 +1,152 @@ +package iostreams + +import ( + "bytes" + "errors" + "io" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestBinaryContentType(t *testing.T) { + tests := []struct { + name string + content []byte + wantMIME string + wantBinary bool + }{ + { + name: "empty content is not binary", + content: []byte{}, + wantBinary: false, + }, + { + name: "plain text is not binary", + content: []byte("hello world\n"), + wantBinary: false, + }, + { + name: "png is binary", + content: append([]byte("\x89PNG\r\n\x1a\n"), make([]byte, 16)...), + wantMIME: "image/png", + wantBinary: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + mime, ok := BinaryContentType(tt.content) + assert.Equal(t, tt.wantBinary, ok) + assert.Equal(t, tt.wantMIME, mime) + }) + } +} + +func TestContainsEscapeSequence(t *testing.T) { + assert.False(t, ContainsEscapeSequence([]byte("plain text"))) + assert.True(t, ContainsEscapeSequence([]byte("danger\x1b[31m"))) +} + +func TestCopyGuardedContent(t *testing.T) { + png := append([]byte("\x89PNG\r\n\x1a\n"), make([]byte, 16)...) + readErr := errors.New("boom") + + tests := []struct { + name string + content []byte + // reader overrides content for cases a byte slice cannot express, such + // as a mid-stream read failure. + reader io.Reader + isTTY bool + wantOut []byte + // wantErrIs matches a sentinel error with errors.Is; wantErrAs matches a + // typed error (e.g. BinaryTerminalError, which carries a MIME field) with + // errors.As, so its value must be a pointer to that error type. + wantErrIs error + wantErrAs any + }{ + { + name: "clean text is written", + content: []byte("hello world\n"), + isTTY: true, + wantOut: []byte("hello world\n"), + }, + { + name: "text with escape is refused", + content: []byte("danger\x1b[31mtext"), + isTTY: true, + wantErrIs: ErrEscapeSequence, + }, + { + name: "text with escape is refused when piped", + content: []byte("danger\x1b[31mtext"), + isTTY: false, + wantErrIs: ErrEscapeSequence, + }, + { + name: "binary to terminal is refused", + content: png, + isTTY: true, + wantErrAs: &BinaryTerminalError{}, + }, + { + name: "binary when piped is written raw", + content: png, + isTTY: false, + wantOut: png, + }, + { + name: "empty content writes nothing", + content: []byte{}, + isTTY: true, + wantOut: nil, + }, + { + // Content past the sniff window is still inspected, so an escape + // hiding beyond the first chunk is caught. + name: "text with escape past the sniff window is refused", + content: append(bytes.Repeat([]byte("a"), contentSniffLen*2), []byte("\x1b[31m")...), + isTTY: false, + wantErrIs: ErrEscapeSequence, + }, + { + name: "read failure unrelated to EOF is surfaced", + reader: io.MultiReader(strings.NewReader("hi"), errReader{readErr}), + isTTY: false, + wantErrIs: readErr, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r := tt.reader + if r == nil { + r = bytes.NewReader(tt.content) + } + + var buf bytes.Buffer + err := CopyGuardedContent(&buf, r, tt.isTTY) + + if tt.wantErrAs != nil { + require.ErrorAs(t, err, tt.wantErrAs) + assert.Empty(t, buf.Bytes()) + return + } + if tt.wantErrIs != nil { + require.ErrorIs(t, err, tt.wantErrIs) + assert.Empty(t, buf.Bytes()) + return + } + + require.NoError(t, err) + assert.Equal(t, tt.wantOut, buf.Bytes()) + }) + } +} + +type errReader struct{ err error } + +func (e errReader) Read([]byte) (int, error) { return 0, e.err } diff --git a/pkg/iostreams/iostreams.go b/pkg/iostreams/iostreams.go index 8eeb03725d5..429cadda891 100644 --- a/pkg/iostreams/iostreams.go +++ b/pkg/iostreams/iostreams.go @@ -13,11 +13,13 @@ import ( "time" "github.com/briandowns/spinner" + "github.com/cli/go-gh/v2/pkg/asciisanitizer" ghTerm "github.com/cli/go-gh/v2/pkg/term" "github.com/cli/safeexec" "github.com/google/shlex" "github.com/mattn/go-colorable" "github.com/mattn/go-isatty" + "golang.org/x/text/transform" ) const DefaultWidth = 80 @@ -53,6 +55,15 @@ type IOStreams struct { Out fileWriter ErrOut fileWriter + // ContentOut is the writer for external content (HTTP response bodies, + // gist files, etc.) where the application is not the author of the bytes. + // By default it sanitizes ANSI escape sequences before they reach the + // underlying stdout. SetContentSanitization toggles the sanitization at + // the command layer (e.g. via an --allow-escape-sequences flag). + ContentOut io.Writer + + sanitizeContent bool + terminalTheme string progressIndicatorEnabled bool @@ -241,6 +252,7 @@ func (s *IOStreams) StartPager() error { fd: s.Out.Fd(), WriteCloser: &pagerWriter{pagedOut}, } + s.ContentOut = newContentWriter(s.Out, s.sanitizeContent) err = pagerCmd.Start() if err != nil { return err @@ -475,6 +487,26 @@ func (s *IOStreams) ExperimentalPrompterEnabled() bool { return s.experimentalPrompterEnabled } +// SetContentSanitization toggles ANSI escape sanitization on ContentOut. +// Commands should call this with false when an explicit opt-out flag (e.g. +// --allow-escape-sequences) is set, so subsequent writes of external content +// pass through unmodified. +func (s *IOStreams) SetContentSanitization(enabled bool) { + s.sanitizeContent = enabled + s.ContentOut = newContentWriter(s.Out, enabled) +} + +// newContentWriter returns the writer to wire up as ContentOut. When +// sanitize is true it inserts an asciisanitizer in front of the underlying +// writer; otherwise it returns the underlying writer directly so writes +// reach stdout unchanged. +func newContentWriter(out io.Writer, sanitize bool) io.Writer { + if !sanitize { + return out + } + return transform.NewWriter(out, &asciisanitizer.Sanitizer{}) +} + func System() *IOStreams { terminal := ghTerm.FromEnv() @@ -501,12 +533,14 @@ func System() *IOStreams { } io := &IOStreams{ - In: os.Stdin, - Out: stdout, - ErrOut: stderr, - pagerCommand: os.Getenv("PAGER"), - term: &terminal, + In: os.Stdin, + Out: stdout, + ErrOut: stderr, + pagerCommand: os.Getenv("PAGER"), + term: &terminal, + sanitizeContent: true, } + io.ContentOut = newContentWriter(io.Out, io.sanitizeContent) stdoutIsTTY := io.IsStdoutTTY() stderrIsTTY := io.IsStderrTTY() @@ -557,10 +591,12 @@ func Test() (*IOStreams, *bytes.Buffer, *bytes.Buffer, *bytes.Buffer) { fd: 0, ReadCloser: io.NopCloser(in), }, - Out: &fdWriter{fd: 1, Writer: out}, - ErrOut: &fdWriter{fd: 2, Writer: errOut}, - term: &fakeTerm{}, + Out: &fdWriter{fd: 1, Writer: out}, + ErrOut: &fdWriter{fd: 2, Writer: errOut}, + term: &fakeTerm{}, + sanitizeContent: true, } + io.ContentOut = newContentWriter(io.Out, io.sanitizeContent) io.SetStdinTTY(false) io.SetStdoutTTY(false) io.SetStderrTTY(false) diff --git a/pkg/iostreams/untrusted.go b/pkg/iostreams/untrusted.go new file mode 100644 index 00000000000..0b5058d5b11 --- /dev/null +++ b/pkg/iostreams/untrusted.go @@ -0,0 +1,95 @@ +package iostreams + +import ( + "encoding/json" + "strings" + + "github.com/cli/go-gh/v2/pkg/asciisanitizer" + "golang.org/x/text/transform" +) + +// Untrusted wraps string content the application did not author: HTTP response +// bodies, file contents fetched from a remote, anything that originates outside +// the CLI. The raw bytes are unexported so the only ways out are the methods +// below. +// +// Untrusted satisfies fmt.Stringer, and String sanitizes, so any fmt print path +// (Fprint, Fprintf with %s or %v, Sprint) renders the content with ANSI escape +// sequences neutralized. The only way to reach the raw bytes is Raw, which is +// deliberately easy to grep for and is intended for non-terminal uses such as +// hashing, writing to a file, or piping to another program. +type Untrusted struct { + raw string +} + +// NewUntrusted labels a string as untrusted external content. +func NewUntrusted(s string) Untrusted { + return Untrusted{raw: s} +} + +// NewUntrustedBytes labels a byte slice as untrusted external content. +func NewUntrustedBytes(b []byte) Untrusted { + return Untrusted{raw: string(b)} +} + +// String returns the content with ANSI escape sequences neutralized. It is +// called automatically by the fmt package, so printing an Untrusted value is +// safe by default on every fmt path. +func (u Untrusted) String() string { + sanitized, _, err := transform.String(&asciisanitizer.Sanitizer{}, u.raw) + if err != nil { + return stripControl(u.raw) + } + return sanitized +} + +// Raw returns the unsanitized content. It is the explicit, greppable opt-out +// for non-terminal uses (hashing, writing to disk, piping). Never pass the +// result to a terminal writer. +func (u Untrusted) Raw() string { + return u.raw +} + +// Empty reports whether the content is empty, for callers that branch on +// presence without needing the bytes. +func (u Untrusted) Empty() bool { + return u.raw == "" +} + +// UnmarshalJSON lets a struct field typed as Untrusted be populated directly by +// json.Unmarshal, so provenance is preserved across a JSON decode. This is what +// lets a decoded field (e.g. a streamed log line) stay labeled when +// the surrounding response was never sanitized by the JSON transport. +func (u *Untrusted) UnmarshalJSON(data []byte) error { + var s string + if err := json.Unmarshal(data, &s); err != nil { + return err + } + u.raw = s + return nil +} + +// MarshalJSON emits the raw content as a JSON string so a value round-trips +// faithfully through encode/decode. +func (u Untrusted) MarshalJSON() ([]byte, error) { + return json.Marshal(u.raw) +} + +// RawBytes is Raw as a byte slice, for callers that need []byte (hashing, file +// writes). Same terminal caveat as Raw. +func (u Untrusted) RawBytes() []byte { + return []byte(u.raw) +} + +// stripControl is a defensive fallback used only if the sanitizing transform +// errors, which the asciisanitizer does not do in practice. It drops C0 control +// bytes other than tab, newline, and carriage return so the result can never +// carry an escape sequence. +func stripControl(s string) string { + return strings.Map(func(r rune) rune { + if r < 0x20 && r != '\t' && r != '\n' && r != '\r' { + return -1 + } + return r + }, s) +} diff --git a/pkg/iostreams/untrusted_test.go b/pkg/iostreams/untrusted_test.go new file mode 100644 index 00000000000..c904da4cf04 --- /dev/null +++ b/pkg/iostreams/untrusted_test.go @@ -0,0 +1,79 @@ +package iostreams + +import ( + "encoding/json" + "fmt" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +const esc = "\x1b" + +func TestUntrusted_String_sanitizes(t *testing.T) { + u := NewUntrusted("hello" + esc + "[31mRED" + esc + "[0m") + assert.NotContains(t, u.String(), esc) +} + +// The property that drove the design: fmt reflection must not leak the raw +// bytes through any verb. Because Untrusted implements Stringer, %s, %v, and the +// Print family all route through String() and sanitize. +func TestUntrusted_fmt_paths_never_leak(t *testing.T) { + u := NewUntrusted("x" + esc + "]0;title" + esc + "\\") + cases := map[string]string{ + "%s": fmt.Sprintf("%s", u), + "%v": fmt.Sprintf("%v", u), + "Sprint": fmt.Sprint(u), + "woven": fmt.Sprintf("by %s here", u), + } + for name, out := range cases { + t.Run(name, func(t *testing.T) { + assert.NotContains(t, out, esc) + }) + } +} + +func TestUntrusted_Raw_returnsExactBytes(t *testing.T) { + payload := "x" + esc + "[1mbold" + u := NewUntrusted(payload) + assert.Equal(t, payload, u.Raw()) + assert.Equal(t, payload, string(u.RawBytes())) +} + +func TestUntrustedBytes_roundTrip(t *testing.T) { + u := NewUntrustedBytes([]byte("plain text")) + assert.Equal(t, "plain text", u.String()) +} + +func TestStripControl_dropsC0KeepsWhitespace(t *testing.T) { + assert.Equal(t, "abc\td\ne", stripControl("a\x1bb\x07c\td\ne")) +} + +// The showcase property: an Untrusted struct field is populated by +// json.Unmarshal with provenance intact, so printing it later sanitizes even +// though the bytes arrived through a JSON decode. +func TestUntrusted_survivesJSONDecode(t *testing.T) { + var entry struct { + Content Untrusted `json:"content"` + } + payload := `{"content":"log\u001b[31mline"}` + require.NoError(t, json.Unmarshal([]byte(payload), &entry)) + assert.Equal(t, "log\x1b[31mline", entry.Content.Raw()) + assert.NotContains(t, entry.Content.String(), esc) +} + +func TestUntrusted_jsonRoundTrip(t *testing.T) { + u := NewUntrusted("x\x1b[0m") + b, err := json.Marshal(u) + require.NoError(t, err) + + var back Untrusted + require.NoError(t, json.Unmarshal(b, &back)) + assert.Equal(t, u.Raw(), back.Raw()) +} + +func TestUntrusted_Empty(t *testing.T) { + assert.True(t, NewUntrusted("").Empty()) + assert.False(t, NewUntrusted("x").Empty()) +} From 0c2eea6338a2323cfff000160b9b5a56a38d2a06 Mon Sep 17 00:00:00 2001 From: "Babak K. Shandiz" Date: Fri, 31 Jul 2026 02:22:49 +0100 Subject: [PATCH 2/4] Merge commit from fork * feat(safeurl): add SafeURL package and CodeQL enforcement query Introduce internal/safeurl, which builds HTTP request URLs from variable components that are percent-encoded when rendered, so user or server controlled values cannot break the path or change which resource is addressed. It provides the SafeURL interface, the MutableSafeURL and ImmutableSafeURL implementations, the JoinPath and JoinPathWithHostPrefix builders, and NewImmutableSafeURL for entrusting already-formed URLs. Because percent-encoding leaves a component that is exactly ".." intact as a real path segment, JoinPath and JoinPathWithHostPrefix reject any such component and return an error, which callers bubble up to the command level. Also add a CodeQL query that flags any HTTP request URL argument that is not literally the result of a safeurl.SafeURL.String call. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(api): route REST paths through SafeURL Build the hand-written REST paths in the shared API layer with safeurl so their variable components are escaped. CreateRepoTransformToV4 now takes a safeurl.SafeURL path instead of a string. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(repo): build REST paths with SafeURL Escape the variable components of the repo command REST paths. The new read-file command builds its Contents API URL with safeurl, and the repo create and edit commands pass a safeurl.SafeURL to CreateRepoTransformToV4. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(run): build REST paths with SafeURL Escape the variable components of the run command REST paths. GetJobs no longer mutates the Run; it takes an entrusted jobs URL and a run id and returns the jobs, so its callers assign them explicitly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(release): build REST paths with SafeURL Escape the variable components of the release command REST paths. The upload and delete-asset pipeline threads a safeurl.SafeURL through, so the asset upload URL and AssetForUpload.ExistingURL are carried as SafeURL. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(codespace): build REST paths with SafeURL Escape the variable components of the codespace command REST paths. The NWO validation helper moves out of safeurl into the codespace package, where it is the only remaining caller. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(attestation): build bundle URLs with SafeURL getBundle takes a safeurl.SafeURL, and the server-returned bundle URL is entrusted with NewImmutableSafeURL at the API boundary. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(gist): build REST paths with SafeURL Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(workflow): build REST paths with SafeURL Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor: build REST paths with SafeURL in secret, variable, ssh-key, and gpg-key commands Escape the variable name components of these commands' REST paths and surface any path-traversal error from the builders. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(skills): build REST paths with SafeURL Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor: build remaining REST paths with SafeURL Escape the variable components of the remaining hand-written REST paths across the pr, ruleset, label, copilot, agent-task, auth, cache, extension, status, and api commands, plus the search, feature detection, and update internals, surfacing any path-traversal error from the builders. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(safeurl): cover dot-segment escaping and preservation Add regression tests confirming a pre-encoded "%2e%2e" component is double-encoded rather than treated as traversal, and that single "." components are preserved verbatim instead of being collapsed. These guard against re-introducing url.JoinPath/path.Clean and against bypassing the traversal check via pre-encoding. Covers both JoinPath and JoinPathWithHostPrefix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2f72878e-d567-4a58-937d-f03c94c0287e * feat(codeql): flag hand-assembled strings reaching NewImmutableSafeURL Add a query that reports when the argument to safeurl.NewImmutableSafeURL is tainted by fmt.Sprintf, fmt.Sprint, fmt.Sprintln or a string concatenation. NewImmutableSafeURL renders its argument verbatim, bypassing the escaping and traversal check that JoinPath applies, so it must only receive an already formed, trusted URL. This is a convention guard and cannot verify the trustedness of URLs read from struct fields or returned by API calls. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2f72878e-d567-4a58-937d-f03c94c0287e * refactor: rename pagination cursor to pageURL to avoid shadowing net/url The pagination cursor was named url, which shadows the net/url import in run/shared/shared.go where GetJobs and GetRun still use it. Rename it to pageURL, which also reads more accurately, and apply the same name to the other pagination loops so they stay consistent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2f72878e-d567-4a58-937d-f03c94c0287e * refactor(safeurl): reject owner/repo values with extra slashes RepoPartsFromNWO used strings.Cut, which splits on the first slash and left any remaining slashes in the name part, so "owner/repo/extra" silently parsed as name "repo/extra". Switch to strings.Split and require exactly one slash with a non-empty owner and name, so a value carrying extra slashes cannot smuggle additional path segments through. Also document why this does not reuse ghrepo.FromFullName, which accepts the broader "[HOST/]OWNER/REPO" form. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2f72878e-d567-4a58-937d-f03c94c0287e --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Kynan Ware <47394200+BagToad@users.noreply.github.com> Copilot-Session: 2f72878e-d567-4a58-937d-f03c94c0287e --- .../queries/ImmutableSafeURLConstruction.ql | 58 ++++ .../codeql/queries/SafeURLPathConstruction.ql | 73 ++++ api/queries_pr.go | 9 +- api/queries_pr_review.go | 37 +- api/queries_pr_test.go | 4 +- api/queries_repo.go | 54 ++- internal/codespaces/api/api.go | 174 +++++++--- .../featuredetection/feature_detection.go | 7 +- internal/safeurl/safeurl.go | 164 +++++++++ internal/safeurl/safeurl_test.go | 317 ++++++++++++++++++ internal/skills/discovery/discovery.go | 104 ++++-- internal/skills/discovery/discovery_test.go | 44 +-- internal/update/update.go | 11 +- pkg/cmd/agent-task/capi/job.go | 17 +- pkg/cmd/agent-task/capi/sessions.go | 29 +- pkg/cmd/api/http.go | 2 + pkg/cmd/attestation/api/attestation.go | 5 - pkg/cmd/attestation/api/client.go | 58 ++-- pkg/cmd/attestation/api/client_test.go | 9 +- pkg/cmd/auth/shared/login_flow.go | 8 +- pkg/cmd/auth/shared/oauth_scopes.go | 8 +- pkg/cmd/cache/delete/delete.go | 19 +- pkg/cmd/cache/shared/shared.go | 26 +- pkg/cmd/codespace/common.go | 7 + pkg/cmd/codespace/create.go | 5 + pkg/cmd/codespace/create_test.go | 5 + pkg/cmd/codespace/list.go | 6 + pkg/cmd/codespace/list_test.go | 5 + pkg/cmd/copilot/copilot.go | 22 +- pkg/cmd/copilot/copilot_test.go | 9 +- pkg/cmd/extension/http.go | 47 +-- pkg/cmd/extension/manager.go | 3 +- pkg/cmd/gist/create/create.go | 8 +- pkg/cmd/gist/delete/delete.go | 8 +- pkg/cmd/gist/edit/edit.go | 10 +- pkg/cmd/gist/rename/rename.go | 8 +- pkg/cmd/gist/shared/shared.go | 12 +- pkg/cmd/gist/shared/shared_test.go | 3 +- pkg/cmd/gist/view/view.go | 3 +- pkg/cmd/gpg-key/add/http.go | 8 +- pkg/cmd/gpg-key/delete/http.go | 18 +- pkg/cmd/gpg-key/list/http.go | 16 +- pkg/cmd/label/create.go | 15 +- pkg/cmd/label/delete.go | 8 +- pkg/cmd/pr/close/close_test.go | 6 +- pkg/cmd/pr/diff/diff.go | 14 +- pkg/cmd/pr/merge/merge_test.go | 14 +- pkg/cmd/release/create/create.go | 7 +- pkg/cmd/release/create/http.go | 52 +-- pkg/cmd/release/delete-asset/delete_asset.go | 7 +- pkg/cmd/release/delete/delete.go | 16 +- pkg/cmd/release/delete/delete_test.go | 2 +- pkg/cmd/release/download/download.go | 28 +- pkg/cmd/release/edit/http.go | 10 +- pkg/cmd/release/shared/fetch.go | 37 +- pkg/cmd/release/shared/fetch_test.go | 4 +- pkg/cmd/release/shared/upload.go | 22 +- pkg/cmd/release/shared/upload_test.go | 4 +- pkg/cmd/release/upload/upload.go | 15 +- pkg/cmd/repo/autolink/create/http.go | 10 +- pkg/cmd/repo/autolink/delete/http.go | 11 +- pkg/cmd/repo/autolink/list/http.go | 11 +- pkg/cmd/repo/autolink/view/http.go | 11 +- pkg/cmd/repo/create/http.go | 23 +- pkg/cmd/repo/credits/credits.go | 8 +- pkg/cmd/repo/delete/http.go | 11 +- pkg/cmd/repo/deploy-key/add/http.go | 10 +- pkg/cmd/repo/deploy-key/delete/http.go | 10 +- pkg/cmd/repo/deploy-key/list/http.go | 11 +- pkg/cmd/repo/edit/edit.go | 22 +- pkg/cmd/repo/garden/http.go | 24 +- pkg/cmd/repo/read-file/http.go | 31 +- pkg/cmd/repo/read-file/read_file_test.go | 5 +- pkg/cmd/repo/sync/http.go | 22 +- pkg/cmd/repo/sync/sync_test.go | 16 +- pkg/cmd/repo/view/http.go | 19 +- pkg/cmd/ruleset/check/check.go | 10 +- pkg/cmd/ruleset/view/http.go | 16 +- pkg/cmd/run/cancel/cancel.go | 13 +- pkg/cmd/run/delete/delete.go | 8 +- pkg/cmd/run/download/download.go | 5 +- pkg/cmd/run/download/download_test.go | 5 +- pkg/cmd/run/download/http.go | 7 +- pkg/cmd/run/download/http_test.go | 3 +- pkg/cmd/run/rerun/rerun.go | 16 +- pkg/cmd/run/shared/artifacts.go | 26 +- pkg/cmd/run/shared/shared.go | 106 +++--- pkg/cmd/run/view/logs.go | 10 +- pkg/cmd/run/view/view.go | 23 +- pkg/cmd/run/watch/watch.go | 4 +- pkg/cmd/secret/delete/delete.go | 16 +- pkg/cmd/secret/list/list.go | 79 +++-- pkg/cmd/secret/list/list_test.go | 5 +- pkg/cmd/secret/set/http.go | 55 ++- pkg/cmd/skills/install/install.go | 11 +- pkg/cmd/skills/install/install_test.go | 12 +- pkg/cmd/skills/preview/preview_test.go | 34 +- pkg/cmd/skills/publish/publish.go | 94 ++++-- pkg/cmd/skills/search/search.go | 21 +- pkg/cmd/skills/update/update_test.go | 24 +- pkg/cmd/ssh-key/add/http.go | 19 +- pkg/cmd/ssh-key/delete/http.go | 16 +- pkg/cmd/ssh-key/shared/user_keys.go | 34 +- pkg/cmd/status/status.go | 41 ++- pkg/cmd/variable/delete/delete.go | 14 +- pkg/cmd/variable/get/get.go | 22 +- pkg/cmd/variable/list/list.go | 41 ++- pkg/cmd/variable/list/list_test.go | 5 +- pkg/cmd/variable/set/http.go | 40 ++- pkg/cmd/variable/shared/shared.go | 27 +- pkg/cmd/workflow/disable/disable.go | 9 +- pkg/cmd/workflow/enable/enable.go | 9 +- pkg/cmd/workflow/run/run.go | 10 +- pkg/cmd/workflow/run/run_test.go | 14 +- pkg/cmd/workflow/shared/shared.go | 28 +- pkg/cmd/workflow/view/view_test.go | 8 +- pkg/search/searcher.go | 26 +- 117 files changed, 2147 insertions(+), 775 deletions(-) create mode 100644 .github/codeql/queries/ImmutableSafeURLConstruction.ql create mode 100644 .github/codeql/queries/SafeURLPathConstruction.ql create mode 100644 internal/safeurl/safeurl.go create mode 100644 internal/safeurl/safeurl_test.go diff --git a/.github/codeql/queries/ImmutableSafeURLConstruction.ql b/.github/codeql/queries/ImmutableSafeURLConstruction.ql new file mode 100644 index 00000000000..39daa77f20e --- /dev/null +++ b/.github/codeql/queries/ImmutableSafeURLConstruction.ql @@ -0,0 +1,58 @@ +/** + * @name ImmutableSafeURL built from a hand-assembled string + * @description Flags a call to safeurl.NewImmutableSafeURL whose argument is a locally assembled + * string, that is a value tainted by fmt.Sprintf, fmt.Sprint, fmt.Sprintln or a string + * concatenation. NewImmutableSafeURL renders its argument verbatim, skipping the + * percent-encoding and traversal check that JoinPath applies, so it must only receive an + * already formed, trusted URL such as a server returned field or a pagination link. A + * hand-built path reaching it is a way to route around safeurl and must instead be built + * with safeurl.JoinPath. This query is a convention guard, it cannot and does not verify + * the trustedness of URLs read from struct fields or returned by API calls. + * @kind problem + * @problem.severity warning + * @precision high + * @id cli-cli/immutable-safeurl-construction + * @tags security + * correctness + * maintainability + */ + +import go + +/** + * Holds when `node` is the URL argument of a call to safeurl.NewImmutableSafeURL, the escape hatch + * that renders its argument verbatim without percent-encoding or a traversal check. + */ +predicate isImmutableSafeURLArgument(DataFlow::Node node) { + exists(Function f, DataFlow::CallNode call | + f.hasQualifiedName("github.com/cli/cli/v2/internal/safeurl", "NewImmutableSafeURL") and + call = f.getACall() and + node = call.getArgument(0) + ) +} + +/** + * Holds when `node` is a locally assembled string: the result of fmt.Sprintf, fmt.Sprint or + * fmt.Sprintln, or a string concatenation expression. These are the shapes that build a URL by hand + * rather than reading an already formed value, so they must not reach NewImmutableSafeURL. + */ +predicate isHandAssembledString(DataFlow::Node node) { + exists(Function f | + f.hasQualifiedName("fmt", ["Sprintf", "Sprint", "Sprintln"]) and + node = f.getACall() + ) + or + exists(AddExpr e | + e.getType() instanceof StringType and + node = DataFlow::exprNode(e) + ) +} + +from DataFlow::Node source, DataFlow::Node sink +where + isImmutableSafeURLArgument(sink) and + isHandAssembledString(source) and + TaintTracking::localTaint(source, sink) +select sink, + "This ImmutableSafeURL is built from a hand-assembled string ($@); build the path with safeurl.JoinPath so its components are escaped and traversal-checked.", + source, "assembled here" diff --git a/.github/codeql/queries/SafeURLPathConstruction.ql b/.github/codeql/queries/SafeURLPathConstruction.ql new file mode 100644 index 00000000000..3697b9162a9 --- /dev/null +++ b/.github/codeql/queries/SafeURLPathConstruction.ql @@ -0,0 +1,73 @@ +/** + * @name HTTP request URL not built with safeurl.SafeURL + * @description Flags any HTTP request, a REST API call being the common case, whose URL argument is + * not literally a call to (safeurl.SafeURL).String. The argument expression itself must + * be a SafeURL.String call; any other form, such as a string literal, string + * concatenation, or fmt.Sprintf, is reported. This keeps every hand built URL routed + * through safeurl so its variable path components are percent-encoded. + * @kind problem + * @problem.severity warning + * @precision high + * @id cli-cli/safeurl-path-construction + * @tags security + * correctness + * maintainability + */ + +import go + +/** + * Holds when `node` is the URL argument of an HTTP request, a REST API call being the common case. + * + * Covered entry points: + * - (github.com/cli/cli/v2/api.Client).REST and .RESTWithNext, where the path is argument 2. + * - net/http.NewRequest, where the URL is argument 1. + * - net/http.NewRequestWithContext, where the URL is argument 2. + * - (net/http.Client).Get, .Head, .Post and .PostForm, where the URL is argument 0. + */ +predicate isHttpUrlArgument(DataFlow::Node node) { + exists(Method m, DataFlow::CallNode call | + m.hasQualifiedName("github.com/cli/cli/v2/api", "Client", ["REST", "RESTWithNext"]) and + call = m.getACall() and + node = call.getArgument(2) + ) + or + exists(Function f, DataFlow::CallNode call | + f.hasQualifiedName("net/http", "NewRequest") and + call = f.getACall() and + node = call.getArgument(1) + ) + or + exists(Function f, DataFlow::CallNode call | + f.hasQualifiedName("net/http", "NewRequestWithContext") and + call = f.getACall() and + node = call.getArgument(2) + ) + or + exists(Method m, DataFlow::CallNode call | + m.hasQualifiedName("net/http", "Client", ["Get", "Head", "Post", "PostForm"]) and + call = m.getACall() and + node = call.getArgument(0) + ) +} + +/** + * Holds when `node` is a call to the String method of one of the safeurl URL types: + * the SafeURL interface or either of its implementations, MutableSafeURL and + * ImmutableSafeURL. Matching all three keeps call sites free of explicit conversions: + * a value of the concrete type can be passed to the sink directly without first being + * assigned to a SafeURL typed variable. + */ +predicate isSafeurlStringCall(DataFlow::Node node) { + exists(Method m | + m.hasQualifiedName("github.com/cli/cli/v2/internal/safeurl", + ["SafeURL", "MutableSafeURL", "ImmutableSafeURL"], "String") and + node = m.getACall() + ) +} + +from DataFlow::Node sink +where + isHttpUrlArgument(sink) and + not isSafeurlStringCall(sink) +select sink, "This HTTP request URL is not passed directly as the result of safeurl.SafeURL.String." diff --git a/api/queries_pr.go b/api/queries_pr.go index 29342521f8c..10994958ace 100644 --- a/api/queries_pr.go +++ b/api/queries_pr.go @@ -3,10 +3,10 @@ package api import ( "fmt" "net/http" - "net/url" "time" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/shurcooL/githubv4" ) @@ -845,8 +845,11 @@ func ConvertPullRequestToDraft(client *Client, repo ghrepo.Interface, pr *PullRe } func BranchDeleteRemote(client *Client, repo ghrepo.Interface, branch string) error { - path := fmt.Sprintf("repos/%s/%s/git/refs/heads/%s", repo.RepoOwner(), repo.RepoName(), url.PathEscape(branch)) - return client.REST(repo.RepoHost(), "DELETE", path, nil, nil) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "git", "refs", fmt.Sprintf("heads/%s", branch)) + if err != nil { + return err + } + return client.REST(repo.RepoHost(), "DELETE", path.String(), nil, nil) } type RefComparison struct { diff --git a/api/queries_pr_review.go b/api/queries_pr_review.go index b0a602bf4c9..1526758cd9a 100644 --- a/api/queries_pr_review.go +++ b/api/queries_pr_review.go @@ -4,11 +4,12 @@ import ( "bytes" "encoding/json" "fmt" - "net/url" + "strconv" "strings" "time" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/shurcooL/githubv4" ) @@ -284,12 +285,17 @@ func AddPullRequestReviews(client *Client, repo ghrepo.Interface, prNumber int, users = []string{} } - path := fmt.Sprintf( - "repos/%s/%s/pulls/%d/requested_reviewers", - url.PathEscape(repo.RepoOwner()), - url.PathEscape(repo.RepoName()), - prNumber, + path, err := safeurl.JoinPath( + "repos", + repo.RepoOwner(), + repo.RepoName(), + "pulls", + strconv.Itoa(prNumber), + "requested_reviewers", ) + if err != nil { + return err + } body := struct { Reviewers []string `json:"reviewers"` TeamReviewers []string `json:"team_reviewers"` @@ -302,7 +308,7 @@ func AddPullRequestReviews(client *Client, repo ghrepo.Interface, prNumber int, return err } // The endpoint responds with the updated pull request object; we don't need it here. - return client.REST(repo.RepoHost(), "POST", path, buf, nil) + return client.REST(repo.RepoHost(), "POST", path.String(), buf, nil) } // RemovePullRequestReviews removes requested reviewers from a pull request using the REST API. @@ -317,12 +323,17 @@ func RemovePullRequestReviews(client *Client, repo ghrepo.Interface, prNumber in users = []string{} } - path := fmt.Sprintf( - "repos/%s/%s/pulls/%d/requested_reviewers", - url.PathEscape(repo.RepoOwner()), - url.PathEscape(repo.RepoName()), - prNumber, + path, err := safeurl.JoinPath( + "repos", + repo.RepoOwner(), + repo.RepoName(), + "pulls", + strconv.Itoa(prNumber), + "requested_reviewers", ) + if err != nil { + return err + } body := struct { Reviewers []string `json:"reviewers"` TeamReviewers []string `json:"team_reviewers"` @@ -335,7 +346,7 @@ func RemovePullRequestReviews(client *Client, repo ghrepo.Interface, prNumber in return err } // The endpoint responds with the updated pull request object; we don't need it here. - return client.REST(repo.RepoHost(), "DELETE", path, buf, nil) + return client.REST(repo.RepoHost(), "DELETE", path.String(), buf, nil) } // RequestReviewsByLogin sets requested reviewers on a pull request using the GraphQL mutation. diff --git a/api/queries_pr_test.go b/api/queries_pr_test.go index 633b9a8c35f..cf7e7b04b81 100644 --- a/api/queries_pr_test.go +++ b/api/queries_pr_test.go @@ -23,7 +23,7 @@ func TestBranchDeleteRemote(t *testing.T) { branch: "owner/branch#123", httpStubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads/owner%2Fbranch%23123"), + httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads%2Fowner%2Fbranch%23123"), httpmock.StatusStringResponse(204, "")) }, expectError: false, @@ -33,7 +33,7 @@ func TestBranchDeleteRemote(t *testing.T) { branch: "my-branch", httpStubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads/my-branch"), + httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads%2Fmy-branch"), httpmock.StatusStringResponse(500, `{"message": "oh no"}`)) }, expectError: true, diff --git a/api/queries_repo.go b/api/queries_repo.go index d1bc2df1e29..3e0b75648cb 100644 --- a/api/queries_repo.go +++ b/api/queries_repo.go @@ -17,6 +17,7 @@ import ( "golang.org/x/sync/errgroup" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" ghAPI "github.com/cli/go-gh/v2/pkg/api" "github.com/shurcooL/githubv4" ) @@ -589,7 +590,10 @@ type repositoryV3 struct { // ForkRepo forks the repository on GitHub and returns the new repository func ForkRepo(client *Client, repo ghrepo.Interface, org, newName string, defaultBranchOnly bool) (*Repository, error) { - path := fmt.Sprintf("repos/%s/forks", ghrepo.FullName(repo)) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "forks") + if err != nil { + return nil, err + } params := map[string]interface{}{} if org != "" { @@ -609,7 +613,7 @@ func ForkRepo(client *Client, repo ghrepo.Interface, org, newName string, defaul } result := repositoryV3{} - err := client.REST(repo.RepoHost(), "POST", path, body, &result) + err = client.REST(repo.RepoHost(), "POST", path.String(), body, &result) if err != nil { return nil, err } @@ -643,12 +647,13 @@ func RenameRepo(client *Client, repo ghrepo.Interface, newRepoName string) (*Rep return nil, err } - path := fmt.Sprintf("%srepos/%s", - ghinstance.RESTPrefix(repo.RepoHost()), - ghrepo.FullName(repo)) + path, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName()) + if err != nil { + return nil, err + } result := repositoryV3{} - err := client.REST(repo.RepoHost(), "PATCH", path, body, &result) + err = client.REST(repo.RepoHost(), "PATCH", path.String(), body, &result) if err != nil { return nil, err } @@ -1608,9 +1613,9 @@ func v2Projects(client *Client, repo ghrepo.Interface) ([]ProjectV2, error) { return projectsV2, nil } -func CreateRepoTransformToV4(apiClient *Client, hostname string, method string, path string, body io.Reader) (*Repository, error) { +func CreateRepoTransformToV4(apiClient *Client, hostname string, method string, path safeurl.SafeURL, body io.Reader) (*Repository, error) { var responsev3 repositoryV3 - err := apiClient.REST(hostname, method, path, body, &responsev3) + err := apiClient.REST(hostname, method, path.String(), body, &responsev3) if err != nil { return nil, err @@ -1666,9 +1671,12 @@ func GetRepoIDs(client *Client, host string, repositories []ghrepo.Interface) ([ } func RepoExists(client *Client, repo ghrepo.Interface) (bool, error) { - path := fmt.Sprintf("%srepos/%s/%s", ghinstance.RESTPrefix(repo.RepoHost()), repo.RepoOwner(), repo.RepoName()) + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName()) + if err != nil { + return false, err + } - resp, err := client.HTTP().Head(path) + resp, err := client.HTTP().Head(u.String()) if err != nil { return false, err } @@ -1690,7 +1698,11 @@ func RepoExists(client *Client, repo ghrepo.Interface) (bool, error) { func RepoLicenses(httpClient *http.Client, hostname string) ([]License, error) { var licenses []License client := NewClientFromHTTP(httpClient) - err := client.REST(hostname, "GET", "licenses", nil, &licenses) + path, err := safeurl.JoinPath("licenses") + if err != nil { + return nil, err + } + err = client.REST(hostname, "GET", path.String(), nil, &licenses) if err != nil { return nil, err } @@ -1702,8 +1714,11 @@ func RepoLicenses(httpClient *http.Client, hostname string) ([]License, error) { func RepoLicense(httpClient *http.Client, hostname string, licenseName string) (*License, error) { var license License client := NewClientFromHTTP(httpClient) - path := fmt.Sprintf("licenses/%s", licenseName) - err := client.REST(hostname, "GET", path, nil, &license) + path, err := safeurl.JoinPath("licenses", licenseName) + if err != nil { + return nil, err + } + err = client.REST(hostname, "GET", path.String(), nil, &license) if err != nil { return nil, err } @@ -1715,7 +1730,11 @@ func RepoLicense(httpClient *http.Client, hostname string, licenseName string) ( func RepoGitIgnoreTemplates(httpClient *http.Client, hostname string) ([]string, error) { var gitIgnoreTemplates []string client := NewClientFromHTTP(httpClient) - err := client.REST(hostname, "GET", "gitignore/templates", nil, &gitIgnoreTemplates) + path, err := safeurl.JoinPath("gitignore", "templates") + if err != nil { + return nil, err + } + err = client.REST(hostname, "GET", path.String(), nil, &gitIgnoreTemplates) if err != nil { return nil, err } @@ -1727,8 +1746,11 @@ func RepoGitIgnoreTemplates(httpClient *http.Client, hostname string) ([]string, func RepoGitIgnoreTemplate(httpClient *http.Client, hostname string, gitIgnoreTemplateName string) (*GitIgnore, error) { var gitIgnoreTemplate GitIgnore client := NewClientFromHTTP(httpClient) - path := fmt.Sprintf("gitignore/templates/%s", gitIgnoreTemplateName) - err := client.REST(hostname, "GET", path, nil, &gitIgnoreTemplate) + path, err := safeurl.JoinPath("gitignore", "templates", gitIgnoreTemplateName) + if err != nil { + return nil, err + } + err = client.REST(hostname, "GET", path.String(), nil, &gitIgnoreTemplate) if err != nil { return nil, err } diff --git a/internal/codespaces/api/api.go b/internal/codespaces/api/api.go index df2e180d7aa..29a852cb68f 100644 --- a/internal/codespaces/api/api.go +++ b/internal/codespaces/api/api.go @@ -42,6 +42,7 @@ import ( "github.com/cenkalti/backoff/v4" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/opentracing/opentracing-go" ) @@ -115,7 +116,11 @@ func (a *API) ServerURL() string { // GetUser returns the user associated with the given token. func (a *API) GetUser(ctx context.Context) (*User, error) { - req, err := http.NewRequest(http.MethodGet, a.githubAPI+"/user", nil) + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "user") + if err != nil { + return nil, err + } + req, err := http.NewRequest(http.MethodGet, u.String(), nil) if err != nil { return nil, fmt.Errorf("error creating request: %w", err) } @@ -160,7 +165,15 @@ type Repository struct { // GetRepository returns the repository associated with the given owner and name. func (a *API) GetRepository(ctx context.Context, nwo string) (*Repository, error) { - req, err := http.NewRequest(http.MethodGet, a.githubAPI+"/repos/"+strings.ToLower(nwo), nil) + owner, name, err := safeurl.RepoPartsFromNWO(strings.ToLower(nwo)) + if err != nil { + return nil, err + } + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "repos", owner, name) + if err != nil { + return nil, err + } + req, err := http.NewRequest(http.MethodGet, u.String(), nil) if err != nil { return nil, fmt.Errorf("error creating request: %w", err) } @@ -364,31 +377,55 @@ func (a *API) ListCodespaces(ctx context.Context, opts ListCodespacesOptions) (c } var ( - listURL string + listURL safeurl.SafeURL spanName string ) if opts.RepoName != "" { - listURL = fmt.Sprintf("%s/repos/%s/codespaces?per_page=%d", a.githubAPI, opts.RepoName, perPage) + owner, name, err := safeurl.RepoPartsFromNWO(opts.RepoName) + if err != nil { + return nil, err + } + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "repos", owner, name, "codespaces") + if err != nil { + return nil, err + } + u.SetQuery("per_page", strconv.Itoa(perPage)) + listURL = u spanName = "/repos/*/codespaces" } else if opts.OrgName != "" { // the endpoints below can only be called by the organization admins orgName := opts.OrgName if opts.UserName != "" { userName := opts.UserName - listURL = fmt.Sprintf("%s/orgs/%s/members/%s/codespaces?per_page=%d", a.githubAPI, orgName, userName, perPage) + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "orgs", orgName, "members", userName, "codespaces") + if err != nil { + return nil, err + } + u.SetQuery("per_page", strconv.Itoa(perPage)) + listURL = u spanName = "/orgs/*/members/*/codespaces" } else { - listURL = fmt.Sprintf("%s/orgs/%s/codespaces?per_page=%d", a.githubAPI, orgName, perPage) + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "orgs", orgName, "codespaces") + if err != nil { + return nil, err + } + u.SetQuery("per_page", strconv.Itoa(perPage)) + listURL = u spanName = "/orgs/*/codespaces" } } else { - listURL = fmt.Sprintf("%s/user/codespaces?per_page=%d", a.githubAPI, perPage) + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "user", "codespaces") + if err != nil { + return nil, err + } + u.SetQuery("per_page", strconv.Itoa(perPage)) + listURL = u spanName = "/user/codespaces" } for { - req, err := http.NewRequest(http.MethodGet, listURL, nil) + req, err := http.NewRequest(http.MethodGet, listURL.String(), nil) if err != nil { return nil, fmt.Errorf("error creating request: %w", err) } @@ -425,9 +462,9 @@ func (a *API) ListCodespaces(ctx context.Context, opts ListCodespacesOptions) (c q := u.Query() q.Set("per_page", strconv.Itoa(newPerPage)) u.RawQuery = q.Encode() - listURL = u.String() + listURL = safeurl.NewImmutableSafeURL(u.String()) } else { - listURL = nextURL + listURL = safeurl.NewImmutableSafeURL(nextURL) } } @@ -447,10 +484,15 @@ func findNextPage(linkValue string) string { func (a *API) GetOrgMemberCodespace(ctx context.Context, orgName string, userName string, codespaceName string) (*Codespace, error) { perPage := 100 - listURL := fmt.Sprintf("%s/orgs/%s/members/%s/codespaces?per_page=%d", a.githubAPI, orgName, userName, perPage) + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "orgs", orgName, "members", userName, "codespaces") + if err != nil { + return nil, err + } + u.SetQuery("per_page", strconv.Itoa(perPage)) + var listURL safeurl.SafeURL = u for { - req, err := http.NewRequest(http.MethodGet, listURL, nil) + req, err := http.NewRequest(http.MethodGet, listURL.String(), nil) if err != nil { return nil, fmt.Errorf("error creating request: %w", err) } @@ -485,7 +527,7 @@ func (a *API) GetOrgMemberCodespace(ctx context.Context, orgName string, userNam if nextURL == "" { break } - listURL = nextURL + listURL = safeurl.NewImmutableSafeURL(nextURL) } return nil, fmt.Errorf("codespace not found for user %s with name %s", userName, codespaceName) @@ -496,9 +538,13 @@ func (a *API) GetOrgMemberCodespace(ctx context.Context, orgName string, userNam // If includeConnection is true, it will return the connection information for the codespace. func (a *API) GetCodespace(ctx context.Context, codespaceName string, includeConnection bool) (*Codespace, error) { resp, err := a.withRetry(func() (*http.Response, error) { + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "user", "codespaces", codespaceName) + if err != nil { + return nil, err + } req, err := http.NewRequest( http.MethodGet, - a.githubAPI+"/user/codespaces/"+codespaceName, + u.String(), nil, ) if err != nil { @@ -539,9 +585,13 @@ func (a *API) GetCodespace(ctx context.Context, codespaceName string, includeCon // If the codespace is already running, the returned error from the API is ignored. func (a *API) StartCodespace(ctx context.Context, codespaceName string) error { resp, err := a.withRetry(func() (*http.Response, error) { + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "user", "codespaces", codespaceName, "start") + if err != nil { + return nil, err + } req, err := http.NewRequest( http.MethodPost, - a.githubAPI+"/user/codespaces/"+codespaceName+"/start", + u.String(), nil, ) if err != nil { @@ -567,18 +617,22 @@ func (a *API) StartCodespace(ctx context.Context, codespaceName string) error { } func (a *API) StopCodespace(ctx context.Context, codespaceName string, orgName string, userName string) error { - var stopURL string + var stopURL *safeurl.MutableSafeURL var spanName string + var err error if orgName != "" { - stopURL = fmt.Sprintf("%s/orgs/%s/members/%s/codespaces/%s/stop", a.githubAPI, orgName, userName, codespaceName) + stopURL, err = safeurl.JoinPathWithHostPrefix(a.githubAPI, "orgs", orgName, "members", userName, "codespaces", codespaceName, "stop") spanName = "/orgs/*/members/*/codespaces/*/stop" } else { - stopURL = fmt.Sprintf("%s/user/codespaces/%s/stop", a.githubAPI, codespaceName) + stopURL, err = safeurl.JoinPathWithHostPrefix(a.githubAPI, "user", "codespaces", codespaceName, "stop") spanName = "/user/codespaces/*/stop" } + if err != nil { + return err + } - req, err := http.NewRequest(http.MethodPost, stopURL, nil) + req, err := http.NewRequest(http.MethodPost, stopURL.String(), nil) if err != nil { return fmt.Errorf("error creating request: %w", err) } @@ -605,8 +659,11 @@ type Machine struct { // GetCodespacesMachines returns the codespaces machines for the given repo, branch and location. func (a *API) GetCodespacesMachines(ctx context.Context, repoID int64, branch, location string, devcontainerPath string) ([]*Machine, error) { - reqURL := fmt.Sprintf("%s/repositories/%d/codespaces/machines", a.githubAPI, repoID) - req, err := http.NewRequest(http.MethodGet, reqURL, nil) + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "repositories", strconv.FormatInt(repoID, 10), "codespaces", "machines") + if err != nil { + return nil, err + } + req, err := http.NewRequest(http.MethodGet, u.String(), nil) if err != nil { return nil, fmt.Errorf("error creating request: %w", err) } @@ -645,8 +702,11 @@ func (a *API) GetCodespacesMachines(ctx context.Context, repoID int64, branch, l // GetCodespacesPermissionsCheck returns a bool indicating whether the user has accepted permissions for the given repo and devcontainer path. func (a *API) GetCodespacesPermissionsCheck(ctx context.Context, repoID int64, branch string, devcontainerPath string) (bool, error) { - reqURL := fmt.Sprintf("%s/repositories/%d/codespaces/permissions_check", a.githubAPI, repoID) - req, err := http.NewRequest(http.MethodGet, reqURL, nil) + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "repositories", strconv.FormatInt(repoID, 10), "codespaces", "permissions_check") + if err != nil { + return false, err + } + req, err := http.NewRequest(http.MethodGet, u.String(), nil) if err != nil { return false, fmt.Errorf("error creating request: %w", err) } @@ -692,8 +752,11 @@ type RepoSearchParameters struct { // GetCodespaceRepoSuggestions searches for and returns repo names based on the provided search text. func (a *API) GetCodespaceRepoSuggestions(ctx context.Context, partialSearch string, parameters RepoSearchParameters) ([]string, error) { - reqURL := fmt.Sprintf("%s/search/repositories", a.githubAPI) - req, err := http.NewRequest(http.MethodGet, reqURL, nil) + reqURL, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "search", "repositories") + if err != nil { + return nil, err + } + req, err := http.NewRequest(http.MethodGet, reqURL.String(), nil) if err != nil { return nil, fmt.Errorf("error creating request: %w", err) } @@ -763,7 +826,15 @@ func (a *API) GetCodespaceRepoSuggestions(ctx context.Context, partialSearch str // GetCodespaceBillableOwner returns the billable owner and expected default values for // codespaces created by the user for a given repository. func (a *API) GetCodespaceBillableOwner(ctx context.Context, nwo string) (*User, error) { - req, err := http.NewRequest(http.MethodGet, a.githubAPI+"/repos/"+nwo+"/codespaces/new", nil) + owner, name, err := safeurl.RepoPartsFromNWO(nwo) + if err != nil { + return nil, err + } + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "repos", owner, name, "codespaces", "new") + if err != nil { + return nil, err + } + req, err := http.NewRequest(http.MethodGet, u.String(), nil) if err != nil { return nil, fmt.Errorf("error creating request: %w", err) } @@ -908,7 +979,11 @@ func (a *API) startCreate(ctx context.Context, params *CreateCodespaceParams) (* return nil, fmt.Errorf("error marshaling request: %w", err) } - req, err := http.NewRequest(http.MethodPost, a.githubAPI+"/user/codespaces", bytes.NewBuffer(requestBody)) + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "user", "codespaces") + if err != nil { + return nil, err + } + req, err := http.NewRequest(http.MethodPost, u.String(), bytes.NewBuffer(requestBody)) if err != nil { return nil, fmt.Errorf("error creating request: %w", err) } @@ -974,18 +1049,22 @@ func (a *API) startCreate(ctx context.Context, params *CreateCodespaceParams) (* // DeleteCodespace deletes the given codespace. func (a *API) DeleteCodespace(ctx context.Context, codespaceName string, orgName string, userName string) error { - var deleteURL string + var deleteURL *safeurl.MutableSafeURL var spanName string + var err error if orgName != "" && userName != "" { - deleteURL = fmt.Sprintf("%s/orgs/%s/members/%s/codespaces/%s", a.githubAPI, orgName, userName, codespaceName) + deleteURL, err = safeurl.JoinPathWithHostPrefix(a.githubAPI, "orgs", orgName, "members", userName, "codespaces", codespaceName) spanName = "/orgs/*/members/*/codespaces/*" } else { - deleteURL = a.githubAPI + "/user/codespaces/" + codespaceName + deleteURL, err = safeurl.JoinPathWithHostPrefix(a.githubAPI, "user", "codespaces", codespaceName) spanName = "/user/codespaces/*" } + if err != nil { + return err + } - req, err := http.NewRequest(http.MethodDelete, deleteURL, nil) + req, err := http.NewRequest(http.MethodDelete, deleteURL.String(), nil) if err != nil { return fmt.Errorf("error creating request: %w", err) } @@ -1017,15 +1096,18 @@ func (a *API) ListDevContainers(ctx context.Context, repoID int64, branch string perPage = limit } - v := url.Values{} - v.Set("per_page", strconv.Itoa(perPage)) + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "repositories", strconv.FormatInt(repoID, 10), "codespaces", "devcontainers") + if err != nil { + return nil, err + } + u.SetQuery("per_page", strconv.Itoa(perPage)) if branch != "" { - v.Set("ref", branch) + u.SetQuery("ref", branch) } - listURL := fmt.Sprintf("%s/repositories/%d/codespaces/devcontainers?%s", a.githubAPI, repoID, v.Encode()) + var listURL safeurl.SafeURL = u for { - req, err := http.NewRequest(http.MethodGet, listURL, nil) + req, err := http.NewRequest(http.MethodGet, listURL.String(), nil) if err != nil { return nil, fmt.Errorf("error creating request: %w", err) } @@ -1062,9 +1144,9 @@ func (a *API) ListDevContainers(ctx context.Context, repoID int64, branch string q := u.Query() q.Set("per_page", strconv.Itoa(newPerPage)) u.RawQuery = q.Encode() - listURL = u.String() + listURL = safeurl.NewImmutableSafeURL(u.String()) } else { - listURL = nextURL + listURL = safeurl.NewImmutableSafeURL(nextURL) } } @@ -1083,7 +1165,11 @@ func (a *API) EditCodespace(ctx context.Context, codespaceName string, params *E return nil, fmt.Errorf("error marshaling request: %w", err) } - req, err := http.NewRequest(http.MethodPatch, a.githubAPI+"/user/codespaces/"+codespaceName, bytes.NewBuffer(requestBody)) + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "user", "codespaces", codespaceName) + if err != nil { + return nil, err + } + req, err := http.NewRequest(http.MethodPatch, u.String(), bytes.NewBuffer(requestBody)) if err != nil { return nil, fmt.Errorf("error creating request: %w", err) } @@ -1139,7 +1225,15 @@ type getCodespaceRepositoryContentsResponse struct { } func (a *API) GetCodespaceRepositoryContents(ctx context.Context, codespace *Codespace, path string) ([]byte, error) { - req, err := http.NewRequest(http.MethodGet, a.githubAPI+"/repos/"+codespace.Repository.FullName+"/contents/"+path, nil) + owner, name, err := safeurl.RepoPartsFromNWO(codespace.Repository.FullName) + if err != nil { + return nil, err + } + u, err := safeurl.JoinPathWithHostPrefix(a.githubAPI, "repos", owner, name, "contents", path) + if err != nil { + return nil, err + } + req, err := http.NewRequest(http.MethodGet, u.String(), nil) if err != nil { return nil, fmt.Errorf("error creating request: %w", err) } diff --git a/internal/featuredetection/feature_detection.go b/internal/featuredetection/feature_detection.go index 88997708cca..e5bd8034faf 100644 --- a/internal/featuredetection/feature_detection.go +++ b/internal/featuredetection/feature_detection.go @@ -5,6 +5,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/gh" + "github.com/cli/cli/v2/internal/safeurl" "github.com/hashicorp/go-version" "golang.org/x/sync/errgroup" @@ -541,7 +542,11 @@ func resolveEnterpriseVersion(httpClient *http.Client, host string) (*version.Ve } apiClient := api.NewClientFromHTTP(httpClient) - err := apiClient.REST(host, "GET", "meta", nil, &metaResponse) + u, err := safeurl.JoinPath("meta") + if err != nil { + return nil, err + } + err = apiClient.REST(host, "GET", u.String(), nil, &metaResponse) if err != nil { return nil, err } diff --git a/internal/safeurl/safeurl.go b/internal/safeurl/safeurl.go new file mode 100644 index 00000000000..fccf40b9466 --- /dev/null +++ b/internal/safeurl/safeurl.go @@ -0,0 +1,164 @@ +// Package safeurl provides helpers for building REST API URL paths (and full +// URLs, when a host prefix is supplied) from variable components so that user +// or server controlled values cannot break the path or change which resource +// is addressed. +package safeurl + +import ( + "fmt" + "net/url" + "strings" +) + +// RepoPartsFromNWO parses a raw "owner/repo" string and returns the owner and name +// unescaped. It returns an error unless nwo contains exactly one slash with a non-empty +// owner and name, so a value carrying extra slashes cannot smuggle additional path +// segments through as the owner or name. +// +// This intentionally does not reuse ghrepo.FromFullName, which accepts the broader +// "[HOST/]OWNER/REPO" form. The call sites here only ever handle a bare "OWNER/REPO", +// so a stricter parse that rejects an unexpected host component is the safer fit. +func RepoPartsFromNWO(nwo string) (owner, name string, err error) { + parts := strings.Split(nwo, "/") + if len(parts) != 2 || parts[0] == "" || parts[1] == "" { + return "", "", fmt.Errorf("expected the \"OWNER/REPO\" format, got %q", nwo) + } + return parts[0], parts[1], nil +} + +// SafeURL is the sealed interface implemented by the URL types in this package. +// It exists so that a value known to address a safe REST API URL can be passed +// around and rendered without exposing how it was built. +type SafeURL interface { + String() string + + // The sealed method keeps the set of implementations closed to this package, + // so callers outside it cannot forge a value that claims to be safe. + sealed() +} + +// MutableSafeURL is a REST API URL built from a host prefix, path components, and query +// parameters. The path components and query parameters are URL encoded (aka +// percent-encoded) when the URL is rendered so that caller supplied values cannot +// alter the structure of the URL or change which resource it addresses; the host +// prefix is used as given. The zero value renders as the empty string. +type MutableSafeURL struct { + prefix string + components []string + query url.Values +} + +// JoinPath returns a SafeURL for the path made up of the given components. It +// returns an error if any component is exactly "..", which would traverse the URL +// path and change which resource it addresses. +func JoinPath(components ...string) (*MutableSafeURL, error) { + if err := checkTraversal(components); err != nil { + return nil, err + } + return &MutableSafeURL{components: components}, nil +} + +// JoinPathWithHostPrefix returns a SafeURL for the given host prefix and the path +// made up of the given components. It returns an error if any component is exactly +// "..", which would traverse the URL path and change which resource it addresses. +func JoinPathWithHostPrefix(hostPrefix string, components ...string) (*MutableSafeURL, error) { + if err := checkTraversal(components); err != nil { + return nil, err + } + return &MutableSafeURL{prefix: hostPrefix, components: components}, nil +} + +// checkTraversal returns an error if any component is exactly "..". Such a component +// survives percent-encoding as a real path segment and would traverse the URL path. +// A single "." is left alone because it does not traverse and is a legitimate value +// in some paths. +func checkTraversal(components []string) error { + for _, c := range components { + if c == ".." { + return fmt.Errorf("path component %q would traverse the URL path", c) + } + } + return nil +} + +func (u *MutableSafeURL) sealed() {} + +// SetQuery sets the query parameter key to value, replacing any existing value. +func (u *MutableSafeURL) SetQuery(key, value string) { + if u.query == nil { + u.query = url.Values{} + } + u.query.Set(key, value) +} + +// String renders the full URL. Path components and query parameters are URL encoded +// (aka percent-encoded) while the host prefix is included as given. The zero value +// renders as the empty string. +func (u *MutableSafeURL) String() string { + result := joinPathWithHostPrefix(u.prefix, u.components...) + if len(u.query) > 0 { + result += "?" + u.query.Encode() + } + return result +} + +// ImmutableSafeURL is a SafeURL that renders a fixed URL string verbatim. It exists +// so that a URL which was not built from percent-encoded components, such as a full +// URL returned by the server (a pagination "next" link, an asset download URL, and +// the like), can still flow through the SafeURL typed code paths. Because the stored +// value is rendered as given without any encoding, it is only safe to wrap a URL that +// was created from trusted components or received from a trusted source. +type ImmutableSafeURL struct { + url string +} + +// NewImmutableSafeURL returns an ImmutableSafeURL that renders url verbatim. Only pass +// a URL you built yourself from trusted components or received from a trusted source, +// such as a server response; this bypasses all percent-encoding, so passing a value +// that embeds unescaped user or third party input reintroduces the injection risk that +// SafeURL exists to prevent. +func NewImmutableSafeURL(url string) *ImmutableSafeURL { + return &ImmutableSafeURL{url: url} +} + +func (u *ImmutableSafeURL) sealed() {} + +// String returns the wrapped URL verbatim. +func (u *ImmutableSafeURL) String() string { + return u.url +} + +// joinPath builds a REST API URL path by percent-encoding each component with +// url.PathEscape and joining them with single slash separators. +// +// With no components, the empty string is returned. +func joinPath(components ...string) string { + // We build the path by hand rather than with url.JoinPath because url.JoinPath runs path.Clean + // on the result, which resolves any "." or ".." segments. Percent-encoding does not encode dots, + // so a component equal to "." or ".." would survive escaping and then be collapsed by the clean, + // silently changing which resource the path addresses. + escaped := make([]string, len(components)) + for i, c := range components { + escaped[i] = url.PathEscape(c) + } + return strings.Join(escaped, "/") +} + +// joinPathWithHostPrefix builds a full REST API URL by prepending hostPrefix to the path produced by +// JoinPath. A single slash is ensured at the join between hostPrefix and the path so they separate +// cleanly without doubling up. When hostPrefix is empty, the JoinPath result is returned intact, and +// when the joined path is empty, hostPrefix is returned intact. hostPrefix is used verbatim while each +// component is percent-encoded. +func joinPathWithHostPrefix(hostPrefix string, components ...string) string { + path := joinPath(components...) + if hostPrefix == "" { + return path + } + if path == "" { + return hostPrefix + } + if !strings.HasSuffix(hostPrefix, "/") { + return hostPrefix + "/" + path + } + return hostPrefix + path +} diff --git a/internal/safeurl/safeurl_test.go b/internal/safeurl/safeurl_test.go new file mode 100644 index 00000000000..41e641f4ab8 --- /dev/null +++ b/internal/safeurl/safeurl_test.go @@ -0,0 +1,317 @@ +package safeurl_test + +import ( + "testing" + + "github.com/cli/cli/v2/internal/safeurl" + "github.com/stretchr/testify/require" +) + +var _ safeurl.SafeURL = (*safeurl.MutableSafeURL)(nil) +var _ safeurl.SafeURL = (*safeurl.ImmutableSafeURL)(nil) + +func TestRepoPartsFromNWO(t *testing.T) { + + tests := []struct { + name string + nwo string + wantOwner string + wantName string + wantErr bool + }{ + { + name: "owner and repo", + nwo: "octocat/hello-world", + wantOwner: "octocat", + wantName: "hello-world", + }, + { + name: "no separator", + nwo: "octocat", + wantErr: true, + }, + { + name: "empty", + nwo: "", + wantErr: true, + }, + { + name: "missing name", + nwo: "octocat/", + wantErr: true, + }, + { + name: "missing owner", + nwo: "/hello-world", + wantErr: true, + }, + { + name: "parts are returned unescaped", + nwo: "my owner/my repo", + wantOwner: "my owner", + wantName: "my repo", + }, + { + name: "extra separators are rejected", + nwo: "foo/bar/codespaces", + wantErr: true, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + + owner, name, err := safeurl.RepoPartsFromNWO(tt.nwo) + if tt.wantErr { + require.Error(t, err) + } else { + require.NoError(t, err) + require.Equal(t, tt.wantOwner, owner) + require.Equal(t, tt.wantName, name) + } + }) + } +} + +func TestJoinPathRejectsTraversal(t *testing.T) { + tests := []struct { + name string + components []string + }{ + { + name: "only a .. component", + components: []string{".."}, + }, + { + name: "a .. component in the middle", + components: []string{"repos", "octocat", "..", "hello-world"}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + _, errJoinPath := safeurl.JoinPath(tt.components...) + require.Error(t, errJoinPath) + _, errJoinPathWithHostPrefix := safeurl.JoinPathWithHostPrefix("https://api.github.com", tt.components...) + require.Error(t, errJoinPathWithHostPrefix) + }) + } +} + +func TestMutableSafeURLString(t *testing.T) { + tests := []struct { + name string + url func(t *testing.T) (*safeurl.MutableSafeURL, error) + want string + }{ + { + name: "zero value renders empty", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return &safeurl.MutableSafeURL{}, nil + }, + want: "", + }, + { + name: "path only", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPath("foo", "bar", "baz") + }, + want: "foo/bar/baz", + }, + { + name: "single path component", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPath("foo") + }, + want: "foo", + }, + { + name: "empty components produce empty segments", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPath("", "bar", "") + }, + want: "/bar/", + }, + { + name: "escapes path components", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPath("foo", "bar baz", "a/b") + }, + want: "foo/bar%20baz/a%2Fb", + }, + { + name: "pre-encoded dot-dot cannot bypass the traversal check", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPath("foo", "bar", "%2e%2e", "baz") + }, + want: "foo/bar/%252e%252e/baz", + }, + { + name: "single dot component is preserved verbatim", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPath("foo", "bar", ".", "baz") + }, + want: "foo/bar/./baz", + }, + { + name: "leading single dot components are preserved verbatim", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPath(".", ".", "foo", "bar") + }, + want: "././foo/bar", + }, + { + name: "pre-encoded dot-dot cannot bypass the traversal check with host prefix", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPathWithHostPrefix("https://host", "foo", "bar", "%2e%2e", "baz") + }, + want: "https://host/foo/bar/%252e%252e/baz", + }, + { + name: "single dot component is preserved verbatim with host prefix", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPathWithHostPrefix("https://host", "foo", "bar", ".", "baz") + }, + want: "https://host/foo/bar/./baz", + }, + { + name: "leading single dot components are preserved verbatim with host prefix", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPathWithHostPrefix("https://host", ".", ".", "foo", "bar") + }, + want: "https://host/././foo/bar", + }, + { + name: "host prefix and path", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPathWithHostPrefix("https://host", "foo", "bar", "baz") + }, + want: "https://host/foo/bar/baz", + }, + { + name: "host prefix remains intact", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPathWithHostPrefix("https://host/with/slash", "foo", "bar", "baz") + }, + want: "https://host/with/slash/foo/bar/baz", + }, + { + name: "host prefix with trailing slash", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPathWithHostPrefix("https://host/", "foo", "bar", "baz") + }, + want: "https://host/foo/bar/baz", + }, + { + name: "host prefix without path", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPathWithHostPrefix("https://host") + }, + want: "https://host", + }, + { + name: "host prefix with trailing slash and no path", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + return safeurl.JoinPathWithHostPrefix("https://host/") + }, + want: "https://host/", + }, + { + name: "query only", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + u := &safeurl.MutableSafeURL{} + u.SetQuery("page", "2") + return u, nil + }, + want: "?page=2", + }, + { + name: "path and query", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + u, err := safeurl.JoinPath("foo", "bar", "baz") + require.NoError(t, err) + u.SetQuery("value", "x") + return u, nil + }, + want: "foo/bar/baz?value=x", + }, + { + name: "host prefix, path, and query", + url: func(t *testing.T) (*safeurl.MutableSafeURL, error) { + u, err := safeurl.JoinPathWithHostPrefix("https://host", "foo", "bar") + require.NoError(t, err) + u.SetQuery("value", "x y") + return u, nil + }, + want: "https://host/foo/bar?value=x+y", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + u, err := tt.url(t) + require.NoError(t, err) + require.Equal(t, tt.want, u.String()) + }) + } +} + +func TestMutableSafeURLSetQuery(t *testing.T) { + type query struct { + key string + value string + } + + tests := []struct { + name string + queries []query + want string + }{ + { + name: "replaces existing value rather than appending", + queries: []query{{"a", "1"}, {"a", "2"}}, + want: "foo/bar?a=2", + }, + { + name: "sorts keys deterministically", + queries: []query{{"b", "2"}, {"a", "1"}}, + want: "foo/bar?a=1&b=2", + }, + { + name: "escapes keys and values", + queries: []query{{"a", "x y&z"}}, + want: "foo/bar?a=x+y%26z", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + u, err := safeurl.JoinPath("foo", "bar") + require.NoError(t, err) + for _, q := range tt.queries { + u.SetQuery(q.key, q.value) + } + require.Equal(t, tt.want, u.String()) + }) + } +} + +func TestImmutableSafeURLString(t *testing.T) { + tests := []struct { + name string + url string + want string + }{ + { + name: "empty renders empty", + url: "", + want: "", + }, + { + name: "renders the wrapped url verbatim without encoding", + url: "https://host/foo/bar baz/?value=x y", + want: "https://host/foo/bar baz/?value=x y", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + require.Equal(t, tt.want, safeurl.NewImmutableSafeURL(tt.url).String()) + }) + } +} diff --git a/internal/skills/discovery/discovery.go b/internal/skills/discovery/discovery.go index ead18a0fbfb..694662900e6 100644 --- a/internal/skills/discovery/discovery.go +++ b/internal/skills/discovery/discovery.go @@ -6,7 +6,6 @@ import ( "fmt" "io" "net/http" - "net/url" "os" "path" "path/filepath" @@ -17,6 +16,7 @@ import ( "sync/atomic" "github.com/cli/cli/v2/api" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/skills/frontmatter" "github.com/cli/cli/v2/pkg/iostreams" ) @@ -189,11 +189,14 @@ func parseRepoVisibility(s string) (RepoVisibility, error) { // FetchRepoVisibility returns the repository visibility: "public", "private", or "internal". func FetchRepoVisibility(client *api.Client, host, owner, repo string) (RepoVisibility, error) { - apiPath := fmt.Sprintf("repos/%s/%s", url.PathEscape(owner), url.PathEscape(repo)) + apiPath, err := safeurl.JoinPath("repos", owner, repo) + if err != nil { + return "", err + } var resp struct { Visibility string `json:"visibility"` } - if err := client.REST(host, "GET", apiPath, nil, &resp); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &resp); err != nil { return "", err } return parseRepoVisibility(resp.Visibility) @@ -252,11 +255,14 @@ func resolveExplicitRef(client *api.Client, host, owner, repo, ref string) (*Res return nil, err } - commitPath := fmt.Sprintf("repos/%s/%s/commits/%s", url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(ref)) + commitPath, err := safeurl.JoinPath("repos", owner, repo, "commits", ref) + if err != nil { + return nil, err + } var commitResp struct { SHA string `json:"sha"` } - if err := client.REST(host, "GET", commitPath, nil, &commitResp); err == nil { + if err := client.REST(host, "GET", commitPath.String(), nil, &commitResp); err == nil { return &ResolvedRef{Ref: commitResp.SHA, SHA: commitResp.SHA}, nil } else if !isNotFound(err) { return nil, err @@ -268,25 +274,31 @@ func resolveExplicitRef(client *api.Client, host, owner, repo, ref string) (*Res // resolveTagRef looks up a tag by short name and returns a fully qualified ref. // For annotated tags, the tag object is dereferenced to obtain the commit SHA. func resolveTagRef(client *api.Client, host, owner, repo, tag string) (*ResolvedRef, error) { - tagPath := fmt.Sprintf("repos/%s/%s/git/ref/tags/%s", url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(tag)) + tagPath, err := safeurl.JoinPath("repos", owner, repo, "git", "ref", fmt.Sprintf("tags/%s", tag)) + if err != nil { + return nil, err + } var refResp struct { Object struct { SHA string `json:"sha"` Type string `json:"type"` } `json:"object"` } - if err := client.REST(host, "GET", tagPath, nil, &refResp); err != nil { + if err := client.REST(host, "GET", tagPath.String(), nil, &refResp); err != nil { return nil, fmt.Errorf("tag %q not found in %s/%s: %w", tag, owner, repo, err) } sha := refResp.Object.SHA if refResp.Object.Type == "tag" { - derefPath := fmt.Sprintf("repos/%s/%s/git/tags/%s", url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(sha)) + derefPath, err := safeurl.JoinPath("repos", owner, repo, "git", "tags", sha) + if err != nil { + return nil, err + } var tagResp struct { Object struct { SHA string `json:"sha"` } `json:"object"` } - if err := client.REST(host, "GET", derefPath, nil, &tagResp); err != nil { + if err := client.REST(host, "GET", derefPath.String(), nil, &tagResp); err != nil { return nil, fmt.Errorf("could not dereference annotated tag %q: %w", tag, err) } sha = tagResp.Object.SHA @@ -296,13 +308,16 @@ func resolveTagRef(client *api.Client, host, owner, repo, tag string) (*Resolved // resolveBranchRef looks up a branch by short name and returns a fully qualified ref. func resolveBranchRef(client *api.Client, host, owner, repo, branch string) (*ResolvedRef, error) { - refPath := fmt.Sprintf("repos/%s/%s/git/ref/heads/%s", url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(branch)) + refPath, err := safeurl.JoinPath("repos", owner, repo, "git", "ref", fmt.Sprintf("heads/%s", branch)) + if err != nil { + return nil, err + } var refResp struct { Object struct { SHA string `json:"sha"` } `json:"object"` } - if err := client.REST(host, "GET", refPath, nil, &refResp); err != nil { + if err := client.REST(host, "GET", refPath.String(), nil, &refResp); err != nil { return nil, fmt.Errorf("branch %q not found in %s/%s: %w", branch, owner, repo, err) } return &ResolvedRef{Ref: "refs/heads/" + branch, SHA: refResp.Object.SHA}, nil @@ -324,11 +339,14 @@ type noReleasesError struct { func (e *noReleasesError) Error() string { return e.reason } func resolveLatestRelease(client *api.Client, host, owner, repo string) (*ResolvedRef, error) { - apiPath := fmt.Sprintf("repos/%s/%s/releases/latest", url.PathEscape(owner), url.PathEscape(repo)) + apiPath, err := safeurl.JoinPath("repos", owner, repo, "releases", "latest") + if err != nil { + return nil, err + } var resp struct { TagName string `json:"tag_name"` } - if err := client.REST(host, "GET", apiPath, nil, &resp); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &resp); err != nil { // A 404 means the repository has no releases. This is the // only case where falling back to the default branch is safe. // Any other HTTP error (403, 500, …) or network failure is @@ -346,11 +364,14 @@ func resolveLatestRelease(client *api.Client, host, owner, repo string) (*Resolv } func resolveDefaultBranch(client *api.Client, host, owner, repo string) (*ResolvedRef, error) { - apiPath := fmt.Sprintf("repos/%s/%s", url.PathEscape(owner), url.PathEscape(repo)) + apiPath, err := safeurl.JoinPath("repos", owner, repo) + if err != nil { + return nil, err + } var resp struct { DefaultBranch string `json:"default_branch"` } - if err := client.REST(host, "GET", apiPath, nil, &resp); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &resp); err != nil { return nil, fmt.Errorf("could not determine default branch: %w", err) } branch := resp.DefaultBranch @@ -552,9 +573,13 @@ func DiscoverSkills(client *api.Client, host, owner, repo, commitSHA string) ([] // DiscoverSkillsWithOptions finds all skills in a repository at the given // commit SHA, with configurable discovery behavior. func DiscoverSkillsWithOptions(client *api.Client, host, owner, repo, commitSHA string, opts DiscoverOptions) ([]Skill, error) { - apiPath := fmt.Sprintf("repos/%s/%s/git/trees/%s?recursive=true", url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(commitSHA)) + apiPath, err := safeurl.JoinPath("repos", owner, repo, "git", "trees", commitSHA) + if err != nil { + return nil, err + } + apiPath.SetQuery("recursive", "true") var tree treeResponse - if err := client.REST(host, "GET", apiPath, nil, &tree); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &tree); err != nil { return nil, fmt.Errorf("could not fetch repository tree: %w", err) } @@ -698,7 +723,11 @@ func DiscoverSkillByPathWithOptions(client *api.Client, host, owner, repo, commi } parentPath := path.Dir(skillPath) - apiPath := fmt.Sprintf("repos/%s/%s/contents/%s?ref=%s", url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(parentPath), commitSHA) + apiPath, err := safeurl.JoinPath("repos", owner, repo, "contents", parentPath) + if err != nil { + return nil, err + } + apiPath.SetQuery("ref", commitSHA) var contents []struct { Name string `json:"name"` @@ -706,7 +735,7 @@ func DiscoverSkillByPathWithOptions(client *api.Client, host, owner, repo, commi SHA string `json:"sha"` Type string `json:"type"` } - if err := client.REST(host, "GET", apiPath, nil, &contents); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &contents); err != nil { return nil, fmt.Errorf("path %q not found in %s/%s: %w", parentPath, owner, repo, err) } @@ -721,9 +750,12 @@ func DiscoverSkillByPathWithOptions(client *api.Client, host, owner, repo, commi return nil, fmt.Errorf("skill directory %q not found in %s/%s", skillPath, owner, repo) } - skillTreePath := fmt.Sprintf("repos/%s/%s/git/trees/%s", url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(treeSHA)) + skillTreePath, err := safeurl.JoinPath("repos", owner, repo, "git", "trees", treeSHA) + if err != nil { + return nil, err + } var skillTree treeResponse - if err := client.REST(host, "GET", skillTreePath, nil, &skillTree); err != nil { + if err := client.REST(host, "GET", skillTreePath.String(), nil, &skillTree); err != nil { return nil, fmt.Errorf("could not read skill directory: %w", err) } @@ -779,9 +811,13 @@ func DiscoverSkillByPathWithOptions(client *api.Client, host, owner, repo, commi // DiscoverSkillFiles returns all file paths belonging to a skill directory // by fetching the skill's subtree directly using its tree SHA. func DiscoverSkillFiles(client *api.Client, host, owner, repo, treeSHA, skillPath string) ([]SkillFile, error) { - apiPath := fmt.Sprintf("repos/%s/%s/git/trees/%s?recursive=true", url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(treeSHA)) + apiPath, err := safeurl.JoinPath("repos", owner, repo, "git", "trees", treeSHA) + if err != nil { + return nil, err + } + apiPath.SetQuery("recursive", "true") var tree treeResponse - if err := client.REST(host, "GET", apiPath, nil, &tree); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &tree); err != nil { return nil, fmt.Errorf("could not fetch skill tree: %w", err) } @@ -807,9 +843,13 @@ func DiscoverSkillFiles(client *api.Client, host, owner, repo, treeSHA, skillPat // ListSkillFiles returns all files in a skill directory as public SkillFile // structs with paths relative to the skill root. func ListSkillFiles(client *api.Client, host, owner, repo, treeSHA string) ([]SkillFile, error) { - apiPath := fmt.Sprintf("repos/%s/%s/git/trees/%s?recursive=true", url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(treeSHA)) + apiPath, err := safeurl.JoinPath("repos", owner, repo, "git", "trees", treeSHA) + if err != nil { + return nil, err + } + apiPath.SetQuery("recursive", "true") var tree treeResponse - if err := client.REST(host, "GET", apiPath, nil, &tree); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &tree); err != nil { return nil, fmt.Errorf("could not fetch skill tree: %w", err) } @@ -842,9 +882,12 @@ func walkTree(client *api.Client, host, owner, repo, sha, prefix string, depth i if depth > maxTreeDepth { return nil, fmt.Errorf("tree depth exceeds %d levels at %s", maxTreeDepth, prefix) } - apiPath := fmt.Sprintf("repos/%s/%s/git/trees/%s", url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(sha)) + apiPath, err := safeurl.JoinPath("repos", owner, repo, "git", "trees", sha) + if err != nil { + return nil, err + } var tree treeResponse - if err := client.REST(host, "GET", apiPath, nil, &tree); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &tree); err != nil { return nil, fmt.Errorf("could not fetch tree %s: %w", prefix, err) } @@ -873,13 +916,16 @@ func walkTree(client *api.Client, host, owner, repo, sha, prefix string, depth i // iostreams.Untrusted and callers must choose sanitized display or raw // round-tripping. func FetchBlob(client *api.Client, host, owner, repo, sha string) (iostreams.Untrusted, error) { - apiPath := fmt.Sprintf("repos/%s/%s/git/blobs/%s", url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(sha)) + apiPath, err := safeurl.JoinPath("repos", owner, repo, "git", "blobs", sha) + if err != nil { + return iostreams.Untrusted{}, err + } var resp struct { SHA string `json:"sha"` Content string `json:"content"` Encoding string `json:"encoding"` } - if err := client.REST(host, "GET", apiPath, nil, &resp); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &resp); err != nil { return iostreams.Untrusted{}, fmt.Errorf("could not fetch blob: %w", err) } diff --git a/internal/skills/discovery/discovery_test.go b/internal/skills/discovery/discovery_test.go index 0ecb8aa5708..cc7c35104a7 100644 --- a/internal/skills/discovery/discovery_test.go +++ b/internal/skills/discovery/discovery_test.go @@ -431,7 +431,7 @@ func TestResolveRef(t *testing.T) { version: "main", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/main"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Fmain"), httpmock.JSONResponse(map[string]interface{}{ "object": map[string]interface{}{"sha": "branch-sha"}, })) @@ -444,10 +444,10 @@ func TestResolveRef(t *testing.T) { version: "v1.0", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/v1.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Fv1.0"), httpmock.StatusStringResponse(404, "not found")) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v1.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv1.0"), httpmock.JSONResponse(map[string]interface{}{ "object": map[string]interface{}{"sha": "abc123", "type": "commit"}, })) @@ -460,10 +460,10 @@ func TestResolveRef(t *testing.T) { version: "v2.0", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/v2.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Fv2.0"), httpmock.StatusStringResponse(404, "not found")) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v2.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv2.0"), httpmock.JSONResponse(map[string]interface{}{ "object": map[string]interface{}{"sha": "tag-obj-sha", "type": "tag"}, })) @@ -481,10 +481,10 @@ func TestResolveRef(t *testing.T) { version: "deadbeef", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/deadbeef"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Fdeadbeef"), httpmock.StatusStringResponse(404, "not found")) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/deadbeef"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fdeadbeef"), httpmock.StatusStringResponse(404, "not found")) reg.Register( httpmock.REST("GET", "repos/monalisa/octocat-skills/commits/deadbeef"), @@ -498,10 +498,10 @@ func TestResolveRef(t *testing.T) { version: "nonexistent", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/nonexistent"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Fnonexistent"), httpmock.StatusStringResponse(404, "not found")) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/nonexistent"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fnonexistent"), httpmock.StatusStringResponse(404, "not found")) reg.Register( httpmock.REST("GET", "repos/monalisa/octocat-skills/commits/nonexistent"), @@ -514,7 +514,7 @@ func TestResolveRef(t *testing.T) { version: "release", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/release"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Frelease"), httpmock.JSONResponse(map[string]interface{}{ "object": map[string]interface{}{"sha": "branch-sha"}, })) @@ -528,7 +528,7 @@ func TestResolveRef(t *testing.T) { version: "refs/tags/v1.0", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v1.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv1.0"), httpmock.JSONResponse(map[string]interface{}{ "object": map[string]interface{}{"sha": "tag-sha", "type": "commit"}, })) @@ -541,7 +541,7 @@ func TestResolveRef(t *testing.T) { version: "refs/heads/feature", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/feature"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Ffeature"), httpmock.JSONResponse(map[string]interface{}{ "object": map[string]interface{}{"sha": "feature-sha"}, })) @@ -554,7 +554,7 @@ func TestResolveRef(t *testing.T) { version: "refs/tags/nonexistent", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/nonexistent"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fnonexistent"), httpmock.StatusStringResponse(404, "not found")) }, wantErr: `tag "nonexistent" not found in monalisa/octocat-skills`, @@ -564,7 +564,7 @@ func TestResolveRef(t *testing.T) { version: "refs/heads/nonexistent", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/nonexistent"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Fnonexistent"), httpmock.StatusStringResponse(404, "not found")) }, wantErr: `branch "nonexistent" not found in monalisa/octocat-skills`, @@ -576,7 +576,7 @@ func TestResolveRef(t *testing.T) { httpmock.REST("GET", "repos/monalisa/octocat-skills/releases/latest"), httpmock.JSONResponse(map[string]interface{}{"tag_name": "v3.0"})) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v3.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv3.0"), httpmock.JSONResponse(map[string]interface{}{ "object": map[string]interface{}{"sha": "release-sha", "type": "commit"}, })) @@ -594,7 +594,7 @@ func TestResolveRef(t *testing.T) { httpmock.REST("GET", "repos/monalisa/octocat-skills"), httpmock.JSONResponse(map[string]interface{}{"default_branch": "main"})) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/main"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Fmain"), httpmock.JSONResponse(map[string]interface{}{ "object": map[string]interface{}{"sha": "branch-sha"}, })) @@ -607,7 +607,7 @@ func TestResolveRef(t *testing.T) { version: "refs/tags/v4.0", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v4.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv4.0"), httpmock.JSONResponse(map[string]interface{}{ "object": map[string]interface{}{"sha": "tag-obj-sha", "type": "tag"}, })) @@ -645,7 +645,7 @@ func TestResolveRef(t *testing.T) { httpmock.REST("GET", "repos/monalisa/octocat-skills"), httpmock.JSONResponse(map[string]interface{}{"default_branch": "main"})) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/main"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Fmain"), httpmock.JSONResponse(map[string]interface{}{ "object": map[string]interface{}{"sha": "fallback-sha"}, })) @@ -670,7 +670,7 @@ func TestResolveRef(t *testing.T) { version: "main", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/main"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Fmain"), httpmock.StatusStringResponse(500, "server error")) }, wantErr: `branch "main" not found in monalisa/octocat-skills`, @@ -680,7 +680,7 @@ func TestResolveRef(t *testing.T) { version: "develop", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/develop"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Fdevelop"), httpmock.StatusStringResponse(403, "forbidden")) }, wantErr: `branch "develop" not found in monalisa/octocat-skills`, @@ -690,10 +690,10 @@ func TestResolveRef(t *testing.T) { version: "v5.0", stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads/v5.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/heads%2Fv5.0"), httpmock.StatusStringResponse(404, "not found")) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v5.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv5.0"), httpmock.StatusStringResponse(500, "server error")) }, wantErr: `tag "v5.0" not found in monalisa/octocat-skills`, diff --git a/internal/update/update.go b/internal/update/update.go index 20cd09606c8..27a7d8a248f 100644 --- a/internal/update/update.go +++ b/internal/update/update.go @@ -14,6 +14,7 @@ import ( "time" "github.com/cli/cli/v2/internal/ci" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/extensions" "github.com/hashicorp/go-version" "github.com/mattn/go-isatty" @@ -112,7 +113,15 @@ func CheckForUpdate(ctx context.Context, client *http.Client, stateFilePath, rep } func getLatestReleaseInfo(ctx context.Context, client *http.Client, repo string) (*ReleaseInfo, error) { - req, err := http.NewRequestWithContext(ctx, "GET", fmt.Sprintf("https://api.github.com/repos/%s/releases/latest", repo), nil) + owner, name, err := safeurl.RepoPartsFromNWO(repo) + if err != nil { + return nil, err + } + u, err := safeurl.JoinPathWithHostPrefix("https://api.github.com", "repos", owner, name, "releases", "latest") + if err != nil { + return nil, err + } + req, err := http.NewRequestWithContext(ctx, "GET", u.String(), nil) if err != nil { return nil, err } diff --git a/pkg/cmd/agent-task/capi/job.go b/pkg/cmd/agent-task/capi/job.go index 551e7c9ae31..eda3106819d 100644 --- a/pkg/cmd/agent-task/capi/job.go +++ b/pkg/cmd/agent-task/capi/job.go @@ -8,8 +8,9 @@ import ( "fmt" "io" "net/http" - "net/url" "time" + + "github.com/cli/cli/v2/internal/safeurl" ) const defaultEventType = "gh_cli" @@ -66,7 +67,10 @@ func (c *CAPIClient) CreateJob(ctx context.Context, owner, repo, problemStatemen return nil, errors.New("problem statement is required") } - url := fmt.Sprintf("%s/%s/%s", c.jobsBasePathV1(), url.PathEscape(owner), url.PathEscape(repo)) + u, err := safeurl.JoinPathWithHostPrefix(c.jobsBasePathV1(), owner, repo) + if err != nil { + return nil, err + } prOpts := JobPullRequest{} if baseBranch != "" { @@ -82,7 +86,7 @@ func (c *CAPIClient) CreateJob(ctx context.Context, owner, repo, problemStatemen b, _ := json.Marshal(payload) - req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(b)) + req, err := http.NewRequestWithContext(ctx, http.MethodPost, u.String(), bytes.NewReader(b)) if err != nil { return nil, err } @@ -132,8 +136,11 @@ func (c *CAPIClient) GetJob(ctx context.Context, owner, repo, jobID string) (*Jo if owner == "" || repo == "" || jobID == "" { return nil, errors.New("owner, repo, and jobID are required") } - url := fmt.Sprintf("%s/%s/%s/%s", c.jobsBasePathV1(), url.PathEscape(owner), url.PathEscape(repo), url.PathEscape(jobID)) - req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, http.NoBody) + u, err := safeurl.JoinPathWithHostPrefix(c.jobsBasePathV1(), owner, repo, jobID) + if err != nil { + return nil, err + } + req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), http.NoBody) if err != nil { return nil, err } diff --git a/pkg/cmd/agent-task/capi/sessions.go b/pkg/cmd/agent-task/capi/sessions.go index 69ea80820c5..9a9d164189e 100644 --- a/pkg/cmd/agent-task/capi/sessions.go +++ b/pkg/cmd/agent-task/capi/sessions.go @@ -10,12 +10,12 @@ import ( "io" "math" "net/http" - "net/url" "slices" "strconv" "time" "github.com/cli/cli/v2/api" + "github.com/cli/cli/v2/internal/safeurl" "github.com/shurcooL/githubv4" "github.com/vmihailenco/msgpack/v5" ) @@ -217,16 +217,16 @@ func (c *CAPIClient) ListLatestSessionsForViewer(ctx context.Context, limit int) return nil, nil } - sessionsURL, err := url.JoinPath(c.capiBaseURL, "agents", "sessions") + sessionsURL, err := safeurl.JoinPathWithHostPrefix(c.capiBaseURL, "agents", "sessions") if err != nil { - return nil, fmt.Errorf("failed to build sessions URL: %w", err) + return nil, err } pageSize := defaultSessionsPerPage seenResources := make(map[int64]struct{}) latestSessions := make([]session, 0, limit) for page := 1; ; page++ { - req, err := http.NewRequestWithContext(ctx, http.MethodGet, sessionsURL, http.NoBody) + req, err := http.NewRequestWithContext(ctx, http.MethodGet, sessionsURL.String(), http.NoBody) if err != nil { return nil, err } @@ -299,9 +299,12 @@ func (c *CAPIClient) GetSession(ctx context.Context, id string) (*Session, error return nil, fmt.Errorf("missing session ID") } - url := fmt.Sprintf("%s/agents/sessions/%s", c.capiBaseURL, url.PathEscape(id)) + u, err := safeurl.JoinPathWithHostPrefix(c.capiBaseURL, "agents", "sessions", id) + if err != nil { + return nil, err + } - req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, http.NoBody) + req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), http.NoBody) if err != nil { return nil, err } @@ -338,9 +341,12 @@ func (c *CAPIClient) GetSessionLogs(ctx context.Context, id string) ([]byte, err return nil, fmt.Errorf("missing session ID") } - url := fmt.Sprintf("%s/agents/sessions/%s/logs", c.capiBaseURL, url.PathEscape(id)) + u, err := safeurl.JoinPathWithHostPrefix(c.capiBaseURL, "agents", "sessions", id, "logs") + if err != nil { + return nil, err + } - req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, http.NoBody) + req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), http.NoBody) if err != nil { return nil, err } @@ -371,9 +377,12 @@ func (c *CAPIClient) ListSessionsByResourceID(ctx context.Context, resourceType return nil, nil } - url := fmt.Sprintf("%s/agents/resource/%s/%d", c.capiBaseURL, url.PathEscape(resourceType), resourceID) + u, err := safeurl.JoinPathWithHostPrefix(c.capiBaseURL, "agents", "resource", resourceType, strconv.FormatInt(resourceID, 10)) + if err != nil { + return nil, err + } - req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, http.NoBody) + req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), http.NoBody) if err != nil { return nil, err } diff --git a/pkg/cmd/api/http.go b/pkg/cmd/api/http.go index b1b503cc2ce..337a07b7d0a 100644 --- a/pkg/cmd/api/http.go +++ b/pkg/cmd/api/http.go @@ -21,6 +21,8 @@ func httpRequest(client *http.Client, hostname string, method string, p string, } else if isGraphQL { requestURL = ghinstance.GraphQLEndpoint(hostname) } else { + // Note that the gh api command takes the path verbatim from the user, so we + // intentionally do not route it through safeurl and do not escape it here. requestURL = ghinstance.RESTPrefix(hostname) + strings.TrimPrefix(p, "/") } diff --git a/pkg/cmd/attestation/api/attestation.go b/pkg/cmd/attestation/api/attestation.go index 68ccff77e64..fc699d7d94c 100644 --- a/pkg/cmd/attestation/api/attestation.go +++ b/pkg/cmd/attestation/api/attestation.go @@ -8,11 +8,6 @@ import ( "github.com/sigstore/sigstore-go/pkg/bundle" ) -const ( - GetAttestationByRepoAndSubjectDigestPath = "repos/%s/attestations/%s" - GetAttestationByOwnerAndSubjectDigestPath = "orgs/%s/attestations/%s" -) - var ErrNoAttestationsFound = errors.New("no attestations found") type Attestation struct { diff --git a/pkg/cmd/attestation/api/client.go b/pkg/cmd/attestation/api/client.go index 41c713d7c0c..0cb3a5a1e81 100644 --- a/pkg/cmd/attestation/api/client.go +++ b/pkg/cmd/attestation/api/client.go @@ -5,12 +5,13 @@ import ( "fmt" "io" "net/http" - neturl "net/url" + "strconv" "strings" "time" "github.com/cenkalti/backoff/v4" "github.com/cli/cli/v2/api" + "github.com/cli/cli/v2/internal/safeurl" ioconfig "github.com/cli/cli/v2/pkg/cmd/attestation/io" "github.com/klauspost/compress/snappy" v1 "github.com/sigstore/protobuf-specs/gen/pb-go/bundle/v1" @@ -100,18 +101,29 @@ func (c *LiveClient) GetByDigest(params FetchParams) ([]*Attestation, error) { return bundles, nil } -func (c *LiveClient) buildRequestURL(params FetchParams) (string, error) { +func (c *LiveClient) buildRequestURL(params FetchParams) (safeurl.SafeURL, error) { if err := params.Validate(); err != nil { - return "", err + return nil, err } - var url string + var u *safeurl.MutableSafeURL if params.Repo != "" { // check if Repo is set first because if Repo has been set, Owner will be set using the value of Repo. // If Repo is not set, the field will remain empty. It will not be populated using the value of Owner. - url = fmt.Sprintf(GetAttestationByRepoAndSubjectDigestPath, params.Repo, params.Digest) + owner, name, err := safeurl.RepoPartsFromNWO(params.Repo) + if err != nil { + return nil, err + } + u, err = safeurl.JoinPath("repos", owner, name, "attestations", params.Digest) + if err != nil { + return nil, err + } } else { - url = fmt.Sprintf(GetAttestationByOwnerAndSubjectDigestPath, params.Owner, params.Digest) + var err error + u, err = safeurl.JoinPath("orgs", params.Owner, "attestations", params.Digest) + if err != nil { + return nil, err + } } perPage := params.Limit @@ -120,15 +132,15 @@ func (c *LiveClient) buildRequestURL(params FetchParams) (string, error) { } // ref: https://github.com/cli/go-gh/blob/d32c104a9a25c9de3d7c7b07a43ae0091441c858/example_gh_test.go#L96 - url = fmt.Sprintf("%s?per_page=%d", url, perPage) + u.SetQuery("per_page", strconv.Itoa(perPage)) if params.PredicateType != "" { - url = fmt.Sprintf("%s&predicate_type=%s", url, neturl.QueryEscape(params.PredicateType)) + u.SetQuery("predicate_type", params.PredicateType) } - return url, nil + return u, nil } func (c *LiveClient) getAttestations(params FetchParams) ([]*Attestation, error) { - url, err := c.buildRequestURL(params) + u, err := c.buildRequestURL(params) if err != nil { return nil, err } @@ -137,10 +149,12 @@ func (c *LiveClient) getAttestations(params FetchParams) ([]*Attestation, error) var resp AttestationsResponse bo := backoff.NewConstantBackOff(getAttestationRetryInterval) + var pageURL safeurl.SafeURL = u + // if no attestation or less than limit, then keep fetching - for url != "" && len(attestations) < params.Limit { + for pageURL.String() != "" && len(attestations) < params.Limit { err := backoff.Retry(func() error { - newURL, restErr := c.githubAPI.RESTWithNext(c.host, http.MethodGet, url, nil, &resp) + newURL, restErr := c.githubAPI.RESTWithNext(c.host, http.MethodGet, pageURL.String(), nil, &resp) if restErr != nil { if shouldRetry(restErr) { return restErr @@ -148,7 +162,7 @@ func (c *LiveClient) getAttestations(params FetchParams) ([]*Attestation, error) return backoff.Permanent(restErr) } - url = newURL + pageURL = safeurl.NewImmutableSafeURL(newURL) // filter by the initiator type if params.Initiator != "" { @@ -201,7 +215,7 @@ func (c *LiveClient) fetchBundleFromAttestations(attestations []*Attestation) ([ } // otherwise fetch the bundle with the provided URL - b, err := c.getBundle(a.BundleURL) + b, err := c.getBundle(safeurl.NewImmutableSafeURL(a.BundleURL)) if err != nil { return fmt.Errorf("failed to fetch bundle with URL: %w", err) } @@ -220,19 +234,19 @@ func (c *LiveClient) fetchBundleFromAttestations(attestations []*Attestation) ([ return fetched, nil } -func (c *LiveClient) getBundle(url string) (*bundle.Bundle, error) { +func (c *LiveClient) getBundle(url safeurl.SafeURL) (*bundle.Bundle, error) { c.logger.VerbosePrintf("Fetching attestation bundle with bundle URL\n\n") var sgBundle *bundle.Bundle bo := backoff.NewConstantBackOff(getAttestationRetryInterval) err := backoff.Retry(func() error { - resp, err := c.externalHttpClient.Get(url) + resp, err := c.externalHttpClient.Get(url.String()) if err != nil { return fmt.Errorf("request to fetch bundle from URL failed: %w", err) } if resp.StatusCode >= 500 && resp.StatusCode <= 599 { - return fmt.Errorf("attestation bundle with URL %s returned status code %d", url, resp.StatusCode) + return fmt.Errorf("attestation bundle with URL %s returned status code %d", url.String(), resp.StatusCode) } defer resp.Body.Close() @@ -279,15 +293,19 @@ func shouldRetry(err error) bool { // GetTrustDomain returns the current trust domain. If the default is used // the empty string is returned func (c *LiveClient) GetTrustDomain() (string, error) { - return c.getTrustDomain(MetaPath) + u, err := safeurl.JoinPath(MetaPath) + if err != nil { + return "", err + } + return c.getTrustDomain(u) } -func (c *LiveClient) getTrustDomain(url string) (string, error) { +func (c *LiveClient) getTrustDomain(u safeurl.SafeURL) (string, error) { var resp MetaResponse bo := backoff.NewConstantBackOff(getAttestationRetryInterval) err := backoff.Retry(func() error { - restErr := c.githubAPI.REST(c.host, http.MethodGet, url, nil, &resp) + restErr := c.githubAPI.REST(c.host, http.MethodGet, u.String(), nil, &resp) if restErr != nil { if shouldRetry(restErr) { return restErr diff --git a/pkg/cmd/attestation/api/client_test.go b/pkg/cmd/attestation/api/client_test.go index e27297b51d2..9f96be3448e 100644 --- a/pkg/cmd/attestation/api/client_test.go +++ b/pkg/cmd/attestation/api/client_test.go @@ -3,6 +3,7 @@ package api import ( "testing" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/attestation/io" "github.com/cli/cli/v2/pkg/cmd/attestation/test/data" "github.com/stretchr/testify/require" @@ -261,7 +262,7 @@ func TestGetBundle(t *testing.T) { logger: io.NewTestHandler(), } - b, err := c.getBundle("https://mybundleurl.com") + b, err := c.getBundle(safeurl.NewImmutableSafeURL("https://mybundleurl.com")) require.NoError(t, err) require.Equal(t, "application/vnd.dev.sigstore.bundle.v0.3+json", b.GetMediaType()) mockHTTPClient.AssertNumberOfCalls(t, "OnGetSuccess", 1) @@ -280,7 +281,7 @@ func TestGetBundle_SuccessfulRetry(t *testing.T) { logger: io.NewTestHandler(), } - b, err := c.getBundle("mybundleurl") + b, err := c.getBundle(safeurl.NewImmutableSafeURL("mybundleurl")) require.NoError(t, err) require.Equal(t, "application/vnd.dev.sigstore.bundle.v0.3+json", b.GetMediaType()) mockHTTPClient.AssertNumberOfCalls(t, "OnGetFailAfterNCalls", 2) @@ -294,7 +295,7 @@ func TestGetBundle_PermanentBackoffFail(t *testing.T) { logger: io.NewTestHandler(), } - b, err := c.getBundle("mybundleurl") + b, err := c.getBundle(safeurl.NewImmutableSafeURL("mybundleurl")) // var permanent *backoff.PermanentError //require.IsType(t, &backoff.PermanentError{}, err) require.Error(t, err) @@ -311,7 +312,7 @@ func TestGetBundle_RequestFail(t *testing.T) { logger: io.NewTestHandler(), } - b, err := c.getBundle("mybundleurl") + b, err := c.getBundle(safeurl.NewImmutableSafeURL("mybundleurl")) require.Error(t, err) require.Nil(t, b) mockHTTPClient.AssertNumberOfCalls(t, "OnGetReqFail", 4) diff --git a/pkg/cmd/auth/shared/login_flow.go b/pkg/cmd/auth/shared/login_flow.go index cd018430d49..c76dc5fb84c 100644 --- a/pkg/cmd/auth/shared/login_flow.go +++ b/pkg/cmd/auth/shared/login_flow.go @@ -14,6 +14,7 @@ import ( "github.com/cli/cli/v2/internal/authflow" "github.com/cli/cli/v2/internal/browser" "github.com/cli/cli/v2/internal/ghinstance" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/ssh-key/add" "github.com/cli/cli/v2/pkg/iostreams" "github.com/cli/cli/v2/pkg/ssh" @@ -258,8 +259,11 @@ func GetCurrentLogin(httpClient httpClient, hostname, authToken string) (string, result := struct { Data struct{ Viewer struct{ Login string } } }{} - apiEndpoint := ghinstance.GraphQLEndpoint(hostname) - req, err := http.NewRequest("POST", apiEndpoint, bytes.NewBuffer(reqBody)) + apiEndpoint, err := safeurl.JoinPathWithHostPrefix(ghinstance.GraphQLEndpoint(hostname)) + if err != nil { + return "", err + } + req, err := http.NewRequest("POST", apiEndpoint.String(), bytes.NewBuffer(reqBody)) if err != nil { return "", err } diff --git a/pkg/cmd/auth/shared/oauth_scopes.go b/pkg/cmd/auth/shared/oauth_scopes.go index 8d9996019b8..bc5e611163a 100644 --- a/pkg/cmd/auth/shared/oauth_scopes.go +++ b/pkg/cmd/auth/shared/oauth_scopes.go @@ -8,6 +8,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" + "github.com/cli/cli/v2/internal/safeurl" ) type MissingScopesError struct { @@ -33,9 +34,12 @@ type httpClient interface { // GetScopes performs a GitHub API request and returns the value of the X-Oauth-Scopes header. func GetScopes(httpClient httpClient, hostname, authToken string) (string, error) { - apiEndpoint := ghinstance.RESTPrefix(hostname) + apiEndpoint, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(hostname)) + if err != nil { + return "", err + } - req, err := http.NewRequest("GET", apiEndpoint, nil) + req, err := http.NewRequest("GET", apiEndpoint.String(), nil) if err != nil { return "", err } diff --git a/pkg/cmd/cache/delete/delete.go b/pkg/cmd/cache/delete/delete.go index 9125b5741ca..6bf28f76419 100644 --- a/pkg/cmd/cache/delete/delete.go +++ b/pkg/cmd/cache/delete/delete.go @@ -4,12 +4,12 @@ import ( "errors" "fmt" "net/http" - "net/url" "strconv" "github.com/MakeNowJust/heredoc" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/text" "github.com/cli/cli/v2/pkg/cmd/cache/shared" "github.com/cli/cli/v2/pkg/cmdutil" @@ -203,8 +203,11 @@ func deleteCaches(opts *DeleteOptions, client *api.Client, repo ghrepo.Interface func deleteCacheByID(client *api.Client, repo ghrepo.Interface, id int64) error { // returns HTTP 204 (NO CONTENT) on success - path := fmt.Sprintf("repos/%s/actions/caches/%d", ghrepo.FullName(repo), id) - return client.REST(repo.RepoHost(), "DELETE", path, nil, nil) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "caches", strconv.FormatInt(id, 10)) + if err != nil { + return err + } + return client.REST(repo.RepoHost(), "DELETE", path.String(), nil, nil) } // deleteCacheByKey deletes cache entries by given key (and optional ref) and @@ -214,12 +217,16 @@ func deleteCacheByID(client *api.Client, repo ghrepo.Interface, id int64) error // entry. There may be more than one entries with the same key/ref combination, // but those entries will have different IDs. func deleteCacheByKey(client *api.Client, repo ghrepo.Interface, key, ref string) (int, error) { - path := fmt.Sprintf("repos/%s/actions/caches?key=%s", ghrepo.FullName(repo), url.QueryEscape(key)) + u, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "caches") + if err != nil { + return 0, err + } + u.SetQuery("key", key) if ref != "" { - path += fmt.Sprintf("&ref=%s", url.QueryEscape(ref)) + u.SetQuery("ref", ref) } var payload shared.CachePayload - err := client.REST(repo.RepoHost(), "DELETE", path, nil, &payload) + err = client.REST(repo.RepoHost(), "DELETE", u.String(), nil, &payload) if err != nil { return 0, err } diff --git a/pkg/cmd/cache/shared/shared.go b/pkg/cmd/cache/shared/shared.go index a853b143f99..5d7a4996f13 100644 --- a/pkg/cmd/cache/shared/shared.go +++ b/pkg/cmd/cache/shared/shared.go @@ -1,12 +1,12 @@ package shared import ( - "fmt" - "net/url" + "strconv" "time" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmdutil" ) @@ -46,36 +46,40 @@ type GetCachesOptions struct { // Return a list of caches for a repository. Pass a negative limit to request // all pages from the API until all caches have been fetched. func GetCaches(client *api.Client, repo ghrepo.Interface, opts GetCachesOptions) (*CachePayload, error) { - path := fmt.Sprintf("repos/%s/actions/caches", ghrepo.FullName(repo)) + u, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "caches") + if err != nil { + return nil, err + } perPage := 100 if opts.Limit > 0 && opts.Limit < 100 { perPage = opts.Limit } - path += fmt.Sprintf("?per_page=%d", perPage) + u.SetQuery("per_page", strconv.Itoa(perPage)) if opts.Sort != "" { - path += fmt.Sprintf("&sort=%s", opts.Sort) + u.SetQuery("sort", opts.Sort) } if opts.Order != "" { - path += fmt.Sprintf("&direction=%s", opts.Order) + u.SetQuery("direction", opts.Order) } if opts.Key != "" { - path += fmt.Sprintf("&key=%s", url.QueryEscape(opts.Key)) + u.SetQuery("key", opts.Key) } if opts.Ref != "" { - path += fmt.Sprintf("&ref=%s", url.QueryEscape(opts.Ref)) + u.SetQuery("ref", opts.Ref) } + var pageURL safeurl.SafeURL = u var result *CachePayload pagination: - for path != "" { + for pageURL.String() != "" { var response CachePayload - var err error - path, err = client.RESTWithNext(repo.RepoHost(), "GET", path, nil, &response) + next, err := client.RESTWithNext(repo.RepoHost(), "GET", pageURL.String(), nil, &response) if err != nil { return nil, err } + pageURL = safeurl.NewImmutableSafeURL(next) if result == nil { result = &response diff --git a/pkg/cmd/codespace/common.go b/pkg/cmd/codespace/common.go index 45815939b1c..2f1e0594700 100644 --- a/pkg/cmd/codespace/common.go +++ b/pkg/cmd/codespace/common.go @@ -18,6 +18,7 @@ import ( clicontext "github.com/cli/cli/v2/context" "github.com/cli/cli/v2/internal/browser" "github.com/cli/cli/v2/internal/codespaces/api" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/iostreams" "github.com/spf13/cobra" "golang.org/x/term" @@ -251,6 +252,12 @@ func addDeprecatedRepoShorthand(cmd *cobra.Command, target *string) error { return nil } +// validateNWO returns an error if nwo is not a valid "owner/repo" repository reference. +func validateNWO(nwo string) error { + _, _, err := safeurl.RepoPartsFromNWO(nwo) + return err +} + // filterCodespacesByRepoOwner filters a list of codespaces by the owner of the repository. func filterCodespacesByRepoOwner(codespaces []*api.Codespace, repoOwner string) []*api.Codespace { filtered := make([]*api.Codespace, 0, len(codespaces)) diff --git a/pkg/cmd/codespace/create.go b/pkg/cmd/codespace/create.go index 734509a3020..fce1901fe2d 100644 --- a/pkg/cmd/codespace/create.go +++ b/pkg/cmd/codespace/create.go @@ -88,6 +88,11 @@ func newCreateCmd(app *App) *cobra.Command { Short: "Create a codespace", Args: noArgsConstraint, PreRunE: func(cmd *cobra.Command, args []string) error { + if opts.repo != "" { + if err := validateNWO(opts.repo); err != nil { + return cmdutil.FlagErrorf("invalid value for --repo: %v", err) + } + } return cmdutil.MutuallyExclusive( "using --web with --display-name, --idle-timeout, or --retention-period is not supported", opts.useWeb, diff --git a/pkg/cmd/codespace/create_test.go b/pkg/cmd/codespace/create_test.go index 9a959f5dae2..8579069db8c 100644 --- a/pkg/cmd/codespace/create_test.go +++ b/pkg/cmd/codespace/create_test.go @@ -32,6 +32,11 @@ func TestCreateCmdFlagError(t *testing.T) { args: "--web --idle-timeout 30m", wantsErr: fmt.Errorf("using --web with --display-name, --idle-timeout, or --retention-period is not supported"), }, + { + name: "return error when --repo is not in owner/repo format", + args: "--repo foo", + wantsErr: fmt.Errorf(`invalid value for --repo: expected the "OWNER/REPO" format, got "foo"`), + }, } for _, tt := range tests { diff --git a/pkg/cmd/codespace/list.go b/pkg/cmd/codespace/list.go index 238c4a6a74b..ba003fd115f 100644 --- a/pkg/cmd/codespace/list.go +++ b/pkg/cmd/codespace/list.go @@ -35,6 +35,12 @@ func newListCmd(app *App) *cobra.Command { Aliases: []string{"ls"}, Args: noArgsConstraint, PreRunE: func(cmd *cobra.Command, args []string) error { + if opts.repo != "" { + if err := validateNWO(opts.repo); err != nil { + return cmdutil.FlagErrorf("invalid value for --repo: %v", err) + } + } + if err := cmdutil.MutuallyExclusive( "using `--org` or `--user` with `--repo` is not allowed", opts.repo != "", diff --git a/pkg/cmd/codespace/list_test.go b/pkg/cmd/codespace/list_test.go index 49bb0b4d29f..8ceadd449c8 100644 --- a/pkg/cmd/codespace/list_test.go +++ b/pkg/cmd/codespace/list_test.go @@ -35,6 +35,11 @@ func TestListCmdFlagError(t *testing.T) { args: "--limit -1", wantsErr: fmt.Errorf("invalid limit: -1"), }, + { + name: "list codespaces, --repo not in owner/repo format", + args: "--repo foo", + wantsErr: fmt.Errorf(`invalid value for --repo: expected the "OWNER/REPO" format, got "foo"`), + }, } for _, tt := range tests { diff --git a/pkg/cmd/copilot/copilot.go b/pkg/cmd/copilot/copilot.go index cc83ef48efa..ede7db92650 100644 --- a/pkg/cmd/copilot/copilot.go +++ b/pkg/cmd/copilot/copilot.go @@ -23,6 +23,7 @@ import ( "github.com/cli/cli/v2/internal/gh/ghtelemetry" "github.com/cli/cli/v2/internal/prompter" "github.com/cli/cli/v2/internal/safepaths" + "github.com/cli/cli/v2/internal/safeurl" ghzip "github.com/cli/cli/v2/internal/zip" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -250,32 +251,37 @@ func downloadCopilot(httpClient *http.Client, ios *iostreams.IOStreams, installD return "", fmt.Errorf("unsupported architecture: %s (supported: x64, arm64)", arch) } - var archiveURL string var archiveName string var isZip bool switch platform { case "win32": archiveName = fmt.Sprintf("copilot-%s-%s.zip", platform, arch) - archiveURL = fmt.Sprintf("https://github.com/github/copilot-cli/releases/latest/download/%s", archiveName) isZip = true case "linux", "darwin": archiveName = fmt.Sprintf("copilot-%s-%s.tar.gz", platform, arch) - archiveURL = fmt.Sprintf("https://github.com/github/copilot-cli/releases/latest/download/%s", archiveName) default: return "", fmt.Errorf("unsupported platform: %s (supported: linux, darwin, windows)", platform) } - checksumsURL := "https://github.com/github/copilot-cli/releases/latest/download/SHA256SUMS.txt" + archiveURL, err := safeurl.JoinPathWithHostPrefix("https://github.com/", "github", "copilot-cli", "releases", "latest", "download", archiveName) + if err != nil { + return "", err + } + + checksumsURL, err := safeurl.JoinPathWithHostPrefix("https://github.com/", "github", "copilot-cli", "releases", "latest", "download", "SHA256SUMS.txt") + if err != nil { + return "", err + } expectedChecksum, err := fetchExpectedChecksum(httpClient, checksumsURL, archiveName) if err != nil { return "", fmt.Errorf("failed to fetch checksums: %w", err) } - ios.StartProgressIndicatorWithLabel(fmt.Sprintf("Downloading Copilot CLI from %s", archiveURL)) + ios.StartProgressIndicatorWithLabel(fmt.Sprintf("Downloading Copilot CLI from %s", archiveURL.String())) defer ios.StopProgressIndicator() - resp, err := httpClient.Get(archiveURL) + resp, err := httpClient.Get(archiveURL.String()) if err != nil { return "", fmt.Errorf("failed to download: %w", err) } @@ -333,8 +339,8 @@ func downloadCopilot(httpClient *http.Client, ios *iostreams.IOStreams, installD } // fetchExpectedChecksum downloads the SHA256SUMS.txt file and returns the expected checksum for the given archive name. -func fetchExpectedChecksum(httpClient *http.Client, checksumsURL, archiveName string) (string, error) { - resp, err := httpClient.Get(checksumsURL) +func fetchExpectedChecksum(httpClient *http.Client, checksumsURL safeurl.SafeURL, archiveName string) (string, error) { + resp, err := httpClient.Get(checksumsURL.String()) if err != nil { return "", err } diff --git a/pkg/cmd/copilot/copilot_test.go b/pkg/cmd/copilot/copilot_test.go index fa173f5286c..58792ef7c81 100644 --- a/pkg/cmd/copilot/copilot_test.go +++ b/pkg/cmd/copilot/copilot_test.go @@ -16,6 +16,7 @@ import ( "testing" "github.com/cli/cli/v2/internal/gh/ghtelemetry" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/telemetry" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/httpmock" @@ -341,7 +342,7 @@ func TestFetchExpectedChecksum(t *testing.T) { ) client := &http.Client{Transport: reg} - checksum, err := fetchExpectedChecksum(client, "https://example.com/checksums", "copilot-linux-x64.tar.gz") + checksum, err := fetchExpectedChecksum(client, safeurl.NewImmutableSafeURL("https://example.com/checksums"), "copilot-linux-x64.tar.gz") require.NoError(t, err, "unexpected error") require.Equal(t, "abc123def456", checksum, "checksum mismatch") }) @@ -355,7 +356,7 @@ func TestFetchExpectedChecksum(t *testing.T) { ) client := &http.Client{Transport: reg} - _, err := fetchExpectedChecksum(client, "https://example.com/checksums", "copilot-win32-x64.zip") + _, err := fetchExpectedChecksum(client, safeurl.NewImmutableSafeURL("https://example.com/checksums"), "copilot-win32-x64.zip") require.Error(t, err, "expected error for missing archive") require.Equal(t, "checksum not found for copilot-win32-x64.zip", err.Error(), "unexpected error") }) @@ -369,7 +370,7 @@ func TestFetchExpectedChecksum(t *testing.T) { ) client := &http.Client{Transport: reg} - checksum, err := fetchExpectedChecksum(client, "https://example.com/checksums", "copilot-darwin-x64.tar.gz") + checksum, err := fetchExpectedChecksum(client, safeurl.NewImmutableSafeURL("https://example.com/checksums"), "copilot-darwin-x64.tar.gz") require.NoError(t, err, "unexpected error") require.Equal(t, "abc123", checksum, "checksum mismatch") }) @@ -382,7 +383,7 @@ func TestFetchExpectedChecksum(t *testing.T) { ) client := &http.Client{Transport: reg} - _, err := fetchExpectedChecksum(client, "https://example.com/checksums", "copilot-linux-x64.tar.gz") + _, err := fetchExpectedChecksum(client, safeurl.NewImmutableSafeURL("https://example.com/checksums"), "copilot-linux-x64.tar.gz") require.Error(t, err, "expected error for HTTP 404") }) } diff --git a/pkg/cmd/extension/http.go b/pkg/cmd/extension/http.go index 90ccd64ceba..4ff8fa65cfb 100644 --- a/pkg/cmd/extension/http.go +++ b/pkg/cmd/extension/http.go @@ -3,7 +3,6 @@ package extension import ( "encoding/json" "errors" - "fmt" "io" "net/http" "os" @@ -11,11 +10,15 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" ) func repoExists(httpClient *http.Client, repo ghrepo.Interface) (bool, error) { - url := fmt.Sprintf("%srepos/%s/%s", ghinstance.RESTPrefix(repo.RepoHost()), repo.RepoOwner(), repo.RepoName()) - req, err := http.NewRequest("GET", url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName()) + if err != nil { + return false, err + } + req, err := http.NewRequest("GET", url.String(), nil) if err != nil { return false, err } @@ -37,10 +40,11 @@ func repoExists(httpClient *http.Client, repo ghrepo.Interface) (bool, error) { } func hasScript(httpClient *http.Client, repo ghrepo.Interface) (bool, error) { - path := fmt.Sprintf("repos/%s/%s/contents/%s", - repo.RepoOwner(), repo.RepoName(), repo.RepoName()) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path - req, err := http.NewRequest("GET", url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "contents", repo.RepoName()) + if err != nil { + return false, err + } + req, err := http.NewRequest("GET", url.String(), nil) if err != nil { return false, err } @@ -74,9 +78,9 @@ type release struct { } // downloadAsset downloads a single asset to the given file path. -func downloadAsset(httpClient *http.Client, asset releaseAsset, destPath string) (downloadErr error) { +func downloadAsset(httpClient *http.Client, assetURL safeurl.SafeURL, destPath string) (downloadErr error) { var req *http.Request - if req, downloadErr = http.NewRequest("GET", asset.APIURL, nil); downloadErr != nil { + if req, downloadErr = http.NewRequest("GET", assetURL.String(), nil); downloadErr != nil { return } @@ -113,9 +117,11 @@ var repositoryNotFoundErr = errors.New("repository not found") // fetchLatestRelease finds the latest published release for a repository. func fetchLatestRelease(httpClient *http.Client, baseRepo ghrepo.Interface) (*release, error) { - path := fmt.Sprintf("repos/%s/%s/releases/latest", baseRepo.RepoOwner(), baseRepo.RepoName()) - url := ghinstance.RESTPrefix(baseRepo.RepoHost()) + path - req, err := http.NewRequest("GET", url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(baseRepo.RepoHost()), "repos", baseRepo.RepoOwner(), baseRepo.RepoName(), "releases", "latest") + if err != nil { + return nil, err + } + req, err := http.NewRequest("GET", url.String(), nil) if err != nil { return nil, err } @@ -149,10 +155,11 @@ func fetchLatestRelease(httpClient *http.Client, baseRepo ghrepo.Interface) (*re // fetchReleaseFromTag finds release by tag name for a repository func fetchReleaseFromTag(httpClient *http.Client, baseRepo ghrepo.Interface, tagName string) (*release, error) { - fullRepoName := fmt.Sprintf("%s/%s", baseRepo.RepoOwner(), baseRepo.RepoName()) - path := fmt.Sprintf("repos/%s/releases/tags/%s", fullRepoName, tagName) - url := ghinstance.RESTPrefix(baseRepo.RepoHost()) + path - req, err := http.NewRequest("GET", url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(baseRepo.RepoHost()), "repos", baseRepo.RepoOwner(), baseRepo.RepoName(), "releases", "tags", tagName) + if err != nil { + return nil, err + } + req, err := http.NewRequest("GET", url.String(), nil) if err != nil { return nil, err } @@ -186,9 +193,11 @@ func fetchReleaseFromTag(httpClient *http.Client, baseRepo ghrepo.Interface, tag // fetchCommitSHA finds full commit SHA from a target ref in a repo func fetchCommitSHA(httpClient *http.Client, baseRepo ghrepo.Interface, targetRef string) (string, error) { - path := fmt.Sprintf("repos/%s/%s/commits/%s", baseRepo.RepoOwner(), baseRepo.RepoName(), targetRef) - url := ghinstance.RESTPrefix(baseRepo.RepoHost()) + path - req, err := http.NewRequest("GET", url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(baseRepo.RepoHost()), "repos", baseRepo.RepoOwner(), baseRepo.RepoName(), "commits", targetRef) + if err != nil { + return "", err + } + req, err := http.NewRequest("GET", url.String(), nil) if err != nil { return "", err } diff --git a/pkg/cmd/extension/manager.go b/pkg/cmd/extension/manager.go index de758f5a3d2..f1528743a39 100644 --- a/pkg/cmd/extension/manager.go +++ b/pkg/cmd/extension/manager.go @@ -20,6 +20,7 @@ import ( "github.com/cli/cli/v2/internal/config" "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/extensions" "github.com/cli/cli/v2/pkg/findsh" "github.com/cli/cli/v2/pkg/iostreams" @@ -346,7 +347,7 @@ func (m *Manager) installBin(repo ghrepo.Interface, target string) error { binPath := filepath.Join(targetDir, name) binPath += ext - err = downloadAsset(m.client, *asset, binPath) + err = downloadAsset(m.client, safeurl.NewImmutableSafeURL(asset.APIURL), binPath) if err != nil { return fmt.Errorf("failed to download asset %s: %w", asset.Name, err) } diff --git a/pkg/cmd/gist/create/create.go b/pkg/cmd/gist/create/create.go index 06b2336b663..6ed4f66263d 100644 --- a/pkg/cmd/gist/create/create.go +++ b/pkg/cmd/gist/create/create.go @@ -18,6 +18,7 @@ import ( "github.com/cli/cli/v2/internal/browser" "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghinstance" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/text" "github.com/cli/cli/v2/pkg/cmd/gist/shared" "github.com/cli/cli/v2/pkg/cmdutil" @@ -272,8 +273,11 @@ func createGist(client *http.Client, hostname, description string, public bool, return nil, err } - u := ghinstance.RESTPrefix(hostname) + "gists" - req, err := http.NewRequest(http.MethodPost, u, requestBody) + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(hostname), "gists") + if err != nil { + return nil, err + } + req, err := http.NewRequest(http.MethodPost, u.String(), requestBody) if err != nil { return nil, err } diff --git a/pkg/cmd/gist/delete/delete.go b/pkg/cmd/gist/delete/delete.go index 319f9265e99..4413bc8bffb 100644 --- a/pkg/cmd/gist/delete/delete.go +++ b/pkg/cmd/gist/delete/delete.go @@ -10,6 +10,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/prompter" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/gist/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -142,8 +143,11 @@ func deleteRun(opts *DeleteOptions) error { } func deleteGist(apiClient *api.Client, hostname string, gistID string) error { - path := "gists/" + gistID - err := apiClient.REST(hostname, "DELETE", path, nil, nil) + path, err := safeurl.JoinPath("gists", gistID) + if err != nil { + return err + } + err = apiClient.REST(hostname, "DELETE", path.String(), nil, nil) if err != nil { var httpErr api.HTTPError if errors.As(err, &httpErr) && httpErr.StatusCode == 404 { diff --git a/pkg/cmd/gist/edit/edit.go b/pkg/cmd/gist/edit/edit.go index d52aaac1848..8195c6aa0f9 100644 --- a/pkg/cmd/gist/edit/edit.go +++ b/pkg/cmd/gist/edit/edit.go @@ -16,6 +16,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/prompter" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/gist/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -287,7 +288,7 @@ func editRun(opts *EditOptions) error { file := gist.Files[filename] if file.Truncated { if _, alreadyEdited := filesToUpdate[filename]; !alreadyEdited { - fullContent, err := shared.GetRawGistFile(client, file.RawURL) + fullContent, err := shared.GetRawGistFile(client, safeurl.NewImmutableSafeURL(file.RawURL)) if err != nil { return err } @@ -404,8 +405,11 @@ func updateGist(apiClient *api.Client, hostname string, gist gistToUpdate) error requestBody := bytes.NewReader(requestByte) result := shared.Gist{} - path := "gists/" + gist.id - err = apiClient.REST(hostname, "POST", path, requestBody, &result) + path, err := safeurl.JoinPath("gists", gist.id) + if err != nil { + return err + } + err = apiClient.REST(hostname, "POST", path.String(), requestBody, &result) if err != nil { return err } diff --git a/pkg/cmd/gist/rename/rename.go b/pkg/cmd/gist/rename/rename.go index 96f630c025d..d5ef2d9199c 100644 --- a/pkg/cmd/gist/rename/rename.go +++ b/pkg/cmd/gist/rename/rename.go @@ -11,6 +11,7 @@ import ( "github.com/MakeNowJust/heredoc" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/gh" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/gist/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -119,7 +120,10 @@ func updateGist(apiClient *api.Client, hostname string, gist *shared.Gist) error Files: gist.Files, } - path := "gists/" + gist.ID + path, err := safeurl.JoinPath("gists", gist.ID) + if err != nil { + return err + } requestByte, err := json.Marshal(body) if err != nil { @@ -130,7 +134,7 @@ func updateGist(apiClient *api.Client, hostname string, gist *shared.Gist) error result := shared.Gist{} - err = apiClient.REST(hostname, "POST", path, requestBody, &result) + err = apiClient.REST(hostname, "POST", path.String(), requestBody, &result) if err != nil { return err diff --git a/pkg/cmd/gist/shared/shared.go b/pkg/cmd/gist/shared/shared.go index 61f09af7c57..7c0a7c07565 100644 --- a/pkg/cmd/gist/shared/shared.go +++ b/pkg/cmd/gist/shared/shared.go @@ -13,6 +13,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/prompter" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/text" "github.com/cli/cli/v2/pkg/iostreams" "github.com/gabriel-vasile/mimetype" @@ -62,10 +63,13 @@ var NotFoundErr = errors.New("not found") func GetGist(client *http.Client, hostname, gistID string) (*Gist, error) { gist := Gist{} - path := fmt.Sprintf("gists/%s", gistID) + path, err := safeurl.JoinPath("gists", gistID) + if err != nil { + return nil, err + } apiClient := api.NewClientFromHTTP(client) - err := apiClient.REST(hostname, "GET", path, nil, &gist) + err = apiClient.REST(hostname, "GET", path.String(), nil, &gist) if err != nil { var httpErr api.HTTPError if errors.As(err, &httpErr) && httpErr.StatusCode == 404 { @@ -251,8 +255,8 @@ func PromptGists(prompter prompter.Prompter, client *http.Client, host string, c // GetRawGistFile fetches the full content of a gist file from its raw URL. The // bytes are external content, so they are returned as iostreams.Untrusted to // force callers to choose between sanitized display and raw round-tripping. -func GetRawGistFile(httpClient *http.Client, rawURL string) (iostreams.Untrusted, error) { - req, err := http.NewRequest("GET", rawURL, nil) +func GetRawGistFile(httpClient *http.Client, rawURL safeurl.SafeURL) (iostreams.Untrusted, error) { + req, err := http.NewRequest("GET", rawURL.String(), nil) if err != nil { return iostreams.Untrusted{}, err } diff --git a/pkg/cmd/gist/shared/shared_test.go b/pkg/cmd/gist/shared/shared_test.go index 28587c1409b..8b891d8b9fe 100644 --- a/pkg/cmd/gist/shared/shared_test.go +++ b/pkg/cmd/gist/shared/shared_test.go @@ -7,6 +7,7 @@ import ( "time" "github.com/cli/cli/v2/internal/prompter" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/httpmock" "github.com/cli/cli/v2/pkg/iostreams" "github.com/stretchr/testify/assert" @@ -298,7 +299,7 @@ func TestGetRawGistFile(t *testing.T) { ) client := &http.Client{Transport: reg} - result, err := GetRawGistFile(client, "https://gist.githubusercontent.com/raw-url") + result, err := GetRawGistFile(client, safeurl.NewImmutableSafeURL("https://gist.githubusercontent.com/raw-url")) if tt.wantErr { assert.Error(t, err) diff --git a/pkg/cmd/gist/view/view.go b/pkg/cmd/gist/view/view.go index 0cff1b5d5b1..51ec38d0443 100644 --- a/pkg/cmd/gist/view/view.go +++ b/pkg/cmd/gist/view/view.go @@ -10,6 +10,7 @@ import ( "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/prompter" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/text" "github.com/cli/cli/v2/pkg/cmd/gist/shared" "github.com/cli/cli/v2/pkg/cmdutil" @@ -149,7 +150,7 @@ func viewRun(opts *ViewOptions) error { // path fetches the full content from the raw URL. content := iostreams.NewUntrusted(gf.Content) if gf.Truncated { - fullContent, err := shared.GetRawGistFile(client, gf.RawURL) + fullContent, err := shared.GetRawGistFile(client, safeurl.NewImmutableSafeURL(gf.RawURL)) if err != nil { return err } diff --git a/pkg/cmd/gpg-key/add/http.go b/pkg/cmd/gpg-key/add/http.go index 4b2a6e97c4f..b1f0fca74cd 100644 --- a/pkg/cmd/gpg-key/add/http.go +++ b/pkg/cmd/gpg-key/add/http.go @@ -9,6 +9,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" + "github.com/cli/cli/v2/internal/safeurl" ) var errScopesMissing = errors.New("insufficient OAuth scopes") @@ -16,7 +17,10 @@ var errDuplicateKey = errors.New("key already exists") var errWrongFormat = errors.New("key in wrong format") func gpgKeyUpload(httpClient *http.Client, hostname string, keyFile io.Reader, title string) error { - url := ghinstance.RESTPrefix(hostname) + "user/gpg_keys" + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(hostname), "user", "gpg_keys") + if err != nil { + return err + } keyBytes, err := io.ReadAll(keyFile) if err != nil { @@ -35,7 +39,7 @@ func gpgKeyUpload(httpClient *http.Client, hostname string, keyFile io.Reader, t return err } - req, err := http.NewRequest("POST", url, bytes.NewBuffer(payloadBytes)) + req, err := http.NewRequest("POST", u.String(), bytes.NewBuffer(payloadBytes)) if err != nil { return err } diff --git a/pkg/cmd/gpg-key/delete/http.go b/pkg/cmd/gpg-key/delete/http.go index 22b43133b86..9b6c2a46eae 100644 --- a/pkg/cmd/gpg-key/delete/http.go +++ b/pkg/cmd/gpg-key/delete/http.go @@ -2,12 +2,12 @@ package delete import ( "encoding/json" - "fmt" "io" "net/http" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" + "github.com/cli/cli/v2/internal/safeurl" ) type gpgKey struct { @@ -16,8 +16,11 @@ type gpgKey struct { } func deleteGPGKey(httpClient *http.Client, host, id string) error { - url := fmt.Sprintf("%suser/gpg_keys/%s", ghinstance.RESTPrefix(host), id) - req, err := http.NewRequest("DELETE", url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(host), "user", "gpg_keys", id) + if err != nil { + return err + } + req, err := http.NewRequest("DELETE", url.String(), nil) if err != nil { return err } @@ -36,9 +39,12 @@ func deleteGPGKey(httpClient *http.Client, host, id string) error { } func getGPGKeys(httpClient *http.Client, host string) ([]gpgKey, error) { - resource := "user/gpg_keys" - url := fmt.Sprintf("%s%s?per_page=%d", ghinstance.RESTPrefix(host), resource, 100) - req, err := http.NewRequest("GET", url, nil) + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(host), "user", "gpg_keys") + if err != nil { + return nil, err + } + u.SetQuery("per_page", "100") + req, err := http.NewRequest("GET", u.String(), nil) if err != nil { return nil, err } diff --git a/pkg/cmd/gpg-key/list/http.go b/pkg/cmd/gpg-key/list/http.go index 804119f355e..1b00684590e 100644 --- a/pkg/cmd/gpg-key/list/http.go +++ b/pkg/cmd/gpg-key/list/http.go @@ -3,7 +3,6 @@ package list import ( "encoding/json" "errors" - "fmt" "io" "net/http" "strings" @@ -11,6 +10,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" + "github.com/cli/cli/v2/internal/safeurl" ) var errScopes = errors.New("insufficient OAuth scopes") @@ -38,12 +38,18 @@ type gpgKey struct { } func userKeys(httpClient *http.Client, host, userHandle string) ([]gpgKey, error) { - resource := "user/gpg_keys" + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(host), "user", "gpg_keys") + if err != nil { + return nil, err + } if userHandle != "" { - resource = fmt.Sprintf("users/%s/gpg_keys", userHandle) + u, err = safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(host), "users", userHandle, "gpg_keys") + if err != nil { + return nil, err + } } - url := fmt.Sprintf("%s%s?per_page=%d", ghinstance.RESTPrefix(host), resource, 100) - req, err := http.NewRequest("GET", url, nil) + u.SetQuery("per_page", "100") + req, err := http.NewRequest("GET", u.String(), nil) if err != nil { return nil, err } diff --git a/pkg/cmd/label/create.go b/pkg/cmd/label/create.go index 9d6b2e9ee86..58954372989 100644 --- a/pkg/cmd/label/create.go +++ b/pkg/cmd/label/create.go @@ -13,6 +13,7 @@ import ( "github.com/MakeNowJust/heredoc" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" "github.com/spf13/cobra" @@ -127,7 +128,10 @@ func createRun(opts *createOptions) error { func createLabel(client *http.Client, repo ghrepo.Interface, opts *createOptions) error { apiClient := api.NewClientFromHTTP(client) - path := fmt.Sprintf("repos/%s/%s/labels", repo.RepoOwner(), repo.RepoName()) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "labels") + if err != nil { + return err + } requestByte, err := json.Marshal(map[string]string{ "name": opts.Name, "description": opts.Description, @@ -137,7 +141,7 @@ func createLabel(client *http.Client, repo ghrepo.Interface, opts *createOptions return err } requestBody := bytes.NewReader(requestByte) - err = apiClient.REST(repo.RepoHost(), "POST", path, requestBody, nil) + err = apiClient.REST(repo.RepoHost(), "POST", path.String(), requestBody, nil) if httpError, ok := err.(api.HTTPError); ok && isLabelAlreadyExistsError(httpError) { err = errLabelAlreadyExists @@ -156,7 +160,10 @@ func createLabel(client *http.Client, repo ghrepo.Interface, opts *createOptions } func updateLabel(apiClient *api.Client, repo ghrepo.Interface, opts *editOptions) error { - path := fmt.Sprintf("repos/%s/%s/labels/%s", repo.RepoOwner(), repo.RepoName(), opts.Name) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "labels", opts.Name) + if err != nil { + return err + } properties := map[string]string{} if opts.Description != "" { properties["description"] = opts.Description @@ -172,7 +179,7 @@ func updateLabel(apiClient *api.Client, repo ghrepo.Interface, opts *editOptions return err } requestBody := bytes.NewReader(requestByte) - err = apiClient.REST(repo.RepoHost(), "PATCH", path, requestBody, nil) + err = apiClient.REST(repo.RepoHost(), "PATCH", path.String(), requestBody, nil) if httpError, ok := err.(api.HTTPError); ok && isLabelAlreadyExistsError(httpError) { err = errLabelAlreadyExists diff --git a/pkg/cmd/label/delete.go b/pkg/cmd/label/delete.go index c9d8f4caea8..8dd1532c127 100644 --- a/pkg/cmd/label/delete.go +++ b/pkg/cmd/label/delete.go @@ -6,6 +6,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" "github.com/spf13/cobra" @@ -94,7 +95,10 @@ func deleteRun(opts *deleteOptions) error { func deleteLabel(client *http.Client, repo ghrepo.Interface, name string) error { apiClient := api.NewClientFromHTTP(client) - path := fmt.Sprintf("repos/%s/%s/labels/%s", repo.RepoOwner(), repo.RepoName(), name) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "labels", name) + if err != nil { + return err + } - return apiClient.REST(repo.RepoHost(), "DELETE", path, nil, nil) + return apiClient.REST(repo.RepoHost(), "DELETE", path.String(), nil, nil) } diff --git a/pkg/cmd/pr/close/close_test.go b/pkg/cmd/pr/close/close_test.go index 57ee0f0e643..17214915779 100644 --- a/pkg/cmd/pr/close/close_test.go +++ b/pkg/cmd/pr/close/close_test.go @@ -157,7 +157,7 @@ func TestPrClose_deleteBranch_sameRepo(t *testing.T) { }), ) http.Register( - httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads/blueberries"), + httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads%2Fblueberries"), httpmock.StringResponse(`{}`)) cs, cmdTeardown := run.Stub() @@ -223,7 +223,7 @@ func TestPrClose_deleteBranch_sameBranch(t *testing.T) { }), ) http.Register( - httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads/trunk"), + httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads%2Ftrunk"), httpmock.StringResponse(`{}`)) cs, cmdTeardown := run.Stub() @@ -258,7 +258,7 @@ func TestPrClose_deleteBranch_notInGitRepo(t *testing.T) { }), ) http.Register( - httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads/trunk"), + httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads%2Ftrunk"), httpmock.StringResponse(`{}`)) cs, cmdTeardown := run.Stub() diff --git a/pkg/cmd/pr/diff/diff.go b/pkg/cmd/pr/diff/diff.go index 7555442a8ed..3e627b8b806 100644 --- a/pkg/cmd/pr/diff/diff.go +++ b/pkg/cmd/pr/diff/diff.go @@ -9,6 +9,7 @@ import ( "net/http" "path" "regexp" + "strconv" "strings" "github.com/MakeNowJust/heredoc" @@ -16,6 +17,7 @@ import ( "github.com/cli/cli/v2/internal/browser" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/text" "github.com/cli/cli/v2/pkg/cmd/pr/shared" "github.com/cli/cli/v2/pkg/cmdutil" @@ -209,18 +211,16 @@ func diffRun(opts *DiffOptions) error { } func fetchDiff(httpClient *http.Client, baseRepo ghrepo.Interface, prNumber int, asPatch bool) (io.ReadCloser, error) { - url := fmt.Sprintf( - "%srepos/%s/pulls/%d", - ghinstance.RESTPrefix(baseRepo.RepoHost()), - ghrepo.FullName(baseRepo), - prNumber, - ) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(baseRepo.RepoHost()), "repos", baseRepo.RepoOwner(), baseRepo.RepoName(), "pulls", strconv.Itoa(prNumber)) + if err != nil { + return nil, err + } acceptType := "application/vnd.github.v3.diff" if asPatch { acceptType = "application/vnd.github.v3.patch" } - req, err := http.NewRequest("GET", url, nil) + req, err := http.NewRequest("GET", url.String(), nil) if err != nil { return nil, err } diff --git a/pkg/cmd/pr/merge/merge_test.go b/pkg/cmd/pr/merge/merge_test.go index 03ddafa61ae..d7e02aa715a 100644 --- a/pkg/cmd/pr/merge/merge_test.go +++ b/pkg/cmd/pr/merge/merge_test.go @@ -635,7 +635,7 @@ func TestPrMerge_deleteBranch(t *testing.T) { assert.NotContains(t, input, "commitHeadline") })) http.Register( - httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads/blueberries"), + httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads%2Fblueberries"), httpmock.StringResponse(`{}`)) cs, cmdTeardown := run.Stub() @@ -701,7 +701,7 @@ func TestPrMerge_deleteBranch_apiError(t *testing.T) { ✓ Merged pull request OWNER/REPO#10 (Blueberries are a good fruit) ✓ Deleted local branch blueberries and switched to branch main `), - wantErr: "failed to delete remote branch blueberries: HTTP 500: blah blah (https://api.github.com/repos/OWNER/REPO/git/refs/heads/blueberries)", + wantErr: "failed to delete remote branch blueberries: HTTP 500: blah blah (https://api.github.com/repos/OWNER/REPO/git/refs/heads%2Fblueberries)", }, } @@ -732,7 +732,7 @@ func TestPrMerge_deleteBranch_apiError(t *testing.T) { assert.NotContains(t, input, "commitHeadline") })) http.Register( - httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads/blueberries"), + httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads%2Fblueberries"), httpmock.JSONErrorResponse(tt.apiError.StatusCode, tt.apiError)) cs, cmdTeardown := run.Stub() @@ -806,7 +806,7 @@ func TestPrMerge_deleteBranch_nonDefault(t *testing.T) { assert.NotContains(t, input, "commitHeadline") })) http.Register( - httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads/blueberries"), + httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads%2Fblueberries"), httpmock.StringResponse(`{}`)) cs, cmdTeardown := run.Stub() @@ -905,7 +905,7 @@ func TestPrMerge_deleteBranch_checkoutNewBranch(t *testing.T) { assert.NotContains(t, input, "commitHeadline") })) http.Register( - httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads/blueberries"), + httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads%2Fblueberries"), httpmock.StringResponse(`{}`)) cs, cmdTeardown := run.Stub() @@ -955,7 +955,7 @@ func TestPrMerge_deleteNonCurrentBranch(t *testing.T) { assert.NotContains(t, input, "commitHeadline") })) http.Register( - httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads/blueberries"), + httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads%2Fblueberries"), httpmock.StringResponse(`{}`)) cs, cmdTeardown := run.Stub() @@ -1435,7 +1435,7 @@ func TestPRMergeTTY_withDeleteBranch(t *testing.T) { assert.NotContains(t, input, "commitHeadline") })) http.Register( - httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads/blueberries"), + httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/heads%2Fblueberries"), httpmock.StringResponse(`{}`)) cs, cmdTeardown := run.Stub() diff --git a/pkg/cmd/release/create/create.go b/pkg/cmd/release/create/create.go index 8771b2477a8..5a8d995bc08 100644 --- a/pkg/cmd/release/create/create.go +++ b/pkg/cmd/release/create/create.go @@ -13,6 +13,7 @@ import ( "github.com/cli/cli/v2/git" "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/text" "github.com/cli/cli/v2/pkg/cmd/release/shared" "github.com/cli/cli/v2/pkg/cmdutil" @@ -534,7 +535,7 @@ func createRun(opts *CreateOptions) error { if !draftWhileUploading { return err } - if cleanupErr := deleteRelease(httpClient, newRelease); cleanupErr != nil { + if cleanupErr := deleteRelease(httpClient, safeurl.NewImmutableSafeURL(newRelease.APIURL)); cleanupErr != nil { return fmt.Errorf("%w\ncleaning up draft failed: %v", err, cleanupErr) } return err @@ -547,14 +548,14 @@ func createRun(opts *CreateOptions) error { } opts.IO.StartProgressIndicator() - err = shared.ConcurrentUpload(httpClient, uploadURL, opts.Concurrency, opts.Assets) + err = shared.ConcurrentUpload(httpClient, safeurl.NewImmutableSafeURL(uploadURL), opts.Concurrency, opts.Assets) opts.IO.StopProgressIndicator() if err != nil { return cleanupDraftRelease(err) } if draftWhileUploading { - rel, err := publishRelease(httpClient, newRelease.APIURL, opts.DiscussionCategory, opts.IsLatest) + rel, err := publishRelease(httpClient, safeurl.NewImmutableSafeURL(newRelease.APIURL), opts.DiscussionCategory, opts.IsLatest) if err != nil { return cleanupDraftRelease(err) } diff --git a/pkg/cmd/release/create/http.go b/pkg/cmd/release/create/http.go index 4311a389693..abee74e8be1 100644 --- a/pkg/cmd/release/create/http.go +++ b/pkg/cmd/release/create/http.go @@ -8,13 +8,14 @@ import ( "fmt" "io" "net/http" - "net/url" "slices" + "strconv" "strings" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/release/shared" "github.com/shurcooL/githubv4" @@ -60,9 +61,12 @@ func remoteTagExists(httpClient *http.Client, repo ghrepo.Interface, tagName str } func getTags(httpClient *http.Client, repo ghrepo.Interface, limit int) ([]tag, error) { - path := fmt.Sprintf("repos/%s/%s/tags?per_page=%d", repo.RepoOwner(), repo.RepoName(), limit) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path - req, err := http.NewRequest("GET", url, nil) + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "tags") + if err != nil { + return nil, err + } + u.SetQuery("per_page", strconv.Itoa(limit)) + req, err := http.NewRequest("GET", u.String(), nil) if err != nil { return nil, err } @@ -106,9 +110,11 @@ func generateReleaseNotes(httpClient *http.Client, repo ghrepo.Interface, tagNam return nil, err } - path := fmt.Sprintf("repos/%s/%s/releases/generate-notes", repo.RepoOwner(), repo.RepoName()) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path - req, err := http.NewRequest("POST", url, bytes.NewBuffer(bodyBytes)) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "releases", "generate-notes") + if err != nil { + return nil, err + } + req, err := http.NewRequest("POST", url.String(), bytes.NewBuffer(bodyBytes)) if err != nil { return nil, err } @@ -142,9 +148,11 @@ func generateReleaseNotes(httpClient *http.Client, repo ghrepo.Interface, tagNam } func publishedReleaseExists(httpClient *http.Client, repo ghrepo.Interface, tagName string) (bool, error) { - path := fmt.Sprintf("repos/%s/%s/releases/tags/%s", repo.RepoOwner(), repo.RepoName(), url.PathEscape(tagName)) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path - req, err := http.NewRequest("HEAD", url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "releases", "tags", tagName) + if err != nil { + return false, err + } + req, err := http.NewRequest("HEAD", url.String(), nil) if err != nil { return false, err } @@ -172,9 +180,11 @@ func createRelease(httpClient *http.Client, repo ghrepo.Interface, params map[st return nil, err } - path := fmt.Sprintf("repos/%s/%s/releases", repo.RepoOwner(), repo.RepoName()) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path - req, err := http.NewRequest("POST", url, bytes.NewBuffer(bodyBytes)) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "releases") + if err != nil { + return nil, err + } + req, err := http.NewRequest("POST", url.String(), bytes.NewBuffer(bodyBytes)) if err != nil { return nil, err } @@ -220,7 +230,7 @@ func createRelease(httpClient *http.Client, repo ghrepo.Interface, params map[st return &newRelease, err } -func publishRelease(httpClient *http.Client, releaseURL string, discussionCategory string, isLatest *bool) (*shared.Release, error) { +func publishRelease(httpClient *http.Client, releaseURL safeurl.SafeURL, discussionCategory string, isLatest *bool) (*shared.Release, error) { params := map[string]interface{}{"draft": false} if discussionCategory != "" { params["discussion_category_name"] = discussionCategory @@ -234,7 +244,7 @@ func publishRelease(httpClient *http.Client, releaseURL string, discussionCatego if err != nil { return nil, err } - req, err := http.NewRequest("PATCH", releaseURL, bytes.NewBuffer(bodyBytes)) + req, err := http.NewRequest("PATCH", releaseURL.String(), bytes.NewBuffer(bodyBytes)) if err != nil { return nil, err } @@ -261,8 +271,8 @@ func publishRelease(httpClient *http.Client, releaseURL string, discussionCatego return &release, err } -func deleteRelease(httpClient *http.Client, release *shared.Release) error { - req, err := http.NewRequest("DELETE", release.APIURL, nil) +func deleteRelease(httpClient *http.Client, releaseURL safeurl.SafeURL) error { + req, err := http.NewRequest("DELETE", releaseURL.String(), nil) if err != nil { return err } @@ -314,14 +324,18 @@ func isNewRelease(httpClient *http.Client, repo ghrepo.Interface) (bool, error) } tagName := release.TagName - path := fmt.Sprintf("repos/%s/%s/compare/%s...HEAD?per_page=1", repo.RepoOwner(), repo.RepoName(), tagName) + u, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "compare", tagName+"...HEAD") + if err != nil { + return false, err + } + u.SetQuery("per_page", "1") var comparisonStatus struct { Status string `json:"status"` } apiClient := api.NewClientFromHTTP(httpClient) - if err := apiClient.REST(repo.RepoHost(), "GET", path, nil, &comparisonStatus); err != nil { + if err := apiClient.REST(repo.RepoHost(), "GET", u.String(), nil, &comparisonStatus); err != nil { return false, err } diff --git a/pkg/cmd/release/delete-asset/delete_asset.go b/pkg/cmd/release/delete-asset/delete_asset.go index b2e1f22fea1..3aedc3e3a46 100644 --- a/pkg/cmd/release/delete-asset/delete_asset.go +++ b/pkg/cmd/release/delete-asset/delete_asset.go @@ -7,6 +7,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/release/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -96,7 +97,7 @@ func deleteAssetRun(opts *DeleteAssetOptions) error { return fmt.Errorf("asset %s not found in release %s", opts.AssetName, release.TagName) } - err = deleteAsset(httpClient, assetURL) + err = deleteAsset(httpClient, safeurl.NewImmutableSafeURL(assetURL)) if err != nil { return err } @@ -111,8 +112,8 @@ func deleteAssetRun(opts *DeleteAssetOptions) error { return nil } -func deleteAsset(httpClient *http.Client, assetURL string) error { - req, err := http.NewRequest("DELETE", assetURL, nil) +func deleteAsset(httpClient *http.Client, assetURL safeurl.SafeURL) error { + req, err := http.NewRequest("DELETE", assetURL.String(), nil) if err != nil { return err } diff --git a/pkg/cmd/release/delete/delete.go b/pkg/cmd/release/delete/delete.go index 622b188934b..108ebae7ece 100644 --- a/pkg/cmd/release/delete/delete.go +++ b/pkg/cmd/release/delete/delete.go @@ -9,6 +9,7 @@ import ( "github.com/cli/cli/v2/git" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/release/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -92,7 +93,7 @@ func deleteRun(opts *DeleteOptions) error { } } - err = deleteRelease(httpClient, release.APIURL) + err = deleteRelease(httpClient, safeurl.NewImmutableSafeURL(release.APIURL)) if err != nil { return err } @@ -121,8 +122,8 @@ func deleteRun(opts *DeleteOptions) error { return nil } -func deleteRelease(httpClient *http.Client, releaseURL string) error { - req, err := http.NewRequest("DELETE", releaseURL, nil) +func deleteRelease(httpClient *http.Client, releaseURL safeurl.SafeURL) error { + req, err := http.NewRequest("DELETE", releaseURL.String(), nil) if err != nil { return err } @@ -140,10 +141,11 @@ func deleteRelease(httpClient *http.Client, releaseURL string) error { } func deleteTag(httpClient *http.Client, baseRepo ghrepo.Interface, tagName string) error { - path := fmt.Sprintf("repos/%s/%s/git/refs/tags/%s", baseRepo.RepoOwner(), baseRepo.RepoName(), tagName) - url := ghinstance.RESTPrefix(baseRepo.RepoHost()) + path - - req, err := http.NewRequest("DELETE", url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(baseRepo.RepoHost()), "repos", baseRepo.RepoOwner(), baseRepo.RepoName(), "git", "refs", fmt.Sprintf("tags/%s", tagName)) + if err != nil { + return err + } + req, err := http.NewRequest("DELETE", url.String(), nil) if err != nil { return err } diff --git a/pkg/cmd/release/delete/delete_test.go b/pkg/cmd/release/delete/delete_test.go index 2787f247be1..13904ff3501 100644 --- a/pkg/cmd/release/delete/delete_test.go +++ b/pkg/cmd/release/delete/delete_test.go @@ -210,7 +210,7 @@ func Test_deleteRun(t *testing.T) { }`) fakeHTTP.Register(httpmock.REST("DELETE", "repos/OWNER/REPO/releases/23456"), httpmock.StatusStringResponse(204, "")) - fakeHTTP.Register(httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/tags/v1.2.3"), httpmock.StatusStringResponse(204, "")) + fakeHTTP.Register(httpmock.REST("DELETE", "repos/OWNER/REPO/git/refs/tags%2Fv1.2.3"), httpmock.StatusStringResponse(204, "")) rs, teardown := run.Stub() defer teardown(t) diff --git a/pkg/cmd/release/download/download.go b/pkg/cmd/release/download/download.go index d132de5b631..688d94c1472 100644 --- a/pkg/cmd/release/download/download.go +++ b/pkg/cmd/release/download/download.go @@ -17,6 +17,7 @@ import ( "github.com/MakeNowJust/heredoc" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/release/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -233,7 +234,15 @@ func downloadRun(opts *DownloadOptions) error { isTTY: opts.IO.IsStdoutTTY(), } - return downloadAssets(&dest, httpClient, toDownload, opts.Concurrency, isArchive, opts.IO) + targets := make([]downloadTarget, len(toDownload)) + for i, a := range toDownload { + targets[i] = downloadTarget{ + url: safeurl.NewImmutableSafeURL(a.APIURL), + name: a.Name, + } + } + + return downloadAssets(&dest, httpClient, targets, opts.Concurrency, isArchive, opts.IO) } func matchAny(patterns []string, name string) bool { @@ -245,12 +254,17 @@ func matchAny(patterns []string, name string) bool { return false } -func downloadAssets(dest *destinationWriter, httpClient *http.Client, toDownload []shared.ReleaseAsset, numWorkers int, isArchive bool, io *iostreams.IOStreams) error { +type downloadTarget struct { + url safeurl.SafeURL + name string +} + +func downloadAssets(dest *destinationWriter, httpClient *http.Client, toDownload []downloadTarget, numWorkers int, isArchive bool, io *iostreams.IOStreams) error { if numWorkers == 0 { return errors.New("the number of concurrent workers needs to be greater than 0") } - jobs := make(chan shared.ReleaseAsset, len(toDownload)) + jobs := make(chan downloadTarget, len(toDownload)) results := make(chan error, len(toDownload)) if len(toDownload) < numWorkers { @@ -260,8 +274,8 @@ func downloadAssets(dest *destinationWriter, httpClient *http.Client, toDownload for w := 1; w <= numWorkers; w++ { go func() { for a := range jobs { - io.StartProgressIndicatorWithLabel(fmt.Sprintf("Downloading %s", a.Name)) - results <- downloadAsset(dest, httpClient, a.APIURL, a.Name, isArchive) + io.StartProgressIndicatorWithLabel(fmt.Sprintf("Downloading %s", a.name)) + results <- downloadAsset(dest, httpClient, a.url, a.name, isArchive) } }() } @@ -283,12 +297,12 @@ func downloadAssets(dest *destinationWriter, httpClient *http.Client, toDownload return downloadError } -func downloadAsset(dest *destinationWriter, httpClient *http.Client, assetURL, fileName string, isArchive bool) error { +func downloadAsset(dest *destinationWriter, httpClient *http.Client, assetURL safeurl.SafeURL, fileName string, isArchive bool) error { if err := dest.Check(fileName); err != nil { return err } - req, err := http.NewRequest("GET", assetURL, nil) + req, err := http.NewRequest("GET", assetURL.String(), nil) if err != nil { return err } diff --git a/pkg/cmd/release/edit/http.go b/pkg/cmd/release/edit/http.go index bf310da6053..291123ad3ea 100644 --- a/pkg/cmd/release/edit/http.go +++ b/pkg/cmd/release/edit/http.go @@ -6,10 +6,12 @@ import ( "fmt" "io" "net/http" + "strconv" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/release/shared" "github.com/shurcooL/githubv4" ) @@ -20,9 +22,11 @@ func editRelease(httpClient *http.Client, repo ghrepo.Interface, releaseID int64 return nil, err } - path := fmt.Sprintf("repos/%s/%s/releases/%d", repo.RepoOwner(), repo.RepoName(), releaseID) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path - req, err := http.NewRequest("PATCH", url, bytes.NewBuffer(bodyBytes)) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "releases", strconv.FormatInt(releaseID, 10)) + if err != nil { + return nil, err + } + req, err := http.NewRequest("PATCH", url.String(), bytes.NewBuffer(bodyBytes)) if err != nil { return nil, err } diff --git a/pkg/cmd/release/shared/fetch.go b/pkg/cmd/release/shared/fetch.go index 420b83b366b..74bf06657a1 100644 --- a/pkg/cmd/release/shared/fetch.go +++ b/pkg/cmd/release/shared/fetch.go @@ -8,6 +8,7 @@ import ( "io" "net/http" "reflect" + "strconv" "strings" "testing" "time" @@ -15,6 +16,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/httpmock" "github.com/shurcooL/githubv4" "github.com/stretchr/testify/assert" @@ -136,8 +138,11 @@ type fetchResult struct { } func FetchRefSHA(ctx context.Context, httpClient *http.Client, repo ghrepo.Interface, tagName string) (string, error) { - path := fmt.Sprintf("repos/%s/%s/git/ref/tags/%s", repo.RepoOwner(), repo.RepoName(), tagName) - req, err := http.NewRequestWithContext(ctx, "GET", ghinstance.RESTPrefix(repo.RepoHost())+path, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "git", "ref", fmt.Sprintf("tags/%s", tagName)) + if err != nil { + return "", err + } + req, err := http.NewRequestWithContext(ctx, "GET", url.String(), nil) if err != nil { return "", err } @@ -185,13 +190,17 @@ func DigestAlgForRef(digest string) string { // FetchRelease finds a published repository release by its tagName, or a draft release by its pending tag name. func FetchRelease(ctx context.Context, httpClient *http.Client, repo ghrepo.Interface, tagName string) (*Release, error) { + publishedURL, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "releases", "tags", tagName) + if err != nil { + return nil, err + } + cc, cancel := context.WithCancel(ctx) results := make(chan fetchResult, 2) // published release lookup go func() { - path := fmt.Sprintf("repos/%s/%s/releases/tags/%s", repo.RepoOwner(), repo.RepoName(), tagName) - release, err := fetchReleasePath(cc, httpClient, repo.RepoHost(), path) + release, err := fetchReleasePath(cc, httpClient, publishedURL) results <- fetchResult{release: release, error: err} }() @@ -226,8 +235,11 @@ func FetchRelease(ctx context.Context, httpClient *http.Client, repo ghrepo.Inte // FetchLatestRelease finds the latest published release for a repository. func FetchLatestRelease(ctx context.Context, httpClient *http.Client, repo ghrepo.Interface) (*Release, error) { - path := fmt.Sprintf("repos/%s/%s/releases/latest", repo.RepoOwner(), repo.RepoName()) - return fetchReleasePath(ctx, httpClient, repo.RepoHost(), path) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "releases", "latest") + if err != nil { + return nil, err + } + return fetchReleasePath(ctx, httpClient, url) } // fetchDraftRelease returns the first draft release that has tagName as its pending tag. @@ -259,12 +271,15 @@ func fetchDraftRelease(ctx context.Context, httpClient *http.Client, repo ghrepo // Then, use REST to get information about the draft release. In theory, we could have fetched // all the necessary information via GraphQL, but REST is safer for backwards compatibility. - path := fmt.Sprintf("repos/%s/%s/releases/%d", repo.RepoOwner(), repo.RepoName(), query.Repository.Release.DatabaseID) - return fetchReleasePath(ctx, httpClient, repo.RepoHost(), path) + path, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "releases", strconv.FormatInt(query.Repository.Release.DatabaseID, 10)) + if err != nil { + return nil, err + } + return fetchReleasePath(ctx, httpClient, path) } -func fetchReleasePath(ctx context.Context, httpClient *http.Client, host string, p string) (*Release, error) { - req, err := http.NewRequestWithContext(ctx, "GET", ghinstance.RESTPrefix(host)+p, nil) +func fetchReleasePath(ctx context.Context, httpClient *http.Client, url safeurl.SafeURL) (*Release, error) { + req, err := http.NewRequestWithContext(ctx, "GET", url.String(), nil) if err != nil { return nil, err } @@ -312,7 +327,7 @@ func StubFetchRelease(t *testing.T, reg *httpmock.Registry, owner, repoName, tag } func StubFetchRefSHA(t *testing.T, reg *httpmock.Registry, owner, repoName, tagName, sha string) { - path := fmt.Sprintf("repos/%s/%s/git/ref/tags/%s", owner, repoName, tagName) + path := fmt.Sprintf("repos/%s/%s/git/ref/tags%%2F%s", owner, repoName, tagName) reg.Register( httpmock.REST("GET", path), httpmock.StringResponse(fmt.Sprintf(`{"object": {"sha": "%s"}}`, sha)), diff --git a/pkg/cmd/release/shared/fetch_test.go b/pkg/cmd/release/shared/fetch_test.go index 0720b876f6f..e68278f1ed1 100644 --- a/pkg/cmd/release/shared/fetch_test.go +++ b/pkg/cmd/release/shared/fetch_test.go @@ -42,7 +42,7 @@ func TestFetchRefSHA(t *testing.T) { tagName: "v1.2.3", responseStatus: 500, responseMessage: `arbitrary error"`, - errorMessage: "HTTP 500: arbitrary error\" (https://api.github.com/repos/owner/repo/git/ref/tags/v1.2.3)", + errorMessage: "HTTP 500: arbitrary error\" (https://api.github.com/repos/owner/repo/git/ref/tags%2Fv1.2.3)", }, { name: "malformed JSON with 200", @@ -61,7 +61,7 @@ func TestFetchRefSHA(t *testing.T) { repo, err := ghrepo.FromFullName("owner/repo") require.NoError(t, err) - path := "repos/owner/repo/git/ref/tags/" + tt.tagName + path := "repos/owner/repo/git/ref/tags%2F" + tt.tagName if tt.responseStatus == 404 || tt.responseStatus == 500 { fakeHTTP.Register( httpmock.REST("GET", path), diff --git a/pkg/cmd/release/shared/upload.go b/pkg/cmd/release/shared/upload.go index ab7533320e8..9307c8c01c2 100644 --- a/pkg/cmd/release/shared/upload.go +++ b/pkg/cmd/release/shared/upload.go @@ -15,6 +15,7 @@ import ( "github.com/cenkalti/backoff/v4" "github.com/cli/cli/v2/api" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmdutil" "golang.org/x/sync/errgroup" ) @@ -33,7 +34,7 @@ type AssetForUpload struct { MIMEType string Open func() (io.ReadCloser, error) - ExistingURL string + ExistingURL safeurl.SafeURL } func AssetsFromArgs(args []string) (assets []*AssetForUpload, err error) { @@ -111,7 +112,7 @@ func fileExt(fn string) string { return path.Ext(fn) } -func ConcurrentUpload(httpClient httpDoer, uploadURL string, numWorkers int, assets []*AssetForUpload) error { +func ConcurrentUpload(httpClient httpDoer, uploadURL safeurl.SafeURL, numWorkers int, assets []*AssetForUpload) error { if numWorkers == 0 { return errors.New("the number of concurrent workers needs to be greater than 0") } @@ -142,8 +143,8 @@ func shouldRetry(err error) bool { // Allow injecting backoff interval in tests. var retryInterval = time.Millisecond * 200 -func uploadWithDelete(ctx context.Context, httpClient httpDoer, uploadURL string, a AssetForUpload) error { - if a.ExistingURL != "" { +func uploadWithDelete(ctx context.Context, httpClient httpDoer, uploadURL safeurl.SafeURL, a AssetForUpload) error { + if a.ExistingURL != nil && a.ExistingURL.String() != "" { if err := deleteAsset(ctx, httpClient, a.ExistingURL); err != nil { return err } @@ -158,8 +159,8 @@ func uploadWithDelete(ctx context.Context, httpClient httpDoer, uploadURL string }, backoff.WithContext(backoff.WithMaxRetries(bo, 3), ctx)) } -func uploadAsset(ctx context.Context, httpClient httpDoer, uploadURL string, asset AssetForUpload) (*ReleaseAsset, error) { - u, err := url.Parse(uploadURL) +func uploadAsset(ctx context.Context, httpClient httpDoer, uploadURL safeurl.SafeURL, asset AssetForUpload) (*ReleaseAsset, error) { + u, err := url.Parse(uploadURL.String()) if err != nil { return nil, err } @@ -168,13 +169,16 @@ func uploadAsset(ctx context.Context, httpClient httpDoer, uploadURL string, ass params.Set("label", asset.Label) u.RawQuery = params.Encode() + // Since u is derived from uploadURL, an already-trusted safeurl.SafeURL, the resulting URL is safe to declare as such. + safeURL := safeurl.NewImmutableSafeURL(u.String()) + f, err := asset.Open() if err != nil { return nil, err } defer f.Close() - req, err := http.NewRequestWithContext(ctx, "POST", u.String(), f) + req, err := http.NewRequestWithContext(ctx, "POST", safeURL.String(), f) if err != nil { return nil, err } @@ -202,8 +206,8 @@ func uploadAsset(ctx context.Context, httpClient httpDoer, uploadURL string, ass return &newAsset, nil } -func deleteAsset(ctx context.Context, httpClient httpDoer, assetURL string) error { - req, err := http.NewRequestWithContext(ctx, "DELETE", assetURL, nil) +func deleteAsset(ctx context.Context, httpClient httpDoer, assetURL safeurl.SafeURL) error { + req, err := http.NewRequestWithContext(ctx, "DELETE", assetURL.String(), nil) if err != nil { return err } diff --git a/pkg/cmd/release/shared/upload_test.go b/pkg/cmd/release/shared/upload_test.go index 26bed11c017..8271fa6ae6e 100644 --- a/pkg/cmd/release/shared/upload_test.go +++ b/pkg/cmd/release/shared/upload_test.go @@ -7,6 +7,8 @@ import ( "io" "net/http" "testing" + + "github.com/cli/cli/v2/internal/safeurl" ) func Test_typeForFilename(t *testing.T) { @@ -97,7 +99,7 @@ func Test_uploadWithDelete_retry(t *testing.T) { Body: io.NopCloser(bytes.NewBufferString(`{}`)), }, nil }) - err := uploadWithDelete(ctx, client, "http://example.com/upload", AssetForUpload{ + err := uploadWithDelete(ctx, client, safeurl.NewImmutableSafeURL("http://example.com/upload"), AssetForUpload{ Name: "asset", Label: "", Size: 8, diff --git a/pkg/cmd/release/upload/upload.go b/pkg/cmd/release/upload/upload.go index 827dcdc6421..35b10860d75 100644 --- a/pkg/cmd/release/upload/upload.go +++ b/pkg/cmd/release/upload/upload.go @@ -9,6 +9,7 @@ import ( "github.com/MakeNowJust/heredoc" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/text" "github.com/cli/cli/v2/pkg/cmd/release/shared" "github.com/cli/cli/v2/pkg/cmdutil" @@ -90,17 +91,12 @@ func uploadRun(opts *UploadOptions) error { return err } - uploadURL := release.UploadURL - if idx := strings.IndexRune(uploadURL, '{'); idx > 0 { - uploadURL = uploadURL[:idx] - } - var existingNames []string for _, a := range opts.Assets { sanitizedFileName := sanitizeFileName(a.Name) for _, ea := range release.Assets { if ea.Name == sanitizedFileName { - a.ExistingURL = ea.APIURL + a.ExistingURL = safeurl.NewImmutableSafeURL(ea.APIURL) existingNames = append(existingNames, ea.Name) break } @@ -111,8 +107,13 @@ func uploadRun(opts *UploadOptions) error { return fmt.Errorf("asset under the same name already exists: %v", existingNames) } + uploadURL := release.UploadURL + if idx := strings.IndexRune(uploadURL, '{'); idx > 0 { + uploadURL = uploadURL[:idx] + } + opts.IO.StartProgressIndicator() - err = shared.ConcurrentUpload(httpClient, uploadURL, opts.Concurrency, opts.Assets) + err = shared.ConcurrentUpload(httpClient, safeurl.NewImmutableSafeURL(uploadURL), opts.Concurrency, opts.Assets) opts.IO.StopProgressIndicator() if err != nil { return err diff --git a/pkg/cmd/repo/autolink/create/http.go b/pkg/cmd/repo/autolink/create/http.go index 5f187319f88..1de86e42cb8 100644 --- a/pkg/cmd/repo/autolink/create/http.go +++ b/pkg/cmd/repo/autolink/create/http.go @@ -4,12 +4,12 @@ import ( "bytes" "encoding/json" "errors" - "fmt" "net/http" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/repo/autolink/shared" ) @@ -24,8 +24,10 @@ type AutolinkCreateRequest struct { } func (a *AutolinkCreator) Create(repo ghrepo.Interface, request AutolinkCreateRequest) (*shared.Autolink, error) { - path := fmt.Sprintf("repos/%s/%s/autolinks", repo.RepoOwner(), repo.RepoName()) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "autolinks") + if err != nil { + return nil, err + } requestByte, err := json.Marshal(request) if err != nil { @@ -33,7 +35,7 @@ func (a *AutolinkCreator) Create(repo ghrepo.Interface, request AutolinkCreateRe } requestBody := bytes.NewReader(requestByte) - req, err := http.NewRequest(http.MethodPost, url, requestBody) + req, err := http.NewRequest(http.MethodPost, url.String(), requestBody) if err != nil { return nil, err } diff --git a/pkg/cmd/repo/autolink/delete/http.go b/pkg/cmd/repo/autolink/delete/http.go index d6bc53e840f..ed35d6328df 100644 --- a/pkg/cmd/repo/autolink/delete/http.go +++ b/pkg/cmd/repo/autolink/delete/http.go @@ -7,6 +7,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" ) type AutolinkDeleter struct { @@ -14,9 +15,11 @@ type AutolinkDeleter struct { } func (a *AutolinkDeleter) Delete(repo ghrepo.Interface, id string) error { - path := fmt.Sprintf("repos/%s/%s/autolinks/%s", repo.RepoOwner(), repo.RepoName(), id) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path - req, err := http.NewRequest(http.MethodDelete, url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "autolinks", id) + if err != nil { + return err + } + req, err := http.NewRequest(http.MethodDelete, url.String(), nil) if err != nil { return err } @@ -28,7 +31,7 @@ func (a *AutolinkDeleter) Delete(repo ghrepo.Interface, id string) error { defer resp.Body.Close() if resp.StatusCode == http.StatusNotFound { - return fmt.Errorf("error deleting autolink: HTTP 404: Perhaps you are missing admin rights to the repository? (https://api.github.com/%s)", path) + return fmt.Errorf("error deleting autolink: HTTP 404: Perhaps you are missing admin rights to the repository? (%s)", url) } else if resp.StatusCode > 299 { return api.HandleHTTPError(resp) } diff --git a/pkg/cmd/repo/autolink/list/http.go b/pkg/cmd/repo/autolink/list/http.go index cdb8e621c61..210495c7613 100644 --- a/pkg/cmd/repo/autolink/list/http.go +++ b/pkg/cmd/repo/autolink/list/http.go @@ -8,6 +8,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/repo/autolink/shared" ) @@ -16,9 +17,11 @@ type AutolinkLister struct { } func (a *AutolinkLister) List(repo ghrepo.Interface) ([]shared.Autolink, error) { - path := fmt.Sprintf("repos/%s/%s/autolinks", repo.RepoOwner(), repo.RepoName()) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path - req, err := http.NewRequest(http.MethodGet, url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "autolinks") + if err != nil { + return nil, err + } + req, err := http.NewRequest(http.MethodGet, url.String(), nil) if err != nil { return nil, err } @@ -30,7 +33,7 @@ func (a *AutolinkLister) List(repo ghrepo.Interface) ([]shared.Autolink, error) defer resp.Body.Close() if resp.StatusCode == http.StatusNotFound { - return nil, fmt.Errorf("error getting autolinks: HTTP 404: Perhaps you are missing admin rights to the repository? (https://api.github.com/%s)", path) + return nil, fmt.Errorf("error getting autolinks: HTTP 404: Perhaps you are missing admin rights to the repository? (%s)", url) } else if resp.StatusCode > 299 { return nil, api.HandleHTTPError(resp) } diff --git a/pkg/cmd/repo/autolink/view/http.go b/pkg/cmd/repo/autolink/view/http.go index cc5638613e4..a604fb8f24d 100644 --- a/pkg/cmd/repo/autolink/view/http.go +++ b/pkg/cmd/repo/autolink/view/http.go @@ -8,6 +8,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/repo/autolink/shared" ) @@ -16,9 +17,11 @@ type AutolinkViewer struct { } func (a *AutolinkViewer) View(repo ghrepo.Interface, id string) (*shared.Autolink, error) { - path := fmt.Sprintf("repos/%s/%s/autolinks/%s", repo.RepoOwner(), repo.RepoName(), id) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path - req, err := http.NewRequest(http.MethodGet, url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "autolinks", id) + if err != nil { + return nil, err + } + req, err := http.NewRequest(http.MethodGet, url.String(), nil) if err != nil { return nil, err } @@ -30,7 +33,7 @@ func (a *AutolinkViewer) View(repo ghrepo.Interface, id string) (*shared.Autolin defer resp.Body.Close() if resp.StatusCode == http.StatusNotFound { - return nil, fmt.Errorf("HTTP 404: Perhaps you are missing admin rights to the repository? (https://api.github.com/%s)", path) + return nil, fmt.Errorf("HTTP 404: Perhaps you are missing admin rights to the repository? (%s)", url) } else if resp.StatusCode > 299 { return nil, api.HandleHTTPError(resp) } diff --git a/pkg/cmd/repo/create/http.go b/pkg/cmd/repo/create/http.go index 725fc48c555..8a93814ca98 100644 --- a/pkg/cmd/repo/create/http.go +++ b/pkg/cmd/repo/create/http.go @@ -8,6 +8,7 @@ import ( "strings" "github.com/cli/cli/v2/api" + "github.com/cli/cli/v2/internal/safeurl" "github.com/shurcooL/githubv4" ) @@ -186,9 +187,15 @@ func repoCreate(client *http.Client, hostname string, input repoCreateInput) (*a InitReadme: input.InitReadme, } - path := "user/repos" + path, err := safeurl.JoinPath("user", "repos") + if err != nil { + return nil, err + } if isOrg { - path = fmt.Sprintf("orgs/%s/repos", input.OwnerLogin) + path, err = safeurl.JoinPath("orgs", input.OwnerLogin, "repos") + if err != nil { + return nil, err + } inputv3.Visibility = strings.ToLower(input.Visibility) } @@ -254,7 +261,11 @@ func (r *ownerResponse) IsOrganization() bool { func resolveOwner(client *api.Client, hostname, orgName string) (*ownerResponse, error) { var response ownerResponse - err := client.REST(hostname, "GET", fmt.Sprintf("users/%s", orgName), nil, &response) + u, err := safeurl.JoinPath("users", orgName) + if err != nil { + return nil, err + } + err = client.REST(hostname, "GET", u.String(), nil, &response) return &response, err } @@ -268,7 +279,11 @@ type teamResponse struct { func resolveOrganizationTeam(client *api.Client, hostname, orgName, teamSlug string) (*teamResponse, error) { var response teamResponse - err := client.REST(hostname, "GET", fmt.Sprintf("orgs/%s/teams/%s", orgName, teamSlug), nil, &response) + u, err := safeurl.JoinPath("orgs", orgName, "teams", teamSlug) + if err != nil { + return nil, err + } + err = client.REST(hostname, "GET", u.String(), nil, &response) return &response, err } diff --git a/pkg/cmd/repo/credits/credits.go b/pkg/cmd/repo/credits/credits.go index a26b6a7312a..42c5766d7ed 100644 --- a/pkg/cmd/repo/credits/credits.go +++ b/pkg/cmd/repo/credits/credits.go @@ -15,6 +15,7 @@ import ( "github.com/MakeNowJust/heredoc" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" "github.com/cli/cli/v2/utils" @@ -142,9 +143,12 @@ func creditsRun(opts *CreditsOptions) error { result := Result{} body := bytes.NewBufferString("") - path := fmt.Sprintf("repos/%s/%s/contributors", baseRepo.RepoOwner(), baseRepo.RepoName()) + path, err := safeurl.JoinPath("repos", baseRepo.RepoOwner(), baseRepo.RepoName(), "contributors") + if err != nil { + return err + } - err = client.REST(baseRepo.RepoHost(), "GET", path, body, &result) + err = client.REST(baseRepo.RepoHost(), "GET", path.String(), body, &result) if err != nil { return err } diff --git a/pkg/cmd/repo/delete/http.go b/pkg/cmd/repo/delete/http.go index 23930e8e054..faffa006e3d 100644 --- a/pkg/cmd/repo/delete/http.go +++ b/pkg/cmd/repo/delete/http.go @@ -1,12 +1,12 @@ package delete import ( - "fmt" "net/http" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" ) func deleteRepo(client *http.Client, repo ghrepo.Interface) error { @@ -16,11 +16,12 @@ func deleteRepo(client *http.Client, repo ghrepo.Interface) error { return http.ErrUseLastResponse } - url := fmt.Sprintf("%srepos/%s", - ghinstance.RESTPrefix(repo.RepoHost()), - ghrepo.FullName(repo)) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName()) + if err != nil { + return err + } - request, err := http.NewRequest("DELETE", url, nil) + request, err := http.NewRequest("DELETE", url.String(), nil) if err != nil { return err } diff --git a/pkg/cmd/repo/deploy-key/add/http.go b/pkg/cmd/repo/deploy-key/add/http.go index d2a933f5fa0..5111049c8ed 100644 --- a/pkg/cmd/repo/deploy-key/add/http.go +++ b/pkg/cmd/repo/deploy-key/add/http.go @@ -3,18 +3,20 @@ package add import ( "bytes" "encoding/json" - "fmt" "io" "net/http" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" ) func uploadDeployKey(httpClient *http.Client, repo ghrepo.Interface, keyFile io.Reader, title string, isWritable bool) error { - path := fmt.Sprintf("repos/%s/%s/keys", repo.RepoOwner(), repo.RepoName()) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "keys") + if err != nil { + return err + } keyBytes, err := io.ReadAll(keyFile) if err != nil { @@ -32,7 +34,7 @@ func uploadDeployKey(httpClient *http.Client, repo ghrepo.Interface, keyFile io. return err } - req, err := http.NewRequest("POST", url, bytes.NewBuffer(payloadBytes)) + req, err := http.NewRequest("POST", url.String(), bytes.NewBuffer(payloadBytes)) if err != nil { return err } diff --git a/pkg/cmd/repo/deploy-key/delete/http.go b/pkg/cmd/repo/deploy-key/delete/http.go index 53de349fcbd..117ce697a29 100644 --- a/pkg/cmd/repo/deploy-key/delete/http.go +++ b/pkg/cmd/repo/deploy-key/delete/http.go @@ -1,20 +1,22 @@ package delete import ( - "fmt" "io" "net/http" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" ) func deleteDeployKey(httpClient *http.Client, repo ghrepo.Interface, id string) error { - path := fmt.Sprintf("repos/%s/%s/keys/%s", repo.RepoOwner(), repo.RepoName(), id) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "keys", id) + if err != nil { + return err + } - req, err := http.NewRequest("DELETE", url, nil) + req, err := http.NewRequest("DELETE", url.String(), nil) if err != nil { return err } diff --git a/pkg/cmd/repo/deploy-key/list/http.go b/pkg/cmd/repo/deploy-key/list/http.go index 134b128dc4e..391d6bbe17d 100644 --- a/pkg/cmd/repo/deploy-key/list/http.go +++ b/pkg/cmd/repo/deploy-key/list/http.go @@ -2,7 +2,6 @@ package list import ( "encoding/json" - "fmt" "io" "net/http" "time" @@ -10,6 +9,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" ) type deployKey struct { @@ -21,9 +21,12 @@ type deployKey struct { } func repoKeys(httpClient *http.Client, repo ghrepo.Interface) ([]deployKey, error) { - path := fmt.Sprintf("repos/%s/%s/keys?per_page=100", repo.RepoOwner(), repo.RepoName()) - url := ghinstance.RESTPrefix(repo.RepoHost()) + path - req, err := http.NewRequest("GET", url, nil) + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "keys") + if err != nil { + return nil, err + } + u.SetQuery("per_page", "100") + req, err := http.NewRequest("GET", u.String(), nil) if err != nil { return nil, err } diff --git a/pkg/cmd/repo/edit/edit.go b/pkg/cmd/repo/edit/edit.go index aff7a5fe188..c215f182ccc 100644 --- a/pkg/cmd/repo/edit/edit.go +++ b/pkg/cmd/repo/edit/edit.go @@ -16,6 +16,7 @@ import ( fd "github.com/cli/cli/v2/internal/featuredetection" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/text" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -298,7 +299,10 @@ func editRun(ctx context.Context, opts *EditOptions) error { } } - apiPath := fmt.Sprintf("repos/%s/%s", repo.RepoOwner(), repo.RepoName()) + apiPath, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName()) + if err != nil { + return err + } body := &bytes.Buffer{} enc := json.NewEncoder(body) @@ -341,7 +345,7 @@ func editRun(ctx context.Context, opts *EditOptions) error { }) } - err := g.Wait() + err = g.Wait() if err != nil { return err } @@ -563,8 +567,11 @@ func parseTopics(s string) []string { } func getTopics(ctx context.Context, httpClient *http.Client, repo ghrepo.Interface) ([]string, error) { - apiPath := fmt.Sprintf("repos/%s/%s/topics", repo.RepoOwner(), repo.RepoName()) - req, err := http.NewRequestWithContext(ctx, "GET", ghinstance.RESTPrefix(repo.RepoHost())+apiPath, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "topics") + if err != nil { + return nil, err + } + req, err := http.NewRequestWithContext(ctx, "GET", url.String(), nil) if err != nil { return nil, err } @@ -601,8 +608,11 @@ func setTopics(ctx context.Context, httpClient *http.Client, repo ghrepo.Interfa return err } - apiPath := fmt.Sprintf("repos/%s/%s/topics", repo.RepoOwner(), repo.RepoName()) - req, err := http.NewRequestWithContext(ctx, "PUT", ghinstance.RESTPrefix(repo.RepoHost())+apiPath, body) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "topics") + if err != nil { + return err + } + req, err := http.NewRequestWithContext(ctx, "PUT", url.String(), body) if err != nil { return err } diff --git a/pkg/cmd/repo/garden/http.go b/pkg/cmd/repo/garden/http.go index d968296f497..903de787632 100644 --- a/pkg/cmd/repo/garden/http.go +++ b/pkg/cmd/repo/garden/http.go @@ -3,14 +3,15 @@ package garden import ( "encoding/json" "errors" - "fmt" "io" "net/http" + "strconv" "strings" "time" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" ) func getCommits(client *http.Client, repo ghrepo.Interface, maxCommits int) ([]*Commit, error) { @@ -25,8 +26,14 @@ func getCommits(client *http.Client, repo ghrepo.Interface, maxCommits int) ([]* commits := []*Commit{} - pathF := func(page int) string { - return fmt.Sprintf("repos/%s/%s/commits?per_page=100&page=%d", repo.RepoOwner(), repo.RepoName(), page) + pathF := func(page int) (*safeurl.MutableSafeURL, error) { + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "commits") + if err != nil { + return nil, err + } + u.SetQuery("per_page", "100") + u.SetQuery("page", strconv.Itoa(page)) + return u, nil } page := 1 @@ -36,7 +43,11 @@ func getCommits(client *http.Client, repo ghrepo.Interface, maxCommits int) ([]* break } result := Result{} - links, err := getResponse(client, repo.RepoHost(), pathF(page), &result) + path, err := pathF(page) + if err != nil { + return nil, err + } + links, err := getResponse(client, path, &result) if err != nil { return nil, err } @@ -69,9 +80,8 @@ func getCommits(client *http.Client, repo ghrepo.Interface, maxCommits int) ([]* // getResponse performs the API call and returns the response's link header values. // If the "Link" header is missing, the returned slice will be nil. -func getResponse(client *http.Client, host, path string, data interface{}) ([]string, error) { - url := ghinstance.RESTPrefix(host) + path - req, err := http.NewRequest("GET", url, nil) +func getResponse(client *http.Client, url safeurl.SafeURL, data interface{}) ([]string, error) { + req, err := http.NewRequest("GET", url.String(), nil) if err != nil { return nil, err } diff --git a/pkg/cmd/repo/read-file/http.go b/pkg/cmd/repo/read-file/http.go index e703ffe8733..20c1ed0e9f1 100644 --- a/pkg/cmd/repo/read-file/http.go +++ b/pkg/cmd/repo/read-file/http.go @@ -6,12 +6,12 @@ import ( "fmt" "io" "net/http" - "net/url" "strings" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" ) // repoFile is the resolved file content and metadata for a single path. @@ -83,9 +83,12 @@ type contentsResponse struct { // It requests the unified object media type so directories, files, symlinks, and // submodules all come back as a single JSON object distinguished by the type field. func fetchContent(httpClient *http.Client, repo ghrepo.Interface, filePath, ref string) (*contentsResponse, error) { - apiPath := contentsAPIPath(repo, filePath, ref) + apiPath, err := contentsAPIPath(repo, filePath, ref) + if err != nil { + return nil, err + } - req, err := http.NewRequest("GET", apiPath, nil) + req, err := http.NewRequest("GET", apiPath.String(), nil) if err != nil { return nil, err } @@ -171,9 +174,12 @@ func fetchFile(httpClient *http.Client, repo ghrepo.Interface, filePath, ref str // fetchRawFile retrieves the raw bytes of a file, used for files larger than the // 1 MB inline content limit of the Contents API. func fetchRawFile(httpClient *http.Client, repo ghrepo.Interface, filePath, ref string) ([]byte, error) { - apiPath := contentsAPIPath(repo, filePath, ref) + apiPath, err := contentsAPIPath(repo, filePath, ref) + if err != nil { + return nil, err + } - req, err := http.NewRequest("GET", apiPath, nil) + req, err := http.NewRequest("GET", apiPath.String(), nil) if err != nil { return nil, err } @@ -193,16 +199,15 @@ func fetchRawFile(httpClient *http.Client, repo ghrepo.Interface, filePath, ref } // contentsAPIPath builds the absolute Contents API URL for a path and optional ref. -func contentsAPIPath(repo ghrepo.Interface, filePath, ref string) string { +func contentsAPIPath(repo ghrepo.Interface, filePath, ref string) (safeurl.SafeURL, error) { // The Contents API accepts a fully percent-encoded path, including path separators // encoded as %2F, so spaces and other special characters are handled transparently. - p := fmt.Sprintf("%srepos/%s/%s/contents/%s", - ghinstance.RESTPrefix(repo.RepoHost()), - repo.RepoOwner(), repo.RepoName(), - url.PathEscape(strings.TrimPrefix(filePath, "/")), - ) + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "contents", strings.TrimPrefix(filePath, "/")) + if err != nil { + return nil, err + } if ref != "" { - p += "?ref=" + url.QueryEscape(ref) + u.SetQuery("ref", ref) } - return p + return u, nil } diff --git a/pkg/cmd/repo/read-file/read_file_test.go b/pkg/cmd/repo/read-file/read_file_test.go index aa90d15fd58..59d3856e40b 100644 --- a/pkg/cmd/repo/read-file/read_file_test.go +++ b/pkg/cmd/repo/read-file/read_file_test.go @@ -751,8 +751,9 @@ func Test_contentsAPIPath(t *testing.T) { for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { - got := contentsAPIPath(repo, tt.filePath, tt.ref) - assert.Equal(t, tt.want, got) + got, err := contentsAPIPath(repo, tt.filePath, tt.ref) + require.NoError(t, err) + assert.Equal(t, tt.want, got.String()) }) } } diff --git a/pkg/cmd/repo/sync/http.go b/pkg/cmd/repo/sync/http.go index 27e9a635169..86cc0468851 100644 --- a/pkg/cmd/repo/sync/http.go +++ b/pkg/cmd/repo/sync/http.go @@ -10,6 +10,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" ) type commit struct { @@ -25,8 +26,11 @@ type commit struct { func latestCommit(client *api.Client, repo ghrepo.Interface, branch string) (commit, error) { var response commit - path := fmt.Sprintf("repos/%s/%s/git/refs/heads/%s", repo.RepoOwner(), repo.RepoName(), branch) - err := client.REST(repo.RepoHost(), "GET", path, nil, &response) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "git", "refs", fmt.Sprintf("heads/%s", branch)) + if err != nil { + return response, err + } + err = client.REST(repo.RepoHost(), "GET", path.String(), nil, &response) return response, err } @@ -48,9 +52,12 @@ func triggerUpstreamMerge(client *api.Client, repo ghrepo.Interface, branch stri MergeType string `json:"merge_type"` BaseBranch string `json:"base_branch"` } - path := fmt.Sprintf("repos/%s/%s/merge-upstream", repo.RepoOwner(), repo.RepoName()) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "merge-upstream") + if err != nil { + return "", err + } var httpErr api.HTTPError - if err := client.REST(repo.RepoHost(), "POST", path, &payload, &response); err != nil { + if err := client.REST(repo.RepoHost(), "POST", path.String(), &payload, &response); err != nil { if errors.As(err, &httpErr) { switch httpErr.StatusCode { case http.StatusUnprocessableEntity, http.StatusConflict: @@ -66,7 +73,10 @@ func triggerUpstreamMerge(client *api.Client, repo ghrepo.Interface, branch stri } func syncFork(client *api.Client, repo ghrepo.Interface, branch, SHA string, force bool) error { - path := fmt.Sprintf("repos/%s/%s/git/refs/heads/%s", repo.RepoOwner(), repo.RepoName(), branch) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "git", "refs", fmt.Sprintf("heads/%s", branch)) + if err != nil { + return err + } body := map[string]interface{}{ "sha": SHA, "force": force, @@ -76,5 +86,5 @@ func syncFork(client *api.Client, repo ghrepo.Interface, branch, SHA string, for return err } requestBody := bytes.NewReader(requestByte) - return client.REST(repo.RepoHost(), "PATCH", path, requestBody, nil) + return client.REST(repo.RepoHost(), "PATCH", path.String(), requestBody, nil) } diff --git a/pkg/cmd/repo/sync/sync_test.go b/pkg/cmd/repo/sync/sync_test.go index 60e6ae392a3..74fa10d6ea0 100644 --- a/pkg/cmd/repo/sync/sync_test.go +++ b/pkg/cmd/repo/sync/sync_test.go @@ -306,10 +306,10 @@ func Test_SyncRun(t *testing.T) { httpmock.REST("POST", "repos/FORKOWNER/REPO-FORK/merge-upstream"), httpmock.StatusStringResponse(422, `{}`)) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/git/refs/heads/trunk"), + httpmock.REST("GET", "repos/OWNER/REPO/git/refs/heads%2Ftrunk"), httpmock.StringResponse(`{"object":{"sha":"0xDEADBEEF"}}`)) reg.Register( - httpmock.REST("PATCH", "repos/FORKOWNER/REPO-FORK/git/refs/heads/trunk"), + httpmock.REST("PATCH", "repos/FORKOWNER/REPO-FORK/git/refs/heads%2Ftrunk"), httpmock.StringResponse(`{}`)) }, wantStdout: "✓ Synced the \"FORKOWNER:trunk\" branch from \"OWNER:trunk\"\n", @@ -395,10 +395,10 @@ func Test_SyncRun(t *testing.T) { httpmock.REST("POST", "repos/OWNER/REPO-FORK/merge-upstream"), httpmock.StatusStringResponse(409, `{"message": "Merge conflict"}`)) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/git/refs/heads/trunk"), + httpmock.REST("GET", "repos/OWNER/REPO/git/refs/heads%2Ftrunk"), httpmock.StringResponse(`{"object":{"sha":"0xDEADBEEF"}}`)) reg.Register( - httpmock.REST("PATCH", "repos/OWNER/REPO-FORK/git/refs/heads/trunk"), + httpmock.REST("PATCH", "repos/OWNER/REPO-FORK/git/refs/heads%2Ftrunk"), httpmock.StringResponse(`{}`)) }, wantStdout: "✓ Synced the \"OWNER:trunk\" branch from \"OWNER:trunk\"\n", @@ -420,10 +420,10 @@ func Test_SyncRun(t *testing.T) { httpmock.REST("POST", "repos/OWNER/REPO-FORK/merge-upstream"), httpmock.StatusStringResponse(409, `{"message": "Merge conflict"}`)) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/git/refs/heads/trunk"), + httpmock.REST("GET", "repos/OWNER/REPO/git/refs/heads%2Ftrunk"), httpmock.StringResponse(`{"object":{"sha":"0xDEADBEEF"}}`)) reg.Register( - httpmock.REST("PATCH", "repos/OWNER/REPO-FORK/git/refs/heads/trunk"), + httpmock.REST("PATCH", "repos/OWNER/REPO-FORK/git/refs/heads%2Ftrunk"), func(req *http.Request) (*http.Response, error) { return &http.Response{ StatusCode: 422, @@ -453,10 +453,10 @@ func Test_SyncRun(t *testing.T) { httpmock.REST("POST", "repos/OWNER/REPO-FORK/merge-upstream"), httpmock.StatusStringResponse(409, `{"message": "Merge conflict"}`)) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/git/refs/heads/trunk"), + httpmock.REST("GET", "repos/OWNER/REPO/git/refs/heads%2Ftrunk"), httpmock.StringResponse(`{"object":{"sha":"0xDEADBEEF"}}`)) reg.Register( - httpmock.REST("PATCH", "repos/OWNER/REPO-FORK/git/refs/heads/trunk"), + httpmock.REST("PATCH", "repos/OWNER/REPO-FORK/git/refs/heads%2Ftrunk"), func(req *http.Request) (*http.Response, error) { return &http.Response{ StatusCode: 422, diff --git a/pkg/cmd/repo/view/http.go b/pkg/cmd/repo/view/http.go index 5988580e5c4..14aef095f82 100644 --- a/pkg/cmd/repo/view/http.go +++ b/pkg/cmd/repo/view/http.go @@ -10,6 +10,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/go-gh/v2/pkg/asciisanitizer" "golang.org/x/text/transform" ) @@ -30,7 +31,12 @@ func RepositoryReadme(client *http.Client, repo ghrepo.Interface, branch string) HTMLURL string `json:"html_url"` } - err := apiClient.REST(repo.RepoHost(), "GET", getReadmePath(repo, branch), nil, &response) + readmePath, err := getReadmePath(repo, branch) + if err != nil { + return nil, err + } + + err = apiClient.REST(repo.RepoHost(), "GET", readmePath.String(), nil, &response) if err != nil { var httpError api.HTTPError if errors.As(err, &httpError) && httpError.StatusCode == 404 { @@ -56,10 +62,13 @@ func RepositoryReadme(client *http.Client, repo ghrepo.Interface, branch string) }, nil } -func getReadmePath(repo ghrepo.Interface, branch string) string { - path := fmt.Sprintf("repos/%s/readme", ghrepo.FullName(repo)) +func getReadmePath(repo ghrepo.Interface, branch string) (safeurl.SafeURL, error) { + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "readme") + if err != nil { + return nil, err + } if branch != "" { - path = fmt.Sprintf("%s?ref=%s", path, branch) + path.SetQuery("ref", branch) } - return path + return path, nil } diff --git a/pkg/cmd/ruleset/check/check.go b/pkg/cmd/ruleset/check/check.go index b56476d840c..1fbbc432028 100644 --- a/pkg/cmd/ruleset/check/check.go +++ b/pkg/cmd/ruleset/check/check.go @@ -12,6 +12,7 @@ import ( "github.com/cli/cli/v2/internal/browser" "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/text" "github.com/cli/cli/v2/pkg/cmd/ruleset/shared" "github.com/cli/cli/v2/pkg/cmdutil" @@ -144,10 +145,13 @@ func checkRun(opts *CheckOptions) error { var rules []shared.RulesetRule - endpoint := fmt.Sprintf("repos/%s/%s/rules/branches/%s", repoI.RepoOwner(), repoI.RepoName(), url.PathEscape(opts.Branch)) + endpoint, err := safeurl.JoinPath("repos", repoI.RepoOwner(), repoI.RepoName(), "rules", "branches", opts.Branch) + if err != nil { + return err + } - if err = client.REST(repoI.RepoHost(), "GET", endpoint, nil, &rules); err != nil { - return fmt.Errorf("GET %s failed: %w", endpoint, err) + if err = client.REST(repoI.RepoHost(), "GET", endpoint.String(), nil, &rules); err != nil { + return fmt.Errorf("GET %s failed: %w", endpoint.String(), err) } w := opts.IO.Out diff --git a/pkg/cmd/ruleset/view/http.go b/pkg/cmd/ruleset/view/http.go index d0b26f5301b..c182917b12b 100644 --- a/pkg/cmd/ruleset/view/http.go +++ b/pkg/cmd/ruleset/view/http.go @@ -1,29 +1,35 @@ package view import ( - "fmt" "net/http" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/ruleset/shared" ) func viewRepoRuleset(httpClient *http.Client, repo ghrepo.Interface, databaseId string) (*shared.RulesetREST, error) { - path := fmt.Sprintf("repos/%s/%s/rulesets/%s", repo.RepoOwner(), repo.RepoName(), databaseId) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "rulesets", databaseId) + if err != nil { + return nil, err + } return viewRuleset(httpClient, repo.RepoHost(), path) } func viewOrgRuleset(httpClient *http.Client, orgLogin string, databaseId string, host string) (*shared.RulesetREST, error) { - path := fmt.Sprintf("orgs/%s/rulesets/%s", orgLogin, databaseId) + path, err := safeurl.JoinPath("orgs", orgLogin, "rulesets", databaseId) + if err != nil { + return nil, err + } return viewRuleset(httpClient, host, path) } -func viewRuleset(httpClient *http.Client, hostname string, path string) (*shared.RulesetREST, error) { +func viewRuleset(httpClient *http.Client, hostname string, path safeurl.SafeURL) (*shared.RulesetREST, error) { apiClient := api.NewClientFromHTTP(httpClient) result := shared.RulesetREST{} - err := apiClient.REST(hostname, "GET", path, nil, &result) + err := apiClient.REST(hostname, "GET", path.String(), nil, &result) if err != nil { return nil, err } diff --git a/pkg/cmd/run/cancel/cancel.go b/pkg/cmd/run/cancel/cancel.go index d73e46a6827..296ca9f7ba1 100644 --- a/pkg/cmd/run/cancel/cancel.go +++ b/pkg/cmd/run/cancel/cancel.go @@ -8,6 +8,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/run/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -142,14 +143,18 @@ func runCancel(opts *CancelOptions) error { } func cancelWorkflowRun(client *api.Client, repo ghrepo.Interface, runID string, force bool) error { - var path string + var path *safeurl.MutableSafeURL + var err error if force { - path = fmt.Sprintf("repos/%s/actions/runs/%s/force-cancel", ghrepo.FullName(repo), runID) + path, err = safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "runs", runID, "force-cancel") } else { - path = fmt.Sprintf("repos/%s/actions/runs/%s/cancel", ghrepo.FullName(repo), runID) + path, err = safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "runs", runID, "cancel") + } + if err != nil { + return err } - err := client.REST(repo.RepoHost(), "POST", path, nil, nil) + err = client.REST(repo.RepoHost(), "POST", path.String(), nil, nil) if err != nil { return err } diff --git a/pkg/cmd/run/delete/delete.go b/pkg/cmd/run/delete/delete.go index 711e98c0296..6fc8e17b722 100644 --- a/pkg/cmd/run/delete/delete.go +++ b/pkg/cmd/run/delete/delete.go @@ -9,6 +9,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" "github.com/cli/cli/v2/internal/prompter" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/run/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -138,8 +139,11 @@ func runDelete(opts *DeleteOptions) error { } func deleteWorkflowRun(client *api.Client, repo ghrepo.Interface, runID string) error { - path := fmt.Sprintf("repos/%s/actions/runs/%s", ghrepo.FullName(repo), runID) - err := client.REST(repo.RepoHost(), "DELETE", path, nil, nil) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "runs", runID) + if err != nil { + return err + } + err = client.REST(repo.RepoHost(), "DELETE", path.String(), nil, nil) if err != nil { return err } diff --git a/pkg/cmd/run/download/download.go b/pkg/cmd/run/download/download.go index 6190325b958..347c17251df 100644 --- a/pkg/cmd/run/download/download.go +++ b/pkg/cmd/run/download/download.go @@ -7,6 +7,7 @@ import ( "github.com/MakeNowJust/heredoc" "github.com/cli/cli/v2/internal/safepaths" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/run/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -28,7 +29,7 @@ type DownloadOptions struct { type platform interface { List(runID string) ([]shared.Artifact, error) - Download(url string, dir safepaths.Absolute) error + Download(url safeurl.SafeURL, dir safepaths.Absolute) error } type iprompter interface { @@ -187,7 +188,7 @@ func runDownload(opts *DownloadOptions) error { } } - err := opts.Platform.Download(a.DownloadURL, destDir) + err := opts.Platform.Download(safeurl.NewImmutableSafeURL(a.DownloadURL), destDir) if err != nil { return fmt.Errorf("error downloading %s: %w", a.Name, err) } diff --git a/pkg/cmd/run/download/download_test.go b/pkg/cmd/run/download/download_test.go index a90ae74b5a9..a001cd78111 100644 --- a/pkg/cmd/run/download/download_test.go +++ b/pkg/cmd/run/download/download_test.go @@ -13,6 +13,7 @@ import ( "github.com/cli/cli/v2/internal/ghrepo" "github.com/cli/cli/v2/internal/prompter" "github.com/cli/cli/v2/internal/safepaths" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/run/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -186,7 +187,7 @@ func (f *fakePlatform) List(runID string) ([]shared.Artifact, error) { return artifacts, nil } -func (f *fakePlatform) Download(url string, dir safepaths.Absolute) error { +func (f *fakePlatform) Download(url safeurl.SafeURL, dir safepaths.Absolute) error { if err := os.MkdirAll(dir.String(), 0755); err != nil { return err } @@ -197,7 +198,7 @@ func (f *fakePlatform) Download(url string, dir safepaths.Absolute) error { // Think fakePlatform { artifacts: ... } rather than fakePlatform.makeArtifactAvailable() for _, run := range f.runs { for _, testArtifact := range run.testArtifacts { - if testArtifact.artifact.DownloadURL == url { + if testArtifact.artifact.DownloadURL == url.String() { for _, file := range testArtifact.files { path := filepath.Join(dir.String(), file) return os.WriteFile(path, []byte{}, 0600) diff --git a/pkg/cmd/run/download/http.go b/pkg/cmd/run/download/http.go index 09293b056d6..a832f924b20 100644 --- a/pkg/cmd/run/download/http.go +++ b/pkg/cmd/run/download/http.go @@ -10,6 +10,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" "github.com/cli/cli/v2/internal/safepaths" + "github.com/cli/cli/v2/internal/safeurl" ghzip "github.com/cli/cli/v2/internal/zip" "github.com/cli/cli/v2/pkg/cmd/run/shared" ) @@ -23,12 +24,12 @@ func (p *apiPlatform) List(runID string) ([]shared.Artifact, error) { return shared.ListArtifacts(p.client, p.repo, runID) } -func (p *apiPlatform) Download(url string, dir safepaths.Absolute) error { +func (p *apiPlatform) Download(url safeurl.SafeURL, dir safepaths.Absolute) error { return downloadArtifact(p.client, url, dir) } -func downloadArtifact(httpClient *http.Client, url string, destDir safepaths.Absolute) error { - req, err := http.NewRequest("GET", url, nil) +func downloadArtifact(httpClient *http.Client, url safeurl.SafeURL, destDir safepaths.Absolute) error { + req, err := http.NewRequest("GET", url.String(), nil) if err != nil { return err } diff --git a/pkg/cmd/run/download/http_test.go b/pkg/cmd/run/download/http_test.go index 75b52ae790e..5b68dff82d4 100644 --- a/pkg/cmd/run/download/http_test.go +++ b/pkg/cmd/run/download/http_test.go @@ -10,6 +10,7 @@ import ( "github.com/cli/cli/v2/internal/ghrepo" "github.com/cli/cli/v2/internal/safepaths" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/httpmock" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -72,7 +73,7 @@ func Test_Download(t *testing.T) { api := &apiPlatform{ client: &http.Client{Transport: reg}, } - require.NoError(t, api.Download("https://api.github.com/repos/OWNER/REPO/actions/artifacts/12345/zip", destDir)) + require.NoError(t, api.Download(safeurl.NewImmutableSafeURL("https://api.github.com/repos/OWNER/REPO/actions/artifacts/12345/zip"), destDir)) var paths []string parentPrefix := tmpDir + string(filepath.Separator) diff --git a/pkg/cmd/run/rerun/rerun.go b/pkg/cmd/run/rerun/rerun.go index 8777e0a8a18..8f8b79a2edf 100644 --- a/pkg/cmd/run/rerun/rerun.go +++ b/pkg/cmd/run/rerun/rerun.go @@ -7,10 +7,12 @@ import ( "fmt" "io" "net/http" + "strconv" "github.com/MakeNowJust/heredoc" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/run/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -196,9 +198,12 @@ func rerunRun(client *api.Client, repo ghrepo.Interface, run *shared.Run, onlyFa return fmt.Errorf("failed to create rerun body: %w", err) } - path := fmt.Sprintf("repos/%s/actions/runs/%d/%s", ghrepo.FullName(repo), run.ID, runVerb) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "runs", strconv.FormatInt(run.ID, 10), runVerb) + if err != nil { + return err + } - err = client.REST(repo.RepoHost(), "POST", path, body, nil) + err = client.REST(repo.RepoHost(), "POST", path.String(), body, nil) if err != nil { var httpError api.HTTPError if errors.As(err, &httpError) && httpError.StatusCode == 403 { @@ -215,9 +220,12 @@ func rerunJob(client *api.Client, repo ghrepo.Interface, job *shared.Job, debug return fmt.Errorf("failed to create rerun body: %w", err) } - path := fmt.Sprintf("repos/%s/actions/jobs/%d/rerun", ghrepo.FullName(repo), job.ID) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "jobs", strconv.FormatInt(job.ID, 10), "rerun") + if err != nil { + return err + } - err = client.REST(repo.RepoHost(), "POST", path, body, nil) + err = client.REST(repo.RepoHost(), "POST", path.String(), body, nil) if err != nil { var httpError api.HTTPError if errors.As(err, &httpError) && httpError.StatusCode == 403 { diff --git a/pkg/cmd/run/shared/artifacts.go b/pkg/cmd/run/shared/artifacts.go index e835958bec9..36d0b39e73c 100644 --- a/pkg/cmd/run/shared/artifacts.go +++ b/pkg/cmd/run/shared/artifacts.go @@ -2,13 +2,14 @@ package shared import ( "encoding/json" - "fmt" "net/http" "regexp" + "strconv" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" ) type Artifact struct { @@ -25,17 +26,24 @@ type artifactsPayload struct { func ListArtifacts(httpClient *http.Client, repo ghrepo.Interface, runID string) ([]Artifact, error) { var results []Artifact + restPrefix := ghinstance.RESTPrefix(repo.RepoHost()) perPage := 100 - path := fmt.Sprintf("repos/%s/%s/actions/artifacts?per_page=%d", repo.RepoOwner(), repo.RepoName(), perPage) + u, err := safeurl.JoinPathWithHostPrefix(restPrefix, "repos", repo.RepoOwner(), repo.RepoName(), "actions", "artifacts") + if err != nil { + return nil, err + } if runID != "" { - path = fmt.Sprintf("repos/%s/%s/actions/runs/%s/artifacts?per_page=%d", repo.RepoOwner(), repo.RepoName(), runID, perPage) + u, err = safeurl.JoinPathWithHostPrefix(restPrefix, "repos", repo.RepoOwner(), repo.RepoName(), "actions", "runs", runID, "artifacts") + if err != nil { + return nil, err + } } - - url := fmt.Sprintf("%s%s", ghinstance.RESTPrefix(repo.RepoHost()), path) + u.SetQuery("per_page", strconv.Itoa(perPage)) + var pageURL safeurl.SafeURL = u for { var payload artifactsPayload - nextURL, err := apiGet(httpClient, url, &payload) + nextURL, err := apiGet(httpClient, pageURL, &payload) if err != nil { return nil, err } @@ -44,14 +52,14 @@ func ListArtifacts(httpClient *http.Client, repo ghrepo.Interface, runID string) if nextURL == "" { break } - url = nextURL + pageURL = safeurl.NewImmutableSafeURL(nextURL) } return results, nil } -func apiGet(httpClient *http.Client, url string, data interface{}) (string, error) { - req, err := http.NewRequest("GET", url, nil) +func apiGet(httpClient *http.Client, url safeurl.SafeURL, data interface{}) (string, error) { + req, err := http.NewRequest("GET", url.String(), nil) if err != nil { return "", err } diff --git a/pkg/cmd/run/shared/shared.go b/pkg/cmd/run/shared/shared.go index 8c191846a1f..6526292e24c 100644 --- a/pkg/cmd/run/shared/shared.go +++ b/pkg/cmd/run/shared/shared.go @@ -6,11 +6,13 @@ import ( "net/http" "net/url" "reflect" + "strconv" "strings" "time" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" workflowShared "github.com/cli/cli/v2/pkg/cmd/workflow/shared" "github.com/cli/cli/v2/pkg/iostreams" ) @@ -106,7 +108,7 @@ type Run struct { HeadSha string `json:"head_sha"` URL string `json:"html_url"` HeadRepository Repo `json:"head_repository"` - Jobs []Job `json:"-"` // populated by GetJobs + Jobs []Job `json:"-"` // Populated manually (separate from fetching the run) } func (r *Run) StartedTime() time.Time { @@ -280,9 +282,12 @@ var ErrMissingAnnotationsPermissions = errors.New("missing annotations permissio func GetAnnotations(client *api.Client, repo ghrepo.Interface, job Job) ([]Annotation, error) { var result []*Annotation - path := fmt.Sprintf("repos/%s/check-runs/%d/annotations", ghrepo.FullName(repo), job.ID) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "check-runs", strconv.FormatInt(job.ID, 10), "annotations") + if err != nil { + return nil, err + } - err := client.REST(repo.RepoHost(), "GET", path, nil, &result) + err = client.REST(repo.RepoHost(), "GET", path.String(), nil, &result) if err != nil { var httpError api.HTTPError if !errors.As(err, &httpError) { @@ -361,49 +366,56 @@ func GetRunsWithFilter(client *api.Client, repo ghrepo.Interface, opts *FilterOp } func GetRuns(client *api.Client, repo ghrepo.Interface, opts *FilterOptions, limit int) (*RunsPayload, error) { - path := fmt.Sprintf("repos/%s/actions/runs", ghrepo.FullName(repo)) + u, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "runs") + if err != nil { + return nil, err + } if opts != nil && opts.WorkflowID > 0 { - path = fmt.Sprintf("repos/%s/actions/workflows/%d/runs", ghrepo.FullName(repo), opts.WorkflowID) + u, err = safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "workflows", strconv.FormatInt(opts.WorkflowID, 10), "runs") + if err != nil { + return nil, err + } } perPage := limit if limit > 100 { perPage = 100 } - path += fmt.Sprintf("?per_page=%d", perPage) - path += "&exclude_pull_requests=true" // significantly reduces payload size + u.SetQuery("per_page", strconv.Itoa(perPage)) + u.SetQuery("exclude_pull_requests", "true") // significantly reduces payload size if opts != nil { if opts.Branch != "" { - path += fmt.Sprintf("&branch=%s", url.QueryEscape(opts.Branch)) + u.SetQuery("branch", opts.Branch) } if opts.Actor != "" { - path += fmt.Sprintf("&actor=%s", url.QueryEscape(opts.Actor)) + u.SetQuery("actor", opts.Actor) } if opts.Status != "" { - path += fmt.Sprintf("&status=%s", url.QueryEscape(opts.Status)) + u.SetQuery("status", opts.Status) } if opts.Event != "" { - path += fmt.Sprintf("&event=%s", url.QueryEscape(opts.Event)) + u.SetQuery("event", opts.Event) } if opts.Created != "" { - path += fmt.Sprintf("&created=%s", url.QueryEscape(opts.Created)) + u.SetQuery("created", opts.Created) } if opts.Commit != "" { - path += fmt.Sprintf("&head_sha=%s", url.QueryEscape(opts.Commit)) + u.SetQuery("head_sha", opts.Commit) } } + var pageURL safeurl.SafeURL = u var result *RunsPayload pagination: - for path != "" { + for pageURL.String() != "" { var response RunsPayload - var err error - path, err = client.RESTWithNext(repo.RepoHost(), "GET", path, nil, &response) + next, err := client.RESTWithNext(repo.RepoHost(), "GET", pageURL.String(), nil, &response) if err != nil { return nil, err } + pageURL = safeurl.NewImmutableSafeURL(next) if result == nil { result = &response @@ -475,38 +487,49 @@ type JobsPayload struct { Jobs []Job } -func GetJobs(client *api.Client, repo ghrepo.Interface, run *Run, attempt uint64) ([]Job, error) { - if run.Jobs != nil { - return run.Jobs, nil - } - - query := url.Values{} - query.Set("per_page", "100") - jobsPath := fmt.Sprintf("%s?%s", run.JobsURL, query.Encode()) - +func GetJobs(client *api.Client, repo ghrepo.Interface, runID int64, jobsURL safeurl.SafeURL, attempt uint64) ([]Job, error) { + var jobsPath safeurl.SafeURL if attempt > 0 { - jobsPath = fmt.Sprintf("repos/%s/actions/runs/%d/attempts/%d/jobs?%s", ghrepo.FullName(repo), run.ID, attempt, query.Encode()) + p, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "runs", strconv.FormatInt(runID, 10), "attempts", strconv.FormatUint(attempt, 10), "jobs") + if err != nil { + return nil, err + } + p.SetQuery("per_page", "100") + jobsPath = p + } else { + u, err := url.Parse(jobsURL.String()) + if err != nil { + return nil, err + } + query := url.Values{} + query.Set("per_page", "100") + u.RawQuery = query.Encode() + // Since u is derived from jobsURL, an already-trusted safeurl.SafeURL, the resulting URL is safe to declare as such. + jobsPath = safeurl.NewImmutableSafeURL(u.String()) } - for jobsPath != "" { + // A non-nil empty slice is returned so callers can tell that jobs were fetched and there are none (if len is zero). + jobs := []Job{} + for jobsPath.String() != "" { var resp JobsPayload - var err error - jobsPath, err = client.RESTWithNext(repo.RepoHost(), http.MethodGet, jobsPath, nil, &resp) + next, err := client.RESTWithNext(repo.RepoHost(), http.MethodGet, jobsPath.String(), nil, &resp) if err != nil { - run.Jobs = nil return nil, err } - - run.Jobs = append(run.Jobs, resp.Jobs...) + jobs = append(jobs, resp.Jobs...) + jobsPath = safeurl.NewImmutableSafeURL(next) } - return run.Jobs, nil + return jobs, nil } func GetJob(client *api.Client, repo ghrepo.Interface, jobID string) (*Job, error) { - path := fmt.Sprintf("repos/%s/actions/jobs/%s", ghrepo.FullName(repo), jobID) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "jobs", jobID) + if err != nil { + return nil, err + } var result Job - err := client.REST(repo.RepoHost(), "GET", path, nil, &result) + err = client.REST(repo.RepoHost(), "GET", path.String(), nil, &result) if err != nil { return nil, err } @@ -539,13 +562,20 @@ func SelectRun(p Prompter, cs *iostreams.ColorScheme, runs []Run) (string, error func GetRun(client *api.Client, repo ghrepo.Interface, runID string, attempt uint64) (*Run, error) { var result Run - path := fmt.Sprintf("repos/%s/actions/runs/%s?exclude_pull_requests=true", ghrepo.FullName(repo), runID) + u, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "runs", runID) + if err != nil { + return nil, err + } if attempt > 0 { - path = fmt.Sprintf("repos/%s/actions/runs/%s/attempts/%d?exclude_pull_requests=true", ghrepo.FullName(repo), runID, attempt) + u, err = safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "runs", runID, "attempts", strconv.FormatUint(attempt, 10)) + if err != nil { + return nil, err + } } + u.SetQuery("exclude_pull_requests", "true") - err := client.REST(repo.RepoHost(), "GET", path, nil, &result) + err = client.REST(repo.RepoHost(), "GET", u.String(), nil, &result) if err != nil { return nil, err } diff --git a/pkg/cmd/run/view/logs.go b/pkg/cmd/run/view/logs.go index 8961381b3b0..ab1232837bb 100644 --- a/pkg/cmd/run/view/logs.go +++ b/pkg/cmd/run/view/logs.go @@ -9,12 +9,14 @@ import ( "regexp" "slices" "sort" + "strconv" "strings" "unicode/utf16" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/run/shared" ) @@ -38,10 +40,12 @@ type apiLogFetcher struct { } func (f *apiLogFetcher) GetLog() (io.ReadCloser, error) { - logURL := fmt.Sprintf("%srepos/%s/actions/jobs/%d/logs", - ghinstance.RESTPrefix(f.repo.RepoHost()), ghrepo.FullName(f.repo), f.jobID) + logURL, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(f.repo.RepoHost()), "repos", f.repo.RepoOwner(), f.repo.RepoName(), "actions", "jobs", strconv.FormatInt(f.jobID, 10), "logs") + if err != nil { + return nil, err + } - req, err := http.NewRequest("GET", logURL, nil) + req, err := http.NewRequest("GET", logURL.String(), nil) if err != nil { return nil, err } diff --git a/pkg/cmd/run/view/view.go b/pkg/cmd/run/view/view.go index 3e5199452e2..95cdc891291 100644 --- a/pkg/cmd/run/view/view.go +++ b/pkg/cmd/run/view/view.go @@ -18,6 +18,7 @@ import ( "github.com/cli/cli/v2/internal/browser" "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/text" "github.com/cli/cli/v2/pkg/cmd/run/shared" "github.com/cli/cli/v2/pkg/cmdutil" @@ -260,11 +261,12 @@ func runView(opts *ViewOptions) error { if shouldFetchJobs(opts) { opts.IO.StartProgressIndicator() - jobs, err = shared.GetJobs(client, repo, run, attempt) + jobs, err = shared.GetJobs(client, repo, run.ID, safeurl.NewImmutableSafeURL(run.JobsURL), attempt) opts.IO.StopProgressIndicator() if err != nil { return err } + run.Jobs = jobs } if opts.Prompt && len(jobs) > 1 { @@ -298,11 +300,12 @@ func runView(opts *ViewOptions) error { if selectedJob == nil && len(jobs) == 0 { opts.IO.StartProgressIndicator() - jobs, err = shared.GetJobs(client, repo, run, attempt) + jobs, err = shared.GetJobs(client, repo, run.ID, safeurl.NewImmutableSafeURL(run.JobsURL), attempt) opts.IO.StopProgressIndicator() if err != nil { return fmt.Errorf("failed to get jobs: %w", err) } + run.Jobs = jobs } else if selectedJob != nil { jobs = []shared.Job{*selectedJob} } @@ -467,8 +470,8 @@ func shouldFetchJobs(opts *ViewOptions) bool { return false } -func getLog(httpClient *http.Client, logURL string) (io.ReadCloser, error) { - req, err := http.NewRequest("GET", logURL, nil) +func getLog(httpClient *http.Client, logURL safeurl.SafeURL) (io.ReadCloser, error) { + req, err := http.NewRequest("GET", logURL.String(), nil) if err != nil { return nil, err } @@ -496,12 +499,16 @@ func getRunLog(cache RunLogCache, httpClient *http.Client, repo ghrepo.Interface if !isCached { // Run log does not exist in cache so retrieve and store it - logURL := fmt.Sprintf("%srepos/%s/actions/runs/%d/logs", - ghinstance.RESTPrefix(repo.RepoHost()), ghrepo.FullName(repo), run.ID) + logURL, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "actions", "runs", strconv.FormatInt(run.ID, 10), "logs") + if err != nil { + return nil, err + } if attempt > 0 { - logURL = fmt.Sprintf("%srepos/%s/actions/runs/%d/attempts/%d/logs", - ghinstance.RESTPrefix(repo.RepoHost()), ghrepo.FullName(repo), run.ID, attempt) + logURL, err = safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(repo.RepoHost()), "repos", repo.RepoOwner(), repo.RepoName(), "actions", "runs", strconv.FormatInt(run.ID, 10), "attempts", strconv.FormatUint(attempt, 10), "logs") + if err != nil { + return nil, err + } } resp, err := getLog(httpClient, logURL) diff --git a/pkg/cmd/run/watch/watch.go b/pkg/cmd/run/watch/watch.go index a73a91e1a03..53ad4bc3540 100644 --- a/pkg/cmd/run/watch/watch.go +++ b/pkg/cmd/run/watch/watch.go @@ -10,6 +10,7 @@ import ( "github.com/MakeNowJust/heredoc" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/text" "github.com/cli/cli/v2/pkg/cmd/run/shared" "github.com/cli/cli/v2/pkg/cmdutil" @@ -221,10 +222,11 @@ func renderRun(out io.Writer, opts WatchOptions, client *api.Client, repo ghrepo return nil, fmt.Errorf("failed to get run: %w", err) } - jobs, err := shared.GetJobs(client, repo, run, 0) + jobs, err := shared.GetJobs(client, repo, run.ID, safeurl.NewImmutableSafeURL(run.JobsURL), 0) if err != nil { return nil, fmt.Errorf("failed to get jobs: %w", err) } + run.Jobs = jobs var annotations []shared.Annotation var missingAnnotationsPermissions bool diff --git a/pkg/cmd/secret/delete/delete.go b/pkg/cmd/secret/delete/delete.go index b1a5b1d3930..2550b8bbe2a 100644 --- a/pkg/cmd/secret/delete/delete.go +++ b/pkg/cmd/secret/delete/delete.go @@ -9,6 +9,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/secret/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -123,24 +124,27 @@ func removeRun(opts *DeleteOptions) error { return err } - var path string + var path *safeurl.MutableSafeURL var host string switch secretEntity { case shared.Organization: - path = fmt.Sprintf("orgs/%s/%s/secrets/%s", orgName, secretApp, opts.SecretName) + path, err = safeurl.JoinPath("orgs", orgName, string(secretApp), "secrets", opts.SecretName) host, _ = cfg.Authentication().DefaultHost() case shared.Environment: - path = fmt.Sprintf("repos/%s/environments/%s/secrets/%s", ghrepo.FullName(baseRepo), envName, opts.SecretName) + path, err = safeurl.JoinPath("repos", baseRepo.RepoOwner(), baseRepo.RepoName(), "environments", envName, "secrets", opts.SecretName) host = baseRepo.RepoHost() case shared.User: - path = fmt.Sprintf("user/codespaces/secrets/%s", opts.SecretName) + path, err = safeurl.JoinPath("user", "codespaces", "secrets", opts.SecretName) host, _ = cfg.Authentication().DefaultHost() case shared.Repository: - path = fmt.Sprintf("repos/%s/%s/secrets/%s", ghrepo.FullName(baseRepo), secretApp, opts.SecretName) + path, err = safeurl.JoinPath("repos", baseRepo.RepoOwner(), baseRepo.RepoName(), string(secretApp), "secrets", opts.SecretName) host = baseRepo.RepoHost() } + if err != nil { + return err + } - err = client.REST(host, "DELETE", path, nil, nil) + err = client.REST(host, "DELETE", path.String(), nil, nil) if err != nil { return fmt.Errorf("failed to delete secret %s: %w", opts.SecretName, err) } diff --git a/pkg/cmd/secret/list/list.go b/pkg/cmd/secret/list/list.go index 66334ea9152..3f47bc748e1 100644 --- a/pkg/cmd/secret/list/list.go +++ b/pkg/cmd/secret/list/list.go @@ -13,6 +13,7 @@ import ( "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghrepo" "github.com/cli/cli/v2/internal/prompter" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/tableprinter" "github.com/cli/cli/v2/pkg/cmd/secret/shared" "github.com/cli/cli/v2/pkg/cmdutil" @@ -248,30 +249,50 @@ func fmtVisibility(s Secret) string { } func getOrgSecrets(client *http.Client, host, orgName string, showSelectedRepoInfo bool, app shared.App) ([]Secret, error) { - secrets, err := getSecrets(client, host, fmt.Sprintf("orgs/%s/%s/secrets", orgName, app)) + u, err := safeurl.JoinPath("orgs", orgName, string(app), "secrets") + if err != nil { + return nil, err + } + secrets, err := getSecrets(client, host, u) if err != nil { return nil, err } if showSelectedRepoInfo { - err = populateSelectedRepositoryInformation(client, host, secrets) - if err != nil { - return nil, err + for i := range secrets { + if secrets[i].SelectedReposURL == "" { + continue + } + count, err := selectedRepositoryCount(client, host, safeurl.NewImmutableSafeURL(secrets[i].SelectedReposURL)) + if err != nil { + return nil, fmt.Errorf("failed determining selected repositories for %s: %w", secrets[i].Name, err) + } + secrets[i].NumSelectedRepos = count } } return secrets, nil } func getUserSecrets(client *http.Client, host string, showSelectedRepoInfo bool) ([]Secret, error) { - secrets, err := getSecrets(client, host, "user/codespaces/secrets") + u, err := safeurl.JoinPath("user", "codespaces", "secrets") + if err != nil { + return nil, err + } + secrets, err := getSecrets(client, host, u) if err != nil { return nil, err } if showSelectedRepoInfo { - err = populateSelectedRepositoryInformation(client, host, secrets) - if err != nil { - return nil, err + for i := range secrets { + if secrets[i].SelectedReposURL == "" { + continue + } + count, err := selectedRepositoryCount(client, host, safeurl.NewImmutableSafeURL(secrets[i].SelectedReposURL)) + if err != nil { + return nil, fmt.Errorf("failed determining selected repositories for %s: %w", secrets[i].Name, err) + } + secrets[i].NumSelectedRepos = count } } @@ -279,45 +300,47 @@ func getUserSecrets(client *http.Client, host string, showSelectedRepoInfo bool) } func getEnvSecrets(client *http.Client, repo ghrepo.Interface, envName string) ([]Secret, error) { - path := fmt.Sprintf("repos/%s/environments/%s/secrets", ghrepo.FullName(repo), envName) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "environments", envName, "secrets") + if err != nil { + return nil, err + } return getSecrets(client, repo.RepoHost(), path) } func getRepoSecrets(client *http.Client, repo ghrepo.Interface, app shared.App) ([]Secret, error) { - return getSecrets(client, repo.RepoHost(), fmt.Sprintf("repos/%s/%s/secrets", ghrepo.FullName(repo), app)) + u, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), string(app), "secrets") + if err != nil { + return nil, err + } + return getSecrets(client, repo.RepoHost(), u) } -func getSecrets(client *http.Client, host, path string) ([]Secret, error) { +func getSecrets(client *http.Client, host string, u *safeurl.MutableSafeURL) ([]Secret, error) { var results []Secret apiClient := api.NewClientFromHTTP(client) - path = fmt.Sprintf("%s?per_page=100", path) - for path != "" { + u.SetQuery("per_page", "100") + var pageURL safeurl.SafeURL = u + for pageURL.String() != "" { response := struct { Secrets []Secret }{} - var err error - path, err = apiClient.RESTWithNext(host, "GET", path, nil, &response) + next, err := apiClient.RESTWithNext(host, "GET", pageURL.String(), nil, &response) if err != nil { return nil, err } + pageURL = safeurl.NewImmutableSafeURL(next) results = append(results, response.Secrets...) } return results, nil } -func populateSelectedRepositoryInformation(client *http.Client, host string, secrets []Secret) error { +func selectedRepositoryCount(client *http.Client, host string, selectedReposURL safeurl.SafeURL) (int, error) { apiClient := api.NewClientFromHTTP(client) - for i, secret := range secrets { - if secret.SelectedReposURL == "" { - continue - } - response := struct { - TotalCount int `json:"total_count"` - }{} - if err := apiClient.REST(host, "GET", secret.SelectedReposURL, nil, &response); err != nil { - return fmt.Errorf("failed determining selected repositories for %s: %w", secret.Name, err) - } - secrets[i].NumSelectedRepos = response.TotalCount + response := struct { + TotalCount int `json:"total_count"` + }{} + if err := apiClient.REST(host, "GET", selectedReposURL.String(), nil, &response); err != nil { + return 0, err } - return nil + return response.TotalCount, nil } diff --git a/pkg/cmd/secret/list/list_test.go b/pkg/cmd/secret/list/list_test.go index da7cb892356..7e6c88a0002 100644 --- a/pkg/cmd/secret/list/list_test.go +++ b/pkg/cmd/secret/list/list_test.go @@ -16,6 +16,7 @@ import ( "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghrepo" "github.com/cli/cli/v2/internal/prompter" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/secret/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/httpmock" @@ -857,7 +858,9 @@ func Test_getSecrets_pagination(t *testing.T) { httpmock.StringResponse(`{"secrets":[{},{}]}`), ) client := &http.Client{Transport: reg} - secrets, err := getSecrets(client, "github.com", "path/to") + u, err := safeurl.JoinPath("path", "to") + require.NoError(t, err) + secrets, err := getSecrets(client, "github.com", u) assert.NoError(t, err) assert.Equal(t, 4, len(secrets)) } diff --git a/pkg/cmd/secret/set/http.go b/pkg/cmd/secret/set/http.go index 7d623be1637..43da048a65a 100644 --- a/pkg/cmd/secret/set/http.go +++ b/pkg/cmd/secret/set/http.go @@ -8,6 +8,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/secret/shared" ) @@ -30,9 +31,9 @@ type PubKey struct { Key string } -func getPubKey(client *api.Client, host, path string) (*PubKey, error) { +func getPubKey(client *api.Client, host string, path safeurl.SafeURL) (*PubKey, error) { pk := PubKey{} - err := client.REST(host, "GET", path, nil, &pk) + err := client.REST(host, "GET", path.String(), nil, &pk) if err != nil { return nil, err } @@ -40,35 +41,52 @@ func getPubKey(client *api.Client, host, path string) (*PubKey, error) { } func getOrgPublicKey(client *api.Client, host, orgName string, app shared.App) (*PubKey, error) { - return getPubKey(client, host, fmt.Sprintf("orgs/%s/%s/secrets/public-key", orgName, app)) + u, err := safeurl.JoinPath("orgs", orgName, string(app), "secrets", "public-key") + if err != nil { + return nil, err + } + return getPubKey(client, host, u) } func getUserPublicKey(client *api.Client, host string) (*PubKey, error) { - return getPubKey(client, host, "user/codespaces/secrets/public-key") + u, err := safeurl.JoinPath("user", "codespaces", "secrets", "public-key") + if err != nil { + return nil, err + } + return getPubKey(client, host, u) } func getRepoPubKey(client *api.Client, repo ghrepo.Interface, app shared.App) (*PubKey, error) { - return getPubKey(client, repo.RepoHost(), fmt.Sprintf("repos/%s/%s/secrets/public-key", - ghrepo.FullName(repo), app)) + u, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), string(app), "secrets", "public-key") + if err != nil { + return nil, err + } + return getPubKey(client, repo.RepoHost(), u) } func getEnvPubKey(client *api.Client, repo ghrepo.Interface, envName string) (*PubKey, error) { - return getPubKey(client, repo.RepoHost(), fmt.Sprintf("repos/%s/environments/%s/secrets/public-key", - ghrepo.FullName(repo), envName)) + u, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "environments", envName, "secrets", "public-key") + if err != nil { + return nil, err + } + return getPubKey(client, repo.RepoHost(), u) } -func putSecret(client *api.Client, host, path string, payload interface{}) error { +func putSecret(client *api.Client, host string, path safeurl.SafeURL, payload interface{}) error { payloadBytes, err := json.Marshal(payload) if err != nil { return fmt.Errorf("failed to serialize: %w", err) } requestBody := bytes.NewReader(payloadBytes) - return client.REST(host, "PUT", path, requestBody, nil) + return client.REST(host, "PUT", path.String(), requestBody, nil) } func putOrgSecret(client *api.Client, host string, pk *PubKey, orgName, visibility, secretName, eValue string, repositoryIDs []int64, app shared.App) error { - path := fmt.Sprintf("orgs/%s/%s/secrets/%s", orgName, app, secretName) + path, err := safeurl.JoinPath("orgs", orgName, string(app), "secrets", secretName) + if err != nil { + return err + } if app == shared.Dependabot { repos := make([]string, len(repositoryIDs)) @@ -102,7 +120,10 @@ func putUserSecret(client *api.Client, host string, pk *PubKey, key, eValue stri KeyID: pk.ID, Repositories: repositoryIDs, } - path := fmt.Sprintf("user/codespaces/secrets/%s", key) + path, err := safeurl.JoinPath("user", "codespaces", "secrets", key) + if err != nil { + return err + } return putSecret(client, host, path, payload) } @@ -111,7 +132,10 @@ func putEnvSecret(client *api.Client, pk *PubKey, repo ghrepo.Interface, envName EncryptedValue: eValue, KeyID: pk.ID, } - path := fmt.Sprintf("repos/%s/environments/%s/secrets/%s", ghrepo.FullName(repo), envName, secretName) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "environments", envName, "secrets", secretName) + if err != nil { + return err + } return putSecret(client, repo.RepoHost(), path, payload) } @@ -120,6 +144,9 @@ func putRepoSecret(client *api.Client, pk *PubKey, repo ghrepo.Interface, secret EncryptedValue: eValue, KeyID: pk.ID, } - path := fmt.Sprintf("repos/%s/%s/secrets/%s", ghrepo.FullName(repo), app, secretName) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), string(app), "secrets", secretName) + if err != nil { + return err + } return putSecret(client, repo.RepoHost(), path, payload) } diff --git a/pkg/cmd/skills/install/install.go b/pkg/cmd/skills/install/install.go index 7a7d3f17a3f..b56b4eeb6cc 100644 --- a/pkg/cmd/skills/install/install.go +++ b/pkg/cmd/skills/install/install.go @@ -19,6 +19,7 @@ import ( "github.com/cli/cli/v2/internal/ghinstance" "github.com/cli/cli/v2/internal/ghrepo" "github.com/cli/cli/v2/internal/prompter" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/skills/discovery" "github.com/cli/cli/v2/internal/skills/frontmatter" "github.com/cli/cli/v2/internal/skills/installer" @@ -1296,14 +1297,16 @@ func filterHiddenDirSkills(opts *InstallOptions, allSkills []discovery.Skill) ([ // installs from the re-publisher. // Returns (repo to redirect to, whether upstream was detected, error). func checkUpstreamProvenance(opts *InstallOptions, client *api.Client, hostname string, skill discovery.Skill, commitSHA string) (ghrepo.Interface, bool, error) { - apiPath := fmt.Sprintf("repos/%s/%s/contents/%s?ref=%s", - opts.repo.RepoOwner(), opts.repo.RepoName(), - skill.Path+"/SKILL.md", commitSHA) + u, err := safeurl.JoinPath("repos", opts.repo.RepoOwner(), opts.repo.RepoName(), "contents", skill.Path+"/SKILL.md") + if err != nil { + return nil, false, err + } + u.SetQuery("ref", commitSHA) var fileResp struct { Content string `json:"content"` Encoding string `json:"encoding"` } - if err := client.REST(hostname, "GET", apiPath, nil, &fileResp); err != nil { + if err := client.REST(hostname, "GET", u.String(), nil, &fileResp); err != nil { return nil, false, nil //nolint:nilerr // best-effort check; failing to fetch is not fatal } if fileResp.Encoding != "base64" { diff --git a/pkg/cmd/skills/install/install_test.go b/pkg/cmd/skills/install/install_test.go index 070f7ac4230..9a78da48042 100644 --- a/pkg/cmd/skills/install/install_test.go +++ b/pkg/cmd/skills/install/install_test.go @@ -221,7 +221,7 @@ func stubResolveVersion(reg *httpmock.Registry, owner, repo, tag, sha string) { httpmock.StringResponse(fmt.Sprintf(`{"tag_name": %q}`, tag)), ) reg.Register( - httpmock.REST("GET", fmt.Sprintf("repos/%s/%s/git/ref/tags/%s", owner, repo, tag)), + httpmock.REST("GET", fmt.Sprintf("repos/%s/%s/git/ref/tags%%2F%s", owner, repo, tag)), httpmock.StringResponse(fmt.Sprintf(`{"object": {"sha": %q, "type": "commit"}}`, sha)), ) } @@ -656,10 +656,10 @@ func TestInstallRun(t *testing.T) { isTTY: true, stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/skills-repo/git/ref/heads/v2.0.0"), + httpmock.REST("GET", "repos/monalisa/skills-repo/git/ref/heads%2Fv2.0.0"), httpmock.StatusStringResponse(404, "not found")) reg.Register( - httpmock.REST("GET", "repos/monalisa/skills-repo/git/ref/tags/v2.0.0"), + httpmock.REST("GET", "repos/monalisa/skills-repo/git/ref/tags%2Fv2.0.0"), httpmock.StringResponse(`{"object": {"sha": "def456", "type": "commit"}}`), ) stubDiscoverTree(reg, "monalisa", "skills-repo", "def456", @@ -766,10 +766,10 @@ func TestInstallRun(t *testing.T) { isTTY: true, stubs: func(reg *httpmock.Registry) { reg.Register( - httpmock.REST("GET", "repos/monalisa/skills-repo/git/ref/heads/v1.2.0"), + httpmock.REST("GET", "repos/monalisa/skills-repo/git/ref/heads%2Fv1.2.0"), httpmock.StatusStringResponse(404, "not found")) reg.Register( - httpmock.REST("GET", "repos/monalisa/skills-repo/git/ref/tags/v1.2.0"), + httpmock.REST("GET", "repos/monalisa/skills-repo/git/ref/tags%2Fv1.2.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) stubDiscoverTree(reg, "monalisa", "skills-repo", "abc123", @@ -2716,7 +2716,7 @@ var republishedContent = heredoc.Doc(` func stubContentsAPI(reg *httpmock.Registry, owner, repo, path, content string) { encoded := base64.StdEncoding.EncodeToString([]byte(content)) reg.Register( - httpmock.REST("GET", fmt.Sprintf("repos/%s/%s/contents/%s", owner, repo, path)), + httpmock.REST("GET", fmt.Sprintf("repos/%s/%s/contents/%s", owner, repo, url.PathEscape(path))), httpmock.StringResponse(fmt.Sprintf(`{"content": %q, "encoding": "base64"}`, encoded)), ) } diff --git a/pkg/cmd/skills/preview/preview_test.go b/pkg/cmd/skills/preview/preview_test.go index 04fae62587e..1ae93026bd7 100644 --- a/pkg/cmd/skills/preview/preview_test.go +++ b/pkg/cmd/skills/preview/preview_test.go @@ -142,7 +142,7 @@ func TestPreviewRun(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/github/awesome-copilot/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/github/awesome-copilot/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -185,7 +185,7 @@ func TestPreviewRun(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -229,7 +229,7 @@ func TestPreviewRun(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -274,7 +274,7 @@ func TestPreviewRun(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -311,7 +311,7 @@ func TestPreviewRun(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -340,7 +340,7 @@ func TestPreviewRun(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -368,11 +368,11 @@ func TestPreviewRun(t *testing.T) { httpStubs: func(reg *httpmock.Registry) { // ResolveRef with explicit version tries branch first, then tag, then commit reg.Register( - httpmock.REST("GET", "repos/github/awesome-copilot/git/ref/heads/abc123def456"), + httpmock.REST("GET", "repos/github/awesome-copilot/git/ref/heads%2Fabc123def456"), httpmock.StatusStringResponse(404, "not found"), ) reg.Register( - httpmock.REST("GET", "repos/github/awesome-copilot/git/ref/tags/abc123def456"), + httpmock.REST("GET", "repos/github/awesome-copilot/git/ref/tags%2Fabc123def456"), httpmock.StatusStringResponse(404, "not found"), ) reg.Register( @@ -464,7 +464,7 @@ func TestPreviewRun_Interactive(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -539,7 +539,7 @@ func TestPreviewRun_ShowsFileTree(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -628,7 +628,7 @@ func TestPreviewRun_ShowsFileTree(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -777,7 +777,7 @@ func TestPreviewRun_RenderLimits(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/monalisa/skills-repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/monalisa/skills-repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -916,7 +916,7 @@ func TestPreviewRun_InteractiveTelemetryCapturesSelectedSkillName(t *testing.T) httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -1028,7 +1028,7 @@ func TestPreviewRun_TelemetryVisibility(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -1223,7 +1223,7 @@ func TestPreviewRun_HiddenDirSkillsExcluded(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -1271,7 +1271,7 @@ func TestPreviewRun_HiddenDirSkillsExcluded(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( @@ -1329,7 +1329,7 @@ func TestPreviewRun_HiddenDirSkillsExcluded(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/owner/repo/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/owner/repo/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "abc123", "type": "commit"}}`), ) reg.Register( diff --git a/pkg/cmd/skills/publish/publish.go b/pkg/cmd/skills/publish/publish.go index a9fbd462f5e..9c5c0f5e2f5 100644 --- a/pkg/cmd/skills/publish/publish.go +++ b/pkg/cmd/skills/publish/publish.go @@ -20,6 +20,7 @@ import ( "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghrepo" "github.com/cli/cli/v2/internal/prompter" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/skills/discovery" "github.com/cli/cli/v2/internal/skills/frontmatter" "github.com/cli/cli/v2/internal/skills/registry" @@ -443,9 +444,12 @@ func repoHasTopic(client *api.Client, host, owner, repo string) bool { if client == nil { return false } - apiPath := fmt.Sprintf("repos/%s/%s/topics", owner, repo) + apiPath, err := safeurl.JoinPath("repos", owner, repo, "topics") + if err != nil { + return false + } var resp repoTopicsResponse - if err := client.REST(host, "GET", apiPath, nil, &resp); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &resp); err != nil { return false } for _, t := range resp.Names { @@ -461,9 +465,13 @@ func fetchTags(client *api.Client, host, owner, repo string) []tagEntry { if client == nil { return nil } - apiPath := fmt.Sprintf("repos/%s/%s/tags?per_page=10", owner, repo) + u, err := safeurl.JoinPath("repos", owner, repo, "tags") + if err != nil { + return nil + } + u.SetQuery("per_page", "10") var tags []tagEntry - if err := client.REST(host, "GET", apiPath, nil, &tags); err != nil { + if err := client.REST(host, "GET", u.String(), nil, &tags); err != nil { return nil } return tags @@ -609,11 +617,14 @@ func runPublishRelease(opts *PublishOptions, client *api.Client, host, owner, re return fmt.Errorf("failed to serialize release request: %w", err) } - releasePath := fmt.Sprintf("repos/%s/%s/releases", owner, repo) + releasePath, err := safeurl.JoinPath("repos", owner, repo, "releases") + if err != nil { + return err + } var releaseResp struct { HTMLURL string `json:"html_url"` } - if err := client.REST(host, "POST", releasePath, bytes.NewReader(releaseJSON), &releaseResp); err != nil { + if err := client.REST(host, "POST", releasePath.String(), bytes.NewReader(releaseJSON), &releaseResp); err != nil { return fmt.Errorf("failed to create release: %w", err) } @@ -683,7 +694,11 @@ func detectDefaultBranch(client *api.Client, host, owner, repo string) string { var result struct { DefaultBranch string `json:"default_branch"` } - if err := client.REST(host, "GET", fmt.Sprintf("repos/%s/%s", owner, repo), nil, &result); err != nil { + apiPath, err := safeurl.JoinPath("repos", owner, repo) + if err != nil { + return "" + } + if err := client.REST(host, "GET", apiPath.String(), nil, &result); err != nil { return "" } return result.DefaultBranch @@ -691,11 +706,14 @@ func detectDefaultBranch(client *api.Client, host, owner, repo string) string { // addAgentSkillsTopic adds the "agent-skills" topic to the repo, preserving existing topics. func addAgentSkillsTopic(client *api.Client, host, owner, repo string) error { - apiPath := fmt.Sprintf("repos/%s/%s/topics", owner, repo) + apiPath, err := safeurl.JoinPath("repos", owner, repo, "topics") + if err != nil { + return err + } // Fetch existing topics var resp repoTopicsResponse - if err := client.REST(host, "GET", apiPath, nil, &resp); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &resp); err != nil { return fmt.Errorf("could not fetch existing topics: %w", err) } @@ -711,7 +729,7 @@ func addAgentSkillsTopic(client *api.Client, host, owner, repo string) error { if err != nil { return fmt.Errorf("could not serialize topics: %w", err) } - return client.REST(host, "PUT", apiPath, bytes.NewReader(topicsJSON), nil) + return client.REST(host, "PUT", apiPath.String(), bytes.NewReader(topicsJSON), nil) } // checkImmutableReleases checks if immutable releases are enabled for the repo. @@ -719,11 +737,14 @@ func checkImmutableReleases(client *api.Client, host, owner, repo string) bool { if client == nil { return false } - apiPath := fmt.Sprintf("repos/%s/%s/immutable-releases", owner, repo) + apiPath, err := safeurl.JoinPath("repos", owner, repo, "immutable-releases") + if err != nil { + return false + } var resp struct { Enabled bool `json:"enabled"` } - if err := client.REST(host, "GET", apiPath, nil, &resp); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &resp); err != nil { return false } return resp.Enabled @@ -731,9 +752,12 @@ func checkImmutableReleases(client *api.Client, host, owner, repo string) bool { // enableImmutableReleases enables immutable releases for the repo. func enableImmutableReleases(client *api.Client, host, owner, repo string) error { - apiPath := fmt.Sprintf("repos/%s/%s/immutable-releases", owner, repo) + apiPath, err := safeurl.JoinPath("repos", owner, repo, "immutable-releases") + if err != nil { + return err + } body := bytes.NewReader([]byte(`{"enabled":true}`)) - return client.REST(host, "PATCH", apiPath, body, nil) + return client.REST(host, "PATCH", apiPath.String(), body, nil) } // checkTagProtection checks whether tag protection rulesets are enabled. @@ -741,9 +765,12 @@ func checkTagProtection(client *api.Client, host, owner, repo string) []publishD if client == nil { return nil } - apiPath := fmt.Sprintf("repos/%s/%s/rulesets", owner, repo) + apiPath, err := safeurl.JoinPath("repos", owner, repo, "rulesets") + if err != nil { + return nil + } var rulesets []rulesetsResponse - if err := client.REST(host, "GET", apiPath, nil, &rulesets); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &rulesets); err != nil { return nil } @@ -764,9 +791,12 @@ func checkSecuritySettings(client *api.Client, host, owner, repo string, skillDi if client == nil { return nil } - apiPath := fmt.Sprintf("repos/%s/%s", owner, repo) + apiPath, err := safeurl.JoinPath("repos", owner, repo) + if err != nil { + return nil + } var resp repoSecurityResponse - if err := client.REST(host, "GET", apiPath, nil, &resp); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &resp); err != nil { return nil } @@ -794,22 +824,26 @@ func checkSecuritySettings(client *api.Client, host, owner, repo string, skillDi hasCode, hasManifests := detectCodeAndManifests(skillDirs) if hasCode { - alertsPath := fmt.Sprintf("repos/%s/%s/code-scanning/alerts?per_page=1&state=open", owner, repo) - if err := client.REST(host, "GET", alertsPath, nil, new([]interface{})); err != nil { - diagnostics = append(diagnostics, publishDiagnostic{ - severity: "info", - message: "skills include code files but code scanning does not appear to be configured (Settings > Code security > Code scanning)", - }) + if u, err := safeurl.JoinPath("repos", owner, repo, "code-scanning", "alerts"); err == nil { + u.SetQuery("per_page", "1") + u.SetQuery("state", "open") + if err := client.REST(host, "GET", u.String(), nil, new([]interface{})); err != nil { + diagnostics = append(diagnostics, publishDiagnostic{ + severity: "info", + message: "skills include code files but code scanning does not appear to be configured (Settings > Code security > Code scanning)", + }) + } } } if hasManifests { - dependabotPath := fmt.Sprintf("repos/%s/%s/vulnerability-alerts", owner, repo) - if err := client.REST(host, "GET", dependabotPath, nil, nil); err != nil { - diagnostics = append(diagnostics, publishDiagnostic{ - severity: "info", - message: "skills include dependency manifests but Dependabot alerts do not appear to be enabled (Settings > Code security > Dependabot)", - }) + if dependabotPath, err := safeurl.JoinPath("repos", owner, repo, "vulnerability-alerts"); err == nil { + if err := client.REST(host, "GET", dependabotPath.String(), nil, nil); err != nil { + diagnostics = append(diagnostics, publishDiagnostic{ + severity: "info", + message: "skills include dependency manifests but Dependabot alerts do not appear to be enabled (Settings > Code security > Dependabot)", + }) + } } } diff --git a/pkg/cmd/skills/search/search.go b/pkg/cmd/skills/search/search.go index 074e338ae72..1a5353d59eb 100644 --- a/pkg/cmd/skills/search/search.go +++ b/pkg/cmd/skills/search/search.go @@ -5,10 +5,10 @@ import ( "fmt" "math" "net/http" - "net/url" "os" "os/exec" "sort" + "strconv" "strings" "sync" @@ -17,6 +17,7 @@ import ( "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/gh/ghtelemetry" "github.com/cli/cli/v2/internal/prompter" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/skills/discovery" "github.com/cli/cli/v2/internal/skills/frontmatter" "github.com/cli/cli/v2/internal/skills/registry" @@ -733,10 +734,15 @@ const rateLimitErrorMessage = "GitHub API rate limit exceeded. Please wait a min // executeSearch performs a single GitHub Code Search API call. func executeSearch(client *api.Client, host, query string, page, pageSize int) (*codeSearchResult, error) { - apiPath := fmt.Sprintf("search/code?q=%s&per_page=%d&page=%d", - url.QueryEscape(query), pageSize, page) + apiPath, err := safeurl.JoinPath("search", "code") + if err != nil { + return nil, err + } + apiPath.SetQuery("q", query) + apiPath.SetQuery("per_page", strconv.Itoa(pageSize)) + apiPath.SetQuery("page", strconv.Itoa(page)) var result codeSearchResult - err := client.REST(host, "GET", apiPath, nil, &result) + err = client.REST(host, "GET", apiPath.String(), nil, &result) if err != nil && isRateLimitError(err) { return nil, fmt.Errorf("%s", rateLimitErrorMessage) } @@ -914,9 +920,12 @@ func fetchRepoStars(client *api.Client, host string, skills []skillResult) map[i sem <- struct{}{} defer func() { <-sem }() - apiPath := fmt.Sprintf("repos/%s/%s", owner, repo) + apiPath, err := safeurl.JoinPath("repos", owner, repo) + if err != nil { + return + } var info repoInfo - if err := client.REST(host, "GET", apiPath, nil, &info); err != nil { + if err := client.REST(host, "GET", apiPath.String(), nil, &info); err != nil { return } mu.Lock() diff --git a/pkg/cmd/skills/update/update_test.go b/pkg/cmd/skills/update/update_test.go index 7a4d4ab18e7..cb6caac6306 100644 --- a/pkg/cmd/skills/update/update_test.go +++ b/pkg/cmd/skills/update/update_test.go @@ -345,7 +345,7 @@ func TestUpdateRun(t *testing.T) { httpmock.REST("GET", "repos/monalisa/octocat-skills/releases/latest"), httpmock.StringResponse(`{"tag_name": "v1.0.0"}`)) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "commit1", "type": "commit"}}`)) reg.Register( httpmock.REST("GET", "repos/monalisa/octocat-skills/git/trees/commit1"), @@ -532,7 +532,7 @@ func TestUpdateRun(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v1.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "commitsha123", "type": "commit"}}`), ) reg.Register( @@ -576,7 +576,7 @@ func TestUpdateRun(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v2.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/hubot/octocat-skills/git/ref/tags/v2.0.0"), + httpmock.REST("GET", "repos/hubot/octocat-skills/git/ref/tags%2Fv2.0.0"), httpmock.StringResponse(`{"object": {"sha": "newcommit456", "type": "commit"}}`), ) reg.Register( @@ -624,7 +624,7 @@ func TestUpdateRun(t *testing.T) { httpmock.StringResponse(`{"tag_name": "v2.0.0"}`), ) reg.Register( - httpmock.REST("GET", "repos/hubot/octocat-skills/git/ref/tags/v2.0.0"), + httpmock.REST("GET", "repos/hubot/octocat-skills/git/ref/tags%2Fv2.0.0"), httpmock.StringResponse(`{"object": {"sha": "newcommit456", "type": "commit"}}`), ) reg.Register( @@ -672,7 +672,7 @@ func TestUpdateRun(t *testing.T) { httpmock.REST("GET", "repos/monalisa/octocat-skills/releases/latest"), httpmock.StringResponse(`{"tag_name": "v3.0.0"}`)) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v3.0.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv3.0.0"), httpmock.StringResponse(`{"object": {"sha": "newcommit789", "type": "commit"}}`)) reg.Register( httpmock.REST("GET", "repos/monalisa/octocat-skills/git/trees/newcommit789"), @@ -735,7 +735,7 @@ func TestUpdateRun(t *testing.T) { httpmock.REST("GET", "repos/monalisa/octocat-skills/releases/latest"), httpmock.StringResponse(`{"tag_name": "v3.0.0"}`)) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v3.0.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv3.0.0"), httpmock.StringResponse(`{"object": {"sha": "newcommit789", "type": "commit"}}`)) reg.Register( httpmock.REST("GET", "repos/monalisa/octocat-skills/git/trees/newcommit789"), @@ -806,7 +806,7 @@ func TestUpdateRun(t *testing.T) { httpmock.REST("GET", "repos/monalisa/octocat-skills/releases/latest"), httpmock.StringResponse(`{"tag_name": "v3.0.0"}`)) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v3.0.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv3.0.0"), httpmock.StringResponse(`{"object": {"sha": "newcommit789", "type": "commit"}}`)) reg.Register( httpmock.REST("GET", "repos/monalisa/octocat-skills/git/trees/newcommit789"), @@ -865,7 +865,7 @@ func TestUpdateRun(t *testing.T) { httpmock.REST("GET", "repos/monalisa/octocat-skills/releases/latest"), httpmock.StringResponse(`{"tag_name": "v3.0.0"}`)) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v3.0.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv3.0.0"), httpmock.StringResponse(`{"object": {"sha": "newcommit789", "type": "commit"}}`)) reg.Register( httpmock.REST("GET", "repos/monalisa/octocat-skills/git/trees/newcommit789"), @@ -927,7 +927,7 @@ func TestUpdateRun(t *testing.T) { httpmock.REST("GET", "repos/monalisa/octocat-skills/releases/latest"), httpmock.StringResponse(`{"tag_name": "v3.0.0"}`)) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v3.0.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv3.0.0"), httpmock.StringResponse(`{"object": {"sha": "newcommit789", "type": "commit"}}`)) reg.Register( httpmock.REST("GET", "repos/monalisa/octocat-skills/git/trees/newcommit789"), @@ -1011,7 +1011,7 @@ func TestUpdateRun(t *testing.T) { httpmock.REST("GET", "repos/monalisa/octocat-skills/releases/latest"), httpmock.StringResponse(`{"tag_name": "v1.0.0"}`)) reg.Register( - httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags/v1.0.0"), + httpmock.REST("GET", "repos/monalisa/octocat-skills/git/ref/tags%2Fv1.0.0"), httpmock.StringResponse(`{"object": {"sha": "commit123", "type": "commit"}}`)) reg.Register( httpmock.REST("GET", "repos/monalisa/octocat-skills/git/trees/commit123"), @@ -1081,7 +1081,7 @@ func TestUpdateRun(t *testing.T) { httpmock.REST("GET", "repos/octocat/hubot-skills/releases/latest"), httpmock.StringResponse(`{"tag_name": "v2.0.0"}`)) reg.Register( - httpmock.REST("GET", "repos/octocat/hubot-skills/git/ref/tags/v2.0.0"), + httpmock.REST("GET", "repos/octocat/hubot-skills/git/ref/tags%2Fv2.0.0"), httpmock.StringResponse(`{"object": {"sha": "newcommit789", "type": "commit"}}`)) reg.Register( httpmock.REST("GET", "repos/octocat/hubot-skills/git/trees/newcommit789"), @@ -1174,7 +1174,7 @@ func TestUpdateRun(t *testing.T) { httpmock.REST("GET", "repos/octocat/hubot-skills/releases/latest"), httpmock.StringResponse(`{"tag_name": "v2.0.0"}`)) reg.Register( - httpmock.REST("GET", "repos/octocat/hubot-skills/git/ref/tags/v2.0.0"), + httpmock.REST("GET", "repos/octocat/hubot-skills/git/ref/tags%2Fv2.0.0"), httpmock.StringResponse(`{"object": {"sha": "newcommit789", "type": "commit"}}`)) reg.Register( httpmock.REST("GET", "repos/octocat/hubot-skills/git/trees/newcommit789"), diff --git a/pkg/cmd/ssh-key/add/http.go b/pkg/cmd/ssh-key/add/http.go index 83aa77bdc86..1efe1d34197 100644 --- a/pkg/cmd/ssh-key/add/http.go +++ b/pkg/cmd/ssh-key/add/http.go @@ -10,12 +10,16 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/ssh-key/shared" ) // Uploads the provided SSH key. Returns true if the key was uploaded, false if it was not. func SSHKeyUpload(httpClient *http.Client, hostname string, keyFile io.Reader, title string) (bool, error) { - url := ghinstance.RESTPrefix(hostname) + "user/keys" + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(hostname), "user", "keys") + if err != nil { + return false, err + } keyBytes, err := io.ReadAll(keyFile) if err != nil { @@ -46,7 +50,7 @@ func SSHKeyUpload(httpClient *http.Client, hostname string, keyFile io.Reader, t "key": fullUserKey, } - err = keyUpload(httpClient, url, payload) + err = keyUpload(httpClient, u, payload) if err != nil { return false, err @@ -57,7 +61,10 @@ func SSHKeyUpload(httpClient *http.Client, hostname string, keyFile io.Reader, t // Uploads the provided SSH Signing key. Returns true if the key was uploaded, false if it was not. func SSHSigningKeyUpload(httpClient *http.Client, hostname string, keyFile io.Reader, title string) (bool, error) { - url := ghinstance.RESTPrefix(hostname) + "user/ssh_signing_keys" + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(hostname), "user", "ssh_signing_keys") + if err != nil { + return false, err + } keyBytes, err := io.ReadAll(keyFile) if err != nil { @@ -88,7 +95,7 @@ func SSHSigningKeyUpload(httpClient *http.Client, hostname string, keyFile io.Re "key": fullUserKey, } - err = keyUpload(httpClient, url, payload) + err = keyUpload(httpClient, u, payload) if err != nil { return false, err @@ -97,13 +104,13 @@ func SSHSigningKeyUpload(httpClient *http.Client, hostname string, keyFile io.Re return true, nil } -func keyUpload(httpClient *http.Client, url string, payload map[string]string) error { +func keyUpload(httpClient *http.Client, u safeurl.SafeURL, payload map[string]string) error { payloadBytes, err := json.Marshal(payload) if err != nil { return err } - req, err := http.NewRequest("POST", url, bytes.NewBuffer(payloadBytes)) + req, err := http.NewRequest("POST", u.String(), bytes.NewBuffer(payloadBytes)) if err != nil { return err } diff --git a/pkg/cmd/ssh-key/delete/http.go b/pkg/cmd/ssh-key/delete/http.go index 906ae6bc906..a23502e6489 100644 --- a/pkg/cmd/ssh-key/delete/http.go +++ b/pkg/cmd/ssh-key/delete/http.go @@ -2,12 +2,12 @@ package delete import ( "encoding/json" - "fmt" "io" "net/http" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" + "github.com/cli/cli/v2/internal/safeurl" ) type sshKey struct { @@ -15,8 +15,11 @@ type sshKey struct { } func deleteSSHKey(httpClient *http.Client, host string, keyID string) error { - url := fmt.Sprintf("%suser/keys/%s", ghinstance.RESTPrefix(host), keyID) - req, err := http.NewRequest("DELETE", url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(host), "user", "keys", keyID) + if err != nil { + return err + } + req, err := http.NewRequest("DELETE", url.String(), nil) if err != nil { return err } @@ -35,8 +38,11 @@ func deleteSSHKey(httpClient *http.Client, host string, keyID string) error { } func getSSHKey(httpClient *http.Client, host string, keyID string) (*sshKey, error) { - url := fmt.Sprintf("%suser/keys/%s", ghinstance.RESTPrefix(host), keyID) - req, err := http.NewRequest("GET", url, nil) + url, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(host), "user", "keys", keyID) + if err != nil { + return nil, err + } + req, err := http.NewRequest("GET", url.String(), nil) if err != nil { return nil, err } diff --git a/pkg/cmd/ssh-key/shared/user_keys.go b/pkg/cmd/ssh-key/shared/user_keys.go index 6a3d286ab62..4f1553afba8 100644 --- a/pkg/cmd/ssh-key/shared/user_keys.go +++ b/pkg/cmd/ssh-key/shared/user_keys.go @@ -2,13 +2,13 @@ package shared import ( "encoding/json" - "fmt" "io" "net/http" "time" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghinstance" + "github.com/cli/cli/v2/internal/safeurl" ) const ( @@ -25,13 +25,19 @@ type sshKey struct { } func UserKeys(httpClient *http.Client, host, userHandle string) ([]sshKey, error) { - resource := "user/keys" + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(host), "user", "keys") + if err != nil { + return nil, err + } if userHandle != "" { - resource = fmt.Sprintf("users/%s/keys", userHandle) + u, err = safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(host), "users", userHandle, "keys") + if err != nil { + return nil, err + } } - url := fmt.Sprintf("%s%s?per_page=%d", ghinstance.RESTPrefix(host), resource, 100) + u.SetQuery("per_page", "100") - keys, err := getUserKeys(httpClient, url) + keys, err := getUserKeys(httpClient, u) if err != nil { return nil, err @@ -45,13 +51,19 @@ func UserKeys(httpClient *http.Client, host, userHandle string) ([]sshKey, error } func UserSigningKeys(httpClient *http.Client, host, userHandle string) ([]sshKey, error) { - resource := "user/ssh_signing_keys" + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(host), "user", "ssh_signing_keys") + if err != nil { + return nil, err + } if userHandle != "" { - resource = fmt.Sprintf("users/%s/ssh_signing_keys", userHandle) + u, err = safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(host), "users", userHandle, "ssh_signing_keys") + if err != nil { + return nil, err + } } - url := fmt.Sprintf("%s%s?per_page=%d", ghinstance.RESTPrefix(host), resource, 100) + u.SetQuery("per_page", "100") - keys, err := getUserKeys(httpClient, url) + keys, err := getUserKeys(httpClient, u) if err != nil { return nil, err @@ -64,8 +76,8 @@ func UserSigningKeys(httpClient *http.Client, host, userHandle string) ([]sshKey return keys, nil } -func getUserKeys(httpClient *http.Client, url string) ([]sshKey, error) { - req, err := http.NewRequest("GET", url, nil) +func getUserKeys(httpClient *http.Client, u safeurl.SafeURL) ([]sshKey, error) { + req, err := http.NewRequest("GET", u.String(), nil) if err != nil { return nil, err } diff --git a/pkg/cmd/status/status.go b/pkg/cmd/status/status.go index c9acce8bd69..6dbd4199986 100644 --- a/pkg/cmd/status/status.go +++ b/pkg/cmd/status/status.go @@ -6,8 +6,8 @@ import ( "errors" "fmt" "net/http" - "net/url" "sort" + "strconv" "strings" "sync" "time" @@ -15,6 +15,7 @@ import ( "github.com/MakeNowJust/heredoc" "github.com/charmbracelet/lipgloss" "github.com/cli/cli/v2/api" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/tableprinter" "github.com/cli/cli/v2/pkg/cmd/factory" "github.com/cli/cli/v2/pkg/cmdutil" @@ -233,7 +234,7 @@ func (s *StatusGetter) CurrentUsername() (string, error) { return currentUsername, nil } -func (s *StatusGetter) ActualMention(commentURL string) (string, error) { +func (s *StatusGetter) ActualMention(commentURL safeurl.SafeURL) (string, error) { currentUsername, err := s.CurrentUsername() if err != nil { return "", err @@ -246,7 +247,7 @@ func (s *StatusGetter) ActualMention(commentURL string) (string, error) { resp := struct { Body string }{} - if err := c.REST(s.hostname(), "GET", commentURL, nil, &resp); err != nil { + if err := c.REST(s.hostname(), "GET", commentURL.String(), nil, &resp); err != nil { return "", err } @@ -264,10 +265,6 @@ func (s *StatusGetter) ActualMention(commentURL string) (string, error) { func (s *StatusGetter) LoadNotifications() error { perPage := 100 c := api.NewClientFromHTTP(s.Client) - query := url.Values{} - query.Add("per_page", fmt.Sprintf("%d", perPage)) - query.Add("participating", "true") - query.Add("all", "true") fetchWorkers := 10 ctx, abortFetching := context.WithCancel(context.Background()) @@ -286,7 +283,7 @@ func (s *StatusGetter) LoadNotifications() error { if !ok { return nil } - actual, err := s.ActualMention(n.Subject.LatestCommentURL) + actual, err := s.ActualMention(safeurl.NewImmutableSafeURL(n.Subject.LatestCommentURL)) if err != nil { var httpErr api.HTTPError @@ -336,10 +333,17 @@ func (s *StatusGetter) LoadNotifications() error { // do that. I'd switch to the GraphQL version, but to my knowledge that does // not work with PATs right now. nIndex := 0 - p := fmt.Sprintf("notifications?%s", query.Encode()) + u, err := safeurl.JoinPath("notifications") + if err != nil { + return err + } + u.SetQuery("per_page", strconv.Itoa(perPage)) + u.SetQuery("participating", "true") + u.SetQuery("all", "true") + var p safeurl.SafeURL = u for pages := 0; pages < 3; pages++ { var resp []Notification - next, err := c.RESTWithNext(s.hostname(), "GET", p, nil, &resp) + next, err := c.RESTWithNext(s.hostname(), "GET", p.String(), nil, &resp) if err != nil { var httpErr api.HTTPError if !errors.As(err, &httpErr) || httpErr.StatusCode != 404 { @@ -365,11 +369,11 @@ func (s *StatusGetter) LoadNotifications() error { if next == "" || len(resp) < perPage { break } - p = next + p = safeurl.NewImmutableSafeURL(next) } close(toFetch) - err := wg.Wait() + err = wg.Wait() close(fetched) <-doneCh sort.Slice(s.Mentions, func(i, j int) bool { @@ -530,8 +534,6 @@ func (s *StatusGetter) LoadSearchResults() error { func (s *StatusGetter) LoadEvents() error { perPage := 100 c := api.NewClientFromHTTP(s.Client) - query := url.Values{} - query.Add("per_page", fmt.Sprintf("%d", perPage)) currentUsername, err := s.CurrentUsername() if err != nil { @@ -541,9 +543,14 @@ func (s *StatusGetter) LoadEvents() error { var events []Event var resp []Event pages := 0 - p := fmt.Sprintf("users/%s/received_events?%s", currentUsername, query.Encode()) + u, err := safeurl.JoinPath("users", currentUsername, "received_events") + if err != nil { + return err + } + u.SetQuery("per_page", strconv.Itoa(perPage)) + var p safeurl.SafeURL = u for pages < 2 { - next, err := c.RESTWithNext(s.hostname(), "GET", p, nil, &resp) + next, err := c.RESTWithNext(s.hostname(), "GET", p.String(), nil, &resp) if err != nil { var httpErr api.HTTPError if !errors.As(err, &httpErr) || httpErr.StatusCode != 404 { @@ -556,7 +563,7 @@ func (s *StatusGetter) LoadEvents() error { } pages++ - p = next + p = safeurl.NewImmutableSafeURL(next) } s.RepoActivity = []StatusItem{} diff --git a/pkg/cmd/variable/delete/delete.go b/pkg/cmd/variable/delete/delete.go index d5132016751..d8c12d28ddf 100644 --- a/pkg/cmd/variable/delete/delete.go +++ b/pkg/cmd/variable/delete/delete.go @@ -8,6 +8,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/variable/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -96,21 +97,24 @@ func removeRun(opts *DeleteOptions) error { return err } - var path string + var path *safeurl.MutableSafeURL var host string switch variableEntity { case shared.Organization: - path = fmt.Sprintf("orgs/%s/actions/variables/%s", orgName, opts.VariableName) + path, err = safeurl.JoinPath("orgs", orgName, "actions", "variables", opts.VariableName) host, _ = cfg.Authentication().DefaultHost() case shared.Environment: - path = fmt.Sprintf("repos/%s/environments/%s/variables/%s", ghrepo.FullName(baseRepo), envName, opts.VariableName) + path, err = safeurl.JoinPath("repos", baseRepo.RepoOwner(), baseRepo.RepoName(), "environments", envName, "variables", opts.VariableName) host = baseRepo.RepoHost() case shared.Repository: - path = fmt.Sprintf("repos/%s/actions/variables/%s", ghrepo.FullName(baseRepo), opts.VariableName) + path, err = safeurl.JoinPath("repos", baseRepo.RepoOwner(), baseRepo.RepoName(), "actions", "variables", opts.VariableName) host = baseRepo.RepoHost() } + if err != nil { + return err + } - err = client.REST(host, "DELETE", path, nil, nil) + err = client.REST(host, "DELETE", path.String(), nil, nil) if err != nil { return fmt.Errorf("failed to delete variable %s: %w", opts.VariableName, err) } diff --git a/pkg/cmd/variable/get/get.go b/pkg/cmd/variable/get/get.go index e4def5a03b2..6247715e2f9 100644 --- a/pkg/cmd/variable/get/get.go +++ b/pkg/cmd/variable/get/get.go @@ -9,6 +9,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/variable/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -97,22 +98,25 @@ func getRun(opts *GetOptions) error { return err } - var path string + var path *safeurl.MutableSafeURL var host string switch variableEntity { case shared.Organization: - path = fmt.Sprintf("orgs/%s/actions/variables/%s", orgName, opts.VariableName) + path, err = safeurl.JoinPath("orgs", orgName, "actions", "variables", opts.VariableName) host, _ = cfg.Authentication().DefaultHost() case shared.Environment: - path = fmt.Sprintf("repos/%s/environments/%s/variables/%s", ghrepo.FullName(baseRepo), envName, opts.VariableName) + path, err = safeurl.JoinPath("repos", baseRepo.RepoOwner(), baseRepo.RepoName(), "environments", envName, "variables", opts.VariableName) host = baseRepo.RepoHost() case shared.Repository: - path = fmt.Sprintf("repos/%s/actions/variables/%s", ghrepo.FullName(baseRepo), opts.VariableName) + path, err = safeurl.JoinPath("repos", baseRepo.RepoOwner(), baseRepo.RepoName(), "actions", "variables", opts.VariableName) host = baseRepo.RepoHost() } + if err != nil { + return err + } var variable shared.Variable - if err = client.REST(host, "GET", path, nil, &variable); err != nil { + if err = client.REST(host, "GET", path.String(), nil, &variable); err != nil { var httpErr api.HTTPError if errors.As(err, &httpErr) && httpErr.StatusCode == http.StatusNotFound { return fmt.Errorf("variable %s was not found", opts.VariableName) @@ -122,8 +126,12 @@ func getRun(opts *GetOptions) error { } if opts.Exporter != nil { - if err := shared.PopulateSelectedRepositoryInformation(client, host, &variable); err != nil { - return err + if variable.SelectedReposURL != "" { + count, err := shared.SelectedRepositoryCount(client, host, safeurl.NewImmutableSafeURL(variable.SelectedReposURL)) + if err != nil { + return fmt.Errorf("failed determining selected repositories for %s: %w", variable.Name, err) + } + variable.NumSelectedRepos = count } return opts.Exporter.Write(opts.IO, &variable) } diff --git a/pkg/cmd/variable/list/list.go b/pkg/cmd/variable/list/list.go index 764c0af4d13..7624da22577 100644 --- a/pkg/cmd/variable/list/list.go +++ b/pkg/cmd/variable/list/list.go @@ -11,6 +11,7 @@ import ( "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/internal/tableprinter" "github.com/cli/cli/v2/pkg/cmd/variable/shared" "github.com/cli/cli/v2/pkg/cmdutil" @@ -193,42 +194,60 @@ func fmtVisibility(s shared.Variable) string { } func getRepoVariables(client *http.Client, repo ghrepo.Interface) ([]shared.Variable, error) { - return getVariables(client, repo.RepoHost(), fmt.Sprintf("repos/%s/actions/variables", ghrepo.FullName(repo))) + u, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "variables") + if err != nil { + return nil, err + } + return getVariables(client, repo.RepoHost(), u) } func getEnvVariables(client *http.Client, repo ghrepo.Interface, envName string) ([]shared.Variable, error) { - path := fmt.Sprintf("repos/%s/environments/%s/variables", ghrepo.FullName(repo), envName) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "environments", envName, "variables") + if err != nil { + return nil, err + } return getVariables(client, repo.RepoHost(), path) } func getOrgVariables(client *http.Client, host, orgName string, showSelectedRepoInfo bool) ([]shared.Variable, error) { - variables, err := getVariables(client, host, fmt.Sprintf("orgs/%s/actions/variables", orgName)) + u, err := safeurl.JoinPath("orgs", orgName, "actions", "variables") + if err != nil { + return nil, err + } + variables, err := getVariables(client, host, u) if err != nil { return nil, err } apiClient := api.NewClientFromHTTP(client) if showSelectedRepoInfo { - err = shared.PopulateMultipleSelectedRepositoryInformation(apiClient, host, variables) - if err != nil { - return nil, err + for i := range variables { + if variables[i].SelectedReposURL == "" { + continue + } + count, err := shared.SelectedRepositoryCount(apiClient, host, safeurl.NewImmutableSafeURL(variables[i].SelectedReposURL)) + if err != nil { + return nil, fmt.Errorf("failed determining selected repositories for %s: %w", variables[i].Name, err) + } + variables[i].NumSelectedRepos = count } } return variables, nil } -func getVariables(client *http.Client, host, path string) ([]shared.Variable, error) { +func getVariables(client *http.Client, host string, u *safeurl.MutableSafeURL) ([]shared.Variable, error) { var results []shared.Variable apiClient := api.NewClientFromHTTP(client) - path = fmt.Sprintf("%s?per_page=100", path) - for path != "" { + u.SetQuery("per_page", "100") + var pageURL safeurl.SafeURL = u + for pageURL.String() != "" { response := struct { Variables []shared.Variable }{} - var err error - path, err = apiClient.RESTWithNext(host, "GET", path, nil, &response) + next, err := apiClient.RESTWithNext(host, "GET", pageURL.String(), nil, &response) if err != nil { return nil, err } + pageURL = safeurl.NewImmutableSafeURL(next) results = append(results, response.Variables...) } return results, nil diff --git a/pkg/cmd/variable/list/list_test.go b/pkg/cmd/variable/list/list_test.go index 4933d3957e4..46c68b1f53e 100644 --- a/pkg/cmd/variable/list/list_test.go +++ b/pkg/cmd/variable/list/list_test.go @@ -12,6 +12,7 @@ import ( "github.com/cli/cli/v2/internal/config" "github.com/cli/cli/v2/internal/gh" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/variable/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/httpmock" @@ -436,7 +437,9 @@ func Test_getVariables_pagination(t *testing.T) { httpmock.StringResponse(`{"variables":[{},{}]}`), ) client := &http.Client{Transport: reg} - variables, err := getVariables(client, "github.com", "path/to") + u, err := safeurl.JoinPath("path", "to") + require.NoError(t, err) + variables, err := getVariables(client, "github.com", u) assert.NoError(t, err) assert.Equal(t, 4, len(variables)) } diff --git a/pkg/cmd/variable/set/http.go b/pkg/cmd/variable/set/http.go index e3acb5e0a7f..2a1f581c676 100644 --- a/pkg/cmd/variable/set/http.go +++ b/pkg/cmd/variable/set/http.go @@ -5,9 +5,11 @@ import ( "encoding/json" "errors" "fmt" + "strconv" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/variable/shared" ) @@ -82,13 +84,13 @@ func setVariable(client *api.Client, host string, opts setOptions) setResult { return result } -func postVariable(client *api.Client, host, path string, payload interface{}) error { +func postVariable(client *api.Client, host string, path safeurl.SafeURL, payload interface{}) error { payloadBytes, err := json.Marshal(payload) if err != nil { return fmt.Errorf("failed to serialize: %w", err) } requestBody := bytes.NewReader(payloadBytes) - return client.REST(host, "POST", path, requestBody, nil) + return client.REST(host, "POST", path.String(), requestBody, nil) } func postOrgVariable(client *api.Client, host, orgName, visibility, variableName, value string, repositoryIDs []int64) error { @@ -98,7 +100,10 @@ func postOrgVariable(client *api.Client, host, orgName, visibility, variableName Visibility: visibility, Repositories: repositoryIDs, } - path := fmt.Sprintf(`orgs/%s/actions/variables`, orgName) + path, err := safeurl.JoinPath("orgs", orgName, "actions", "variables") + if err != nil { + return err + } return postVariable(client, host, path, payload) } @@ -107,7 +112,10 @@ func postEnvVariable(client *api.Client, host string, repoID int64, envName, var Name: variableName, Value: value, } - path := fmt.Sprintf(`repositories/%d/environments/%s/variables`, repoID, envName) + path, err := safeurl.JoinPath("repositories", strconv.FormatInt(repoID, 10), "environments", envName, "variables") + if err != nil { + return err + } return postVariable(client, host, path, payload) } @@ -116,17 +124,20 @@ func postRepoVariable(client *api.Client, repo ghrepo.Interface, variableName, v Name: variableName, Value: value, } - path := fmt.Sprintf(`repos/%s/actions/variables`, ghrepo.FullName(repo)) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "variables") + if err != nil { + return err + } return postVariable(client, repo.RepoHost(), path, payload) } -func patchVariable(client *api.Client, host, path string, payload interface{}) error { +func patchVariable(client *api.Client, host string, path safeurl.SafeURL, payload interface{}) error { payloadBytes, err := json.Marshal(payload) if err != nil { return fmt.Errorf("failed to serialize: %w", err) } requestBody := bytes.NewReader(payloadBytes) - return client.REST(host, "PATCH", path, requestBody, nil) + return client.REST(host, "PATCH", path.String(), requestBody, nil) } func patchOrgVariable(client *api.Client, host, orgName, visibility, variableName, value string, repositoryIDs []int64) error { @@ -135,7 +146,10 @@ func patchOrgVariable(client *api.Client, host, orgName, visibility, variableNam Visibility: visibility, Repositories: repositoryIDs, } - path := fmt.Sprintf(`orgs/%s/actions/variables/%s`, orgName, variableName) + path, err := safeurl.JoinPath("orgs", orgName, "actions", "variables", variableName) + if err != nil { + return err + } return patchVariable(client, host, path, payload) } @@ -143,7 +157,10 @@ func patchEnvVariable(client *api.Client, host string, repoID int64, envName, va payload := setPayload{ Value: value, } - path := fmt.Sprintf(`repositories/%d/environments/%s/variables/%s`, repoID, envName, variableName) + path, err := safeurl.JoinPath("repositories", strconv.FormatInt(repoID, 10), "environments", envName, "variables", variableName) + if err != nil { + return err + } return patchVariable(client, host, path, payload) } @@ -151,6 +168,9 @@ func patchRepoVariable(client *api.Client, repo ghrepo.Interface, variableName, payload := setPayload{ Value: value, } - path := fmt.Sprintf(`repos/%s/actions/variables/%s`, ghrepo.FullName(repo), variableName) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "variables", variableName) + if err != nil { + return err + } return patchVariable(client, repo.RepoHost(), path, payload) } diff --git a/pkg/cmd/variable/shared/shared.go b/pkg/cmd/variable/shared/shared.go index c681242bc3f..de449c9864f 100644 --- a/pkg/cmd/variable/shared/shared.go +++ b/pkg/cmd/variable/shared/shared.go @@ -2,10 +2,10 @@ package shared import ( "errors" - "fmt" "time" "github.com/cli/cli/v2/api" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmdutil" ) @@ -66,27 +66,14 @@ func GetVariableEntity(orgName, envName string) (VariableEntity, error) { return Repository, nil } -func PopulateMultipleSelectedRepositoryInformation(apiClient *api.Client, host string, variables []Variable) error { - for i, variable := range variables { - if err := PopulateSelectedRepositoryInformation(apiClient, host, &variable); err != nil { - return err - } - variables[i] = variable - } - return nil -} - -func PopulateSelectedRepositoryInformation(apiClient *api.Client, host string, variable *Variable) error { - if variable.SelectedReposURL == "" { - return nil - } - +// SelectedRepositoryCount returns how many repositories the variable is visible to, fetched from the +// given entrusted URL. Callers own reading the URL off the variable and writing the result back. +func SelectedRepositoryCount(apiClient *api.Client, host string, selectedReposURL safeurl.SafeURL) (int, error) { response := struct { TotalCount int `json:"total_count"` }{} - if err := apiClient.REST(host, "GET", variable.SelectedReposURL, nil, &response); err != nil { - return fmt.Errorf("failed determining selected repositories for %s: %w", variable.Name, err) + if err := apiClient.REST(host, "GET", selectedReposURL.String(), nil, &response); err != nil { + return 0, err } - variable.NumSelectedRepos = response.TotalCount - return nil + return response.TotalCount, nil } diff --git a/pkg/cmd/workflow/disable/disable.go b/pkg/cmd/workflow/disable/disable.go index 8b2fb62d307..53882a17e17 100644 --- a/pkg/cmd/workflow/disable/disable.go +++ b/pkg/cmd/workflow/disable/disable.go @@ -4,9 +4,11 @@ import ( "errors" "fmt" "net/http" + "strconv" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/workflow/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -84,8 +86,11 @@ func runDisable(opts *DisableOptions) error { return err } - path := fmt.Sprintf("repos/%s/actions/workflows/%d/disable", ghrepo.FullName(repo), workflow.ID) - err = client.REST(repo.RepoHost(), "PUT", path, nil, nil) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "workflows", strconv.FormatInt(workflow.ID, 10), "disable") + if err != nil { + return err + } + err = client.REST(repo.RepoHost(), "PUT", path.String(), nil, nil) if err != nil { return fmt.Errorf("failed to disable workflow: %w", err) } diff --git a/pkg/cmd/workflow/enable/enable.go b/pkg/cmd/workflow/enable/enable.go index 93e8ac00719..1fc6eb755df 100644 --- a/pkg/cmd/workflow/enable/enable.go +++ b/pkg/cmd/workflow/enable/enable.go @@ -4,9 +4,11 @@ import ( "errors" "fmt" "net/http" + "strconv" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/workflow/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -84,8 +86,11 @@ func runEnable(opts *EnableOptions) error { return err } - path := fmt.Sprintf("repos/%s/actions/workflows/%d/enable", ghrepo.FullName(repo), workflow.ID) - err = client.REST(repo.RepoHost(), "PUT", path, nil, nil) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "workflows", strconv.FormatInt(workflow.ID, 10), "enable") + if err != nil { + return err + } + err = client.REST(repo.RepoHost(), "PUT", path.String(), nil, nil) if err != nil { return fmt.Errorf("failed to enable workflow: %w", err) } diff --git a/pkg/cmd/workflow/run/run.go b/pkg/cmd/workflow/run/run.go index 9042b9249db..350c59bcd2c 100644 --- a/pkg/cmd/workflow/run/run.go +++ b/pkg/cmd/workflow/run/run.go @@ -7,9 +7,9 @@ import ( "fmt" "io" "net/http" - "net/url" "reflect" "sort" + "strconv" "strings" "time" @@ -17,6 +17,7 @@ import ( "github.com/cli/cli/v2/api" fd "github.com/cli/cli/v2/internal/featuredetection" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmd/workflow/shared" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" @@ -319,7 +320,10 @@ func runRun(opts *RunOptions) error { return err } - path := fmt.Sprintf("repos/%s/%s/actions/workflows/%d/dispatches", url.PathEscape(repo.RepoOwner()), url.PathEscape(repo.RepoName()), workflow.ID) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "workflows", strconv.FormatInt(workflow.ID, 10), "dispatches") + if err != nil { + return err + } requestBody := map[string]interface{}{ "ref": ref, @@ -358,7 +362,7 @@ func runRun(opts *RunOptions) error { // // As a related note, the new REST API version (which will come with breaking // changes) will probably default to return 200 + run details. - err = client.REST(repo.RepoHost(), "POST", path, body, &response) + err = client.REST(repo.RepoHost(), "POST", path.String(), body, &response) if err != nil { return fmt.Errorf("could not create workflow dispatch event: %w", err) } diff --git a/pkg/cmd/workflow/run/run_test.go b/pkg/cmd/workflow/run/run_test.go index a4e44e5dabd..44a3b27853a 100644 --- a/pkg/cmd/workflow/run/run_test.go +++ b/pkg/cmd/workflow/run/run_test.go @@ -764,7 +764,7 @@ jobs: }, })) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/contents/.github/workflows/minimal.yml"), + httpmock.REST("GET", "repos/OWNER/REPO/contents/.github%2Fworkflows%2Fminimal.yml"), httpmock.JSONResponse(struct{ Content string }{ Content: encodedNoInputsYAMLContent, })) @@ -808,7 +808,7 @@ jobs: }, })) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/contents/.github/workflows/minimal.yml"), + httpmock.REST("GET", "repos/OWNER/REPO/contents/.github%2Fworkflows%2Fminimal.yml"), httpmock.JSONResponse(struct{ Content string }{ Content: encodedNoInputsYAMLContent, })) @@ -861,7 +861,7 @@ jobs: }, })) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/contents/.github/workflows/workflow.yml"), + httpmock.REST("GET", "repos/OWNER/REPO/contents/.github%2Fworkflows%2Fworkflow.yml"), httpmock.JSONResponse(struct{ Content string }{ Content: encodedYAMLContent, })) @@ -914,7 +914,7 @@ jobs: }, })) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/contents/.github/workflows/workflow.yml"), + httpmock.REST("GET", "repos/OWNER/REPO/contents/.github%2Fworkflows%2Fworkflow.yml"), httpmock.JSONResponse(struct{ Content string }{ Content: encodedYAMLContent, })) @@ -976,7 +976,7 @@ jobs: }, })) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/contents/.github/workflows/workflow.yml"), + httpmock.REST("GET", "repos/OWNER/REPO/contents/.github%2Fworkflows%2Fworkflow.yml"), httpmock.JSONResponse(struct{ Content string }{ Content: encodedYAMLContentChoiceIp, })) @@ -1030,7 +1030,7 @@ jobs: }, })) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/contents/.github/workflows/workflow.yml"), + httpmock.REST("GET", "repos/OWNER/REPO/contents/.github%2Fworkflows%2Fworkflow.yml"), httpmock.JSONResponse(struct{ Content string }{ Content: encodedYAMLContentChoiceIp, })) @@ -1091,7 +1091,7 @@ jobs: }, })) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/contents/.github/workflows/workflow.yml"), + httpmock.REST("GET", "repos/OWNER/REPO/contents/.github%2Fworkflows%2Fworkflow.yml"), httpmock.JSONResponse(struct{ Content string }{ Content: encodedYAMLContentMissingChoiceIp, })) diff --git a/pkg/cmd/workflow/shared/shared.go b/pkg/cmd/workflow/shared/shared.go index 04b5fa199aa..2cb6b91ff94 100644 --- a/pkg/cmd/workflow/shared/shared.go +++ b/pkg/cmd/workflow/shared/shared.go @@ -6,13 +6,13 @@ import ( "errors" "fmt" "io" - "net/url" "path" "strconv" "strings" "github.com/cli/cli/v2/api" "github.com/cli/cli/v2/internal/ghrepo" + "github.com/cli/cli/v2/internal/safeurl" "github.com/cli/cli/v2/pkg/cmdutil" "github.com/cli/cli/v2/pkg/iostreams" "github.com/cli/go-gh/v2/pkg/asciisanitizer" @@ -69,9 +69,14 @@ func GetWorkflows(client *api.Client, repo ghrepo.Interface, limit int) ([]Workf } var result WorkflowsPayload - path := fmt.Sprintf("repos/%s/actions/workflows?per_page=%d&page=%d", ghrepo.FullName(repo), perPage, page) + u, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "workflows") + if err != nil { + return nil, err + } + u.SetQuery("per_page", strconv.Itoa(perPage)) + u.SetQuery("page", strconv.Itoa(page)) - err := client.REST(repo.RepoHost(), "GET", path, nil, &result) + err = client.REST(repo.RepoHost(), "GET", u.String(), nil, &result) if err != nil { return nil, err } @@ -159,8 +164,11 @@ func isWorkflowFile(f string) bool { func getWorkflowByID(client *api.Client, repo ghrepo.Interface, ID string) (*Workflow, error) { var workflow Workflow - path := fmt.Sprintf("repos/%s/actions/workflows/%s", ghrepo.FullName(repo), url.PathEscape(ID)) - if err := client.REST(repo.RepoHost(), "GET", path, nil, &workflow); err != nil { + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "actions", "workflows", ID) + if err != nil { + return nil, err + } + if err := client.REST(repo.RepoHost(), "GET", path.String(), nil, &workflow); err != nil { return nil, err } @@ -233,10 +241,12 @@ func ResolveWorkflow(p iprompter, io *iostreams.IOStreams, client *api.Client, r } func GetWorkflowContent(client *api.Client, repo ghrepo.Interface, workflow Workflow, ref string) ([]byte, error) { - path := fmt.Sprintf("repos/%s/contents/%s", ghrepo.FullName(repo), workflow.Path) + path, err := safeurl.JoinPath("repos", repo.RepoOwner(), repo.RepoName(), "contents", workflow.Path) + if err != nil { + return nil, err + } if ref != "" { - q := fmt.Sprintf("?ref=%s", url.QueryEscape(ref)) - path = path + q + path.SetQuery("ref", ref) } type Result struct { @@ -244,7 +254,7 @@ func GetWorkflowContent(client *api.Client, repo ghrepo.Interface, workflow Work } var result Result - err := client.REST(repo.RepoHost(), "GET", path, nil, &result) + err = client.REST(repo.RepoHost(), "GET", path.String(), nil, &result) if err != nil { return nil, err } diff --git a/pkg/cmd/workflow/view/view_test.go b/pkg/cmd/workflow/view/view_test.go index e5df52478be..db36797666b 100644 --- a/pkg/cmd/workflow/view/view_test.go +++ b/pkg/cmd/workflow/view/view_test.go @@ -289,7 +289,7 @@ func TestViewRun(t *testing.T) { httpmock.JSONResponse(aWorkflow), ) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/contents/.github/workflows/flow.yml"), + httpmock.REST("GET", "repos/OWNER/REPO/contents/.github%2Fworkflows%2Fflow.yml"), httpmock.StringResponse(aWorkflowContent), ) }, @@ -308,7 +308,7 @@ func TestViewRun(t *testing.T) { httpmock.JSONResponse(aWorkflow), ) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/contents/.github/workflows/flow.yml"), + httpmock.REST("GET", "repos/OWNER/REPO/contents/.github%2Fworkflows%2Fflow.yml"), httpmock.StringResponse(aWorkflowContent), ) }, @@ -327,7 +327,7 @@ func TestViewRun(t *testing.T) { httpmock.JSONResponse(aWorkflow), ) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/contents/.github/workflows/flow.yml"), + httpmock.REST("GET", "repos/OWNER/REPO/contents/.github%2Fworkflows%2Fflow.yml"), httpmock.StatusStringResponse(404, "not Found"), ) }, @@ -348,7 +348,7 @@ func TestViewRun(t *testing.T) { httpmock.JSONResponse(aWorkflow), ) reg.Register( - httpmock.REST("GET", "repos/OWNER/REPO/contents/.github/workflows/flow.yml"), + httpmock.REST("GET", "repos/OWNER/REPO/contents/.github%2Fworkflows%2Fflow.yml"), httpmock.StringResponse(aWorkflowContent), ) }, diff --git a/pkg/search/searcher.go b/pkg/search/searcher.go index 5b05e1619e5..dd8dd590cac 100644 --- a/pkg/search/searcher.go +++ b/pkg/search/searcher.go @@ -12,6 +12,7 @@ import ( fd "github.com/cli/cli/v2/internal/featuredetection" "github.com/cli/cli/v2/internal/ghinstance" + "github.com/cli/cli/v2/internal/safeurl" ) const ( @@ -197,10 +198,12 @@ func (s searcher) Issues(query Query) (IssuesResult, error) { // // For more information, see https://docs.github.com/en/rest/search/search?apiVersion=2022-11-28. func (s searcher) search(query Query, result interface{}) (string, error) { - path := fmt.Sprintf("%ssearch/%s", ghinstance.RESTPrefix(s.host), query.Kind) - qs := url.Values{} - qs.Set("page", strconv.Itoa(query.Page)) - qs.Set("per_page", strconv.Itoa(query.Limit)) + u, err := safeurl.JoinPathWithHostPrefix(ghinstance.RESTPrefix(s.host), "search", string(query.Kind)) + if err != nil { + return "", err + } + u.SetQuery("page", strconv.Itoa(query.Page)) + u.SetQuery("per_page", strconv.Itoa(query.Limit)) if query.Kind == KindIssues { // TODO advancedIssueSearchCleanup @@ -213,28 +216,27 @@ func (s searcher) search(query Query, result interface{}) (string, error) { } if !features.AdvancedIssueSearchAPI { - qs.Set("q", query.StandardSearchString()) + u.SetQuery("q", query.StandardSearchString()) } else { - qs.Set("q", query.AdvancedIssueSearchString()) + u.SetQuery("q", query.AdvancedIssueSearchString()) // TODO advancedIssueSearchCleanup if features.AdvancedIssueSearchAPIOptIn { // Advanced syntax should be explicitly enabled - qs.Set("advanced_search", "true") + u.SetQuery("advanced_search", "true") } } } else { - qs.Set("q", query.StandardSearchString()) + u.SetQuery("q", query.StandardSearchString()) } if query.Order != "" { - qs.Set(orderKey, query.Order) + u.SetQuery(orderKey, query.Order) } if query.Sort != "" { - qs.Set(sortKey, query.Sort) + u.SetQuery(sortKey, query.Sort) } - url := fmt.Sprintf("%s?%s", path, qs.Encode()) - req, err := http.NewRequest("GET", url, nil) + req, err := http.NewRequest("GET", u.String(), nil) if err != nil { return "", err } From 3f6a16a9f8c7fe9676aa8d8f47b399310dd231c3 Mon Sep 17 00:00:00 2001 From: "Babak K. Shandiz" Date: Fri, 31 Jul 2026 02:37:45 +0100 Subject: [PATCH 3/4] Merge commit from fork Signed-off-by: Babak K. Shandiz --- pkg/cmd/auth/status/status.go | 13 ++++-- pkg/cmd/auth/status/status_test.go | 73 +++++++++++++++++++++++++++++- 2 files changed, 81 insertions(+), 5 deletions(-) diff --git a/pkg/cmd/auth/status/status.go b/pkg/cmd/auth/status/status.go index 658a8d8bc79..f1c597bbac9 100644 --- a/pkg/cmd/auth/status/status.go +++ b/pkg/cmd/auth/status/status.go @@ -329,10 +329,17 @@ func statusRun(opts *StatusOptions) error { return finalErr } +// knownTokenPrefixes contains GitHub's token format prefixes. +// See [GitHub token formats]. +// +// [GitHub token formats]: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/about-authentication-to-github#githubs-token-formats +var knownTokenPrefixes = []string{"github_pat_", "ghp_", "gho_", "ghu_", "ghs_", "ghr_"} + func maskToken(token string) string { - if idx := strings.LastIndexByte(token, '_'); idx > -1 { - prefix := token[0 : idx+1] - return prefix + strings.Repeat("*", len(token)-len(prefix)) + for _, prefix := range knownTokenPrefixes { + if strings.HasPrefix(token, prefix) { + return prefix + strings.Repeat("*", len(token)-len(prefix)) + } } return strings.Repeat("*", len(token)) } diff --git a/pkg/cmd/auth/status/status_test.go b/pkg/cmd/auth/status/status_test.go index cb2abb90ecf..87ee5f5e5a3 100644 --- a/pkg/cmd/auth/status/status_test.go +++ b/pkg/cmd/auth/status/status_test.go @@ -312,7 +312,7 @@ func Test_statusRun(t *testing.T) { name: "PAT V2 token", opts: StatusOptions{}, cfgStubs: func(t *testing.T, c gh.Config) { - login(t, c, "github.com", "monalisa", "github_pat_abc123", "https") + login(t, c, "github.com", "monalisa", "github_pat_abc_123456", "https") }, httpStubs: func(reg *httpmock.Registry) { // mocks for HeaderHasMinimumScopes api requests to github.com @@ -325,7 +325,7 @@ func Test_statusRun(t *testing.T) { ✓ Logged in to github.com account monalisa (GH_CONFIG_DIR/hosts.yml) - Active account: true - Git operations protocol: https - - Token: github_pat_****** + - Token: github_pat_********** `), }, { @@ -782,3 +782,72 @@ func replaceAll(s string, old string, new string) string { replaced = strings.ReplaceAll(replaced, old, new) return replaced } + +func TestMaskToken(t *testing.T) { + tests := []struct { + name string + token string + want string + }{ + { + name: "empty token", + token: "", + want: "", + }, + { + name: "classic personal access token", + token: "ghp_abc123", + want: "ghp_******", + }, + { + name: "oauth token", + token: "gho_abc123", + want: "gho_******", + }, + { + name: "user-to-server token", + token: "ghu_abc123", + want: "ghu_******", + }, + { + name: "server-to-server token", + token: "ghs_abc123", + want: "ghs_******", + }, + { + name: "refresh token", + token: "ghr_abc123", + want: "ghr_******", + }, + { + name: "fine-grained personal access token with internal underscore", + token: "github_pat_abc_123456", + want: "github_pat_**********", + }, + { + name: "token with multiple internal underscores masks everything after prefix", + token: "ghs_aaa_bbb_ccc", + want: "ghs_***********", + }, + { + name: "unknown prefix is fully masked", + token: "unknown_abc123", + want: "**************", + }, + { + name: "token without underscore is fully masked", + token: "abc123", + want: "******", + }, + { + name: "token equal to known prefix has nothing to mask", + token: "gho_", + want: "gho_", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + assert.Equal(t, tt.want, maskToken(tt.token)) + }) + } +} From 55dbb4dc6b7edb10b48e3d7fc5bccd32318d1b55 Mon Sep 17 00:00:00 2001 From: Austin Beattie Date: Thu, 30 Jul 2026 18:38:16 -0700 Subject: [PATCH 4/4] Merge commit from fork * Escape regex metacharacters in attestation SAN matching Apply regexp.QuoteMeta to user-supplied values interpolated into regex patterns in expandToGitHubURLRegex and validateSignerWorkflow. Without escaping, dots in org/repo names act as regex wildcards, allowing attestation spoofing via lookalike repositories. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address review: QuoteMeta entire static string at once Per review feedback, wrap the full formatted string in QuoteMeta rather than individual variables, to better indicate the whole piece is escaped and avoid confusion around trailing/leading chars. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- pkg/cmd/attestation/verify/policy.go | 6 ++-- pkg/cmd/attestation/verify/policy_test.go | 35 +++++++++++++++++------ 2 files changed, 29 insertions(+), 12 deletions(-) diff --git a/pkg/cmd/attestation/verify/policy.go b/pkg/cmd/attestation/verify/policy.go index 1d1595eca70..9f15653686e 100644 --- a/pkg/cmd/attestation/verify/policy.go +++ b/pkg/cmd/attestation/verify/policy.go @@ -24,7 +24,7 @@ func expandToGitHubURL(tenant, ownerOrRepo string) string { func expandToGitHubURLRegex(tenant, ownerOrRepo string) string { url := expandToGitHubURL(tenant, ownerOrRepo) - return fmt.Sprintf("(?i)^%s/", url) + return fmt.Sprintf("(?i)^%s", regexp.QuoteMeta(url+"/")) } func newEnforcementCriteria(opts *Options) (verification.EnforcementCriteria, error) { @@ -155,7 +155,7 @@ func validateSignerWorkflow(hostname, signerWorkflow string) (string, error) { } if match { - return fmt.Sprintf("^https://%s", signerWorkflow), nil + return "^" + regexp.QuoteMeta(fmt.Sprintf("https://%s", signerWorkflow)), nil } // if the provided workflow did not match the expect format @@ -164,5 +164,5 @@ func validateSignerWorkflow(hostname, signerWorkflow string) (string, error) { return "", errors.New("unknown signer workflow host") } - return fmt.Sprintf("^https://%s/%s", hostname, signerWorkflow), nil + return "^" + regexp.QuoteMeta(fmt.Sprintf("https://%s/%s", hostname, signerWorkflow)), nil } diff --git a/pkg/cmd/attestation/verify/policy_test.go b/pkg/cmd/attestation/verify/policy_test.go index ff10cad11d7..ae6e022f05f 100644 --- a/pkg/cmd/attestation/verify/policy_test.go +++ b/pkg/cmd/attestation/verify/policy_test.go @@ -1,6 +1,7 @@ package verify import ( + "regexp" "testing" "github.com/cli/cli/v2/pkg/cmd/attestation/verification" @@ -39,7 +40,7 @@ func TestNewEnforcementCriteria(t *testing.T) { c, err := newEnforcementCriteria(opts) require.NoError(t, err) - require.Equal(t, "(?i)^https://github.com/foo/bar/", c.SANRegex) + require.Equal(t, `(?i)^https://github\.com/foo/bar/`, c.SANRegex) require.Zero(t, c.SAN) }) @@ -55,7 +56,7 @@ func TestNewEnforcementCriteria(t *testing.T) { c, err := newEnforcementCriteria(opts) require.NoError(t, err) - require.Equal(t, "(?i)^https://baz.ghe.com/foo/bar/", c.SANRegex) + require.Equal(t, `(?i)^https://baz\.ghe\.com/foo/bar/`, c.SANRegex) require.Zero(t, c.SAN) }) @@ -70,7 +71,7 @@ func TestNewEnforcementCriteria(t *testing.T) { c, err := newEnforcementCriteria(opts) require.NoError(t, err) - require.Equal(t, "^https://github.com/foo/bar/.github/workflows/attest.yml", c.SANRegex) + require.Equal(t, `^https://github\.com/foo/bar/\.github/workflows/attest\.yml`, c.SANRegex) require.Zero(t, c.SAN) }) @@ -83,7 +84,7 @@ func TestNewEnforcementCriteria(t *testing.T) { c, err := newEnforcementCriteria(opts) require.NoError(t, err) - require.Equal(t, "(?i)^https://github.com/foo/bar/", c.SANRegex) + require.Equal(t, `(?i)^https://github\.com/foo/bar/`, c.SANRegex) }) t.Run("sets SANRegex using opts.Owner", func(t *testing.T) { @@ -94,7 +95,23 @@ func TestNewEnforcementCriteria(t *testing.T) { c, err := newEnforcementCriteria(opts) require.NoError(t, err) - require.Equal(t, "(?i)^https://github.com/foo/", c.SANRegex) + require.Equal(t, `(?i)^https://github\.com/foo/`, c.SANRegex) + }) + + t.Run("SANRegex escapes regex metacharacters in repo names", func(t *testing.T) { + opts := &Options{ + ArtifactPath: artifactPath, + SignerRepo: "my.org/my.repo", + } + + c, err := newEnforcementCriteria(opts) + require.NoError(t, err) + require.Equal(t, `(?i)^https://github\.com/my\.org/my\.repo/`, c.SANRegex) + + // Verify the generated regex does NOT match a lookalike repo + re := regexp.MustCompile(c.SANRegex) + require.True(t, re.MatchString("https://github.com/my.org/my.repo/.github/workflows/build.yml")) + require.False(t, re.MatchString("https://github.com/myXorg/myXrepo/.github/workflows/build.yml")) }) t.Run("sets Extensions.RunnerEnvironment to GitHubRunner value if opts.DenySelfHostedRunner is true", func(t *testing.T) { @@ -280,25 +297,25 @@ func TestValidateSignerWorkflow(t *testing.T) { { name: "workflow with default host", providedSignerWorkflow: "github/artifact-attestations-workflows/.github/workflows/attest.yml", - expectedWorkflowRegex: "^https://github.com/github/artifact-attestations-workflows/.github/workflows/attest.yml", + expectedWorkflowRegex: `^https://github\.com/github/artifact-attestations-workflows/\.github/workflows/attest\.yml`, host: "github.com", }, { name: "workflow with workflow URL included", providedSignerWorkflow: "github.com/github/artifact-attestations-workflows/.github/workflows/attest.yml", - expectedWorkflowRegex: "^https://github.com/github/artifact-attestations-workflows/.github/workflows/attest.yml", + expectedWorkflowRegex: `^https://github\.com/github/artifact-attestations-workflows/\.github/workflows/attest\.yml`, host: "github.com", }, { name: "workflow with GH_HOST set", providedSignerWorkflow: "github/artifact-attestations-workflows/.github/workflows/attest.yml", - expectedWorkflowRegex: "^https://myhost.github.com/github/artifact-attestations-workflows/.github/workflows/attest.yml", + expectedWorkflowRegex: `^https://myhost\.github\.com/github/artifact-attestations-workflows/\.github/workflows/attest\.yml`, host: "myhost.github.com", }, { name: "workflow with authenticated host", providedSignerWorkflow: "github/artifact-attestations-workflows/.github/workflows/attest.yml", - expectedWorkflowRegex: "^https://authedhost.github.com/github/artifact-attestations-workflows/.github/workflows/attest.yml", + expectedWorkflowRegex: `^https://authedhost\.github\.com/github/artifact-attestations-workflows/\.github/workflows/attest\.yml`, host: "authedhost.github.com", }, }