Skip to content

Replace google_api_storage with direct GCS JSON API calls - #5053

Merged
midigofrank merged 3 commits into
release-2.18.0from
fix-broken-project-files-gcs
Aug 6, 2026
Merged

Replace google_api_storage with direct GCS JSON API calls#5053
midigofrank merged 3 commits into
release-2.18.0from
fix-broken-project-files-gcs

Conversation

@midigofrank

@midigofrank midigofrank commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Description

Uploads to Google Cloud Storage have been broken since Tesla 1.18.3 landed in #4913. Tesla now requires multipart part names to be binaries, and google_gax labels its metadata part with an atom, so storage_objects_insert_simple raises before a request is ever sent:

** (FunctionClauseError) no function clause matching in
   Tesla.Multipart.assert_quoted_string_safe!/2
  (google_gax 0.4.1) lib/google_api/gax/connection.ex:183: build_body/3
  (google_api_storage 0.46.1) .../objects.ex:831: storage_objects_insert_simple/7
  (lightning) lib/lightning/workorders/export_worker.ex:114: store_project_file/2

Versions offer no way out: google_gax's last release was 2021-12-28, googleapis/elixir-google-api is archived, and pinning Tesla back to 1.18.2 reintroduces the five CVEs 1.18.3 closed. So lib/lightning/storage/gcs.ex now calls the GCS JSON API directly and the dependency is gone. get_url/1 already bypassed the library, so this only meant reimplementing two requests.

  • store/2 POSTs with uploadType=media, avoiding multipart entirely — the object name is the only metadata we set and it travels in the query string.
  • delete/1 becomes a plain DELETE, with the object path escaped as a single segment (/%2F), and now applies prefix_storage_path/1 — see below.
  • Both check the response status explicitly. Tesla returns {:ok, env} for a 403 or 500, whereas google_gax folded non-2xx into {:error, env} and callers rely on that. Without the check, ExportWorker.store_project_file/2 would record a rejected upload as stored.
  • bucket!/0 reads Lightning.Config.storage(:bucket) rather than the application env, making Lightning.Config the single source of truth for storage config.
  • New Lightning.Storage.GCS.TokenSource stands in for Goth, which only starts when STORAGE_BACKEND=gcs and so never runs under test. It's what makes the module testable — it had no tests before.
  • google_api_storage dropped, taking google_gax and poison with it.

delete/1 ignoring STORAGE_PATH

STORAGE_PATH defaults to ".", which Path.safe_relative/1 normalises away, which is why this stayed hidden. On a deployment with a real prefix, every delete asked for a key that was never written and got a 404 — and Projects.remove_expired_files_for/1 treats a 404 as "already gone" and drops the row anyway:

if match?({:ok, _res}, result) or match?({:error, %{status: 404}}, result) do
  Repo.delete(project_file)
end

Closes #5049

Validation steps

  1. mix test test/lightning/storage/ test/lightning/storage_test.exs test/lightning/export_worker_test.exs
  2. With STORAGE_BACKEND=gcs in your .env, mix test test/lightning/export_worker_test.exs test/lightning/projects_test.exs should pass rather than reaching for a real bucket.
  3. Against a real bucket, with a non-trivial STORAGE_PATH: trigger a history export, confirm the object lands at the prefixed key, then confirm deletion removes it.

Additional notes for the reviewer

  1. Not yet verified end to end against a real bucket — step 3 above is still outstanding.
  2. The %Stream{} wrapper in stream_file/1 is load-bearing, not stylistic. Tesla.Adapter.Finch matches %Stream{} to send a chunked body; File.stream!/2 returns a %File.Stream{}, which doesn't match and fails.

AI Usage

Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):

  • I have used Claude Code
  • I have used another model
  • I have not used AI

You can read more details in our
Responsible AI Policy

Pre-submission checklist

  • I have performed an AI review of my code (we recommend using /review
    with Claude Code)
  • I have implemented and tested all related authorization policies.
    (e.g., :owner, :admin, :editor, :viewer) — no authorization
    surface is touched by this PR
  • I have updated the changelog.
  • I have ticked a box in "AI usage" in this PR

Storage config is resolved at runtime from STORAGE_BACKEND and STORAGE_PATH, so
the suite picked up whatever a developer had in .env. With STORAGE_BACKEND=gcs
set locally, four tests in export_worker_test.exs and projects_test.exs made
live calls against a real bucket and failed.

Stub Lightning.Config.storage/1 in the three case templates that already
stub_with MockConfig, defaulting to the local backend and the "." path the env
vars default to. Unknown keys fall through to Lightning.Config.API, and
individual tests can still override.
Uploads to GCS have been broken since Tesla 1.18.3 (#4913). Tesla now requires
multipart part names to be binaries, and google_gax labels its metadata part
with an atom, so storage_objects_insert_simple raises before sending anything.
Versions offer no way out: google_gax last shipped in 2021,
googleapis/elixir-google-api is archived, and 1.18.2 reintroduces the CVEs
1.18.3 closed.

store/2 now POSTs with uploadType=media, avoiding multipart entirely — the
object name is the only metadata we set and it travels in the query string. The
body is wrapped in Stream.map/2 so Tesla's adapters see a %Stream{} and send it
chunked; a bare %File.Stream{} falls through. delete/1 becomes a plain DELETE.
Both check the status explicitly, since Tesla returns {:ok, env} for a 403 or
500 and callers rely on non-2xx being an error tuple.

delete/1 also applies prefix_storage_path/1, which it never did. On deployments
with STORAGE_PATH set, every delete asked for a key that was never written, got
a 404, and left the object in the bucket while Projects dropped its
project_files row.

bucket!/0 reads Lightning.Config.storage(:bucket) rather than the application
env, making Config the single source of truth for storage config. A TokenSource
behaviour stands in for Goth, which only starts when STORAGE_BACKEND=gcs and so
never runs under test.

Drops google_api_storage, taking google_gax and poison with it.

Closes #5049
@github-project-automation github-project-automation Bot moved this to New Issues in Core Aug 6, 2026
@midigofrank midigofrank changed the title Fix broken project files gcs Replace google_api_storage with direct GCS JSON API calls Aug 6, 2026
@codecov

codecov Bot commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.65217% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 90.7%. Comparing base (3602ba7) to head (3ba55d0).

Files with missing lines Patch % Lines
lib/lightning/storage/gcs/token_source.ex 66.7% 1 Missing ⚠️
Additional details and impacted files
@@               Coverage Diff                @@
##           release-2.18.0   #5053     +/-   ##
================================================
+ Coverage            90.6%   90.7%   +0.1%     
================================================
  Files                 420     421      +1     
  Lines               19957   19972     +15     
================================================
+ Hits                18078   18118     +40     
+ Misses               1879    1854     -25     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

delete/1 now prefixes the object path with STORAGE_PATH, so it goes through
Path.safe_relative/1 and can fail where it never could before. Matching that
with {:ok, _} = raised a MatchError, and delete/1 runs inside the data
retention job's Enum.each over every expired file, so a single unusable path
would abort the cleanup for every project behind it.

prefix_storage_path/1 now returns {:error, {:unsafe_storage_path, path}} and
store/2 and delete/1 route through with/1. An absolute STORAGE_PATH is the
realistic way in: Path.safe_relative/1 rejects the joined path outright.
@midigofrank
midigofrank marked this pull request as ready for review August 6, 2026 15:29
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown

All PR changes are scoped and no S0/S1/S2 concerns emerge. Producing the pass-case output.

Security Review ✅

  • S0 (project scoping): WorkflowChannel.ensure_unique_name still queries via socket.assigns.project.id (authenticated membership), and Storage.delete/1 is invoked only against Projects.File rows already filtered by project_id at lib/lightning/projects.ex:1594-1610.
  • S1 (authorization): N/A — no new web-layer actions or handle_in/handle_event branches; the touched channel handlers (validate_workflow_name, streaming plumbing) retain their existing project-membership gates.
  • S2 (audit trail): N/A — no new writes to project or instance configuration; the AI-assistant and storage refactors don't add config-resource mutations, and the GCS store/delete paths still route through Path.safe_relative before hitting the JSON API.

@midigofrank
midigofrank merged commit 1dbeaa1 into release-2.18.0 Aug 6, 2026
8 checks passed
@midigofrank
midigofrank deleted the fix-broken-project-files-gcs branch August 6, 2026 15:37
@github-project-automation github-project-automation Bot moved this from New Issues to Done in Core Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant