Skip to content

Default new Ruby scrapers to heroku-24 instead of heroku-18 - #1532

Open
Br3nda wants to merge 2 commits into
mainfrom
default-new-ruby-scrapers-to-heroku-24
Open

Br3nda wants to merge 2 commits into
mainfrom
default-new-ruby-scrapers-to-heroku-24

Conversation

@Br3nda

@Br3nda Br3nda commented Sep 11, 2026

Copy link
Copy Markdown
Member

What does this do?

Changes the platform file in the Ruby scraper template (used when a new scraper is created via morph.io's "create a scraper" flow) from heroku-18 to heroku-24.

Why was this needed?

heroku-18's buildstep image hasn't been rebuilt since 2022 - CI build/push for it (and cedar-14) was disabled on 2025-05-21 (see openaustralia/buildstep's workflow files, "disabling the older builds so the existing working images don't get disturbed"). heroku-24 is the only stack still actively rebuilt.

Tested a real Ruby scraper end-to-end against the current openaustralia/buildstep:heroku-24 image: clean bundle install, no buildpack issues.

Scope - deliberately narrow

  • Only affects brand-new Ruby scrapers created from this point forward. default_files/ruby/template/platform is read once, at creation time, by CreateScraperWorker - existing scrapers (with or without their own platform file) are completely unaffected.
  • DEFAULT_PLATFORM (cedar-14, used when a scraper has no platform file at all) is untouched - so old scrapers relying on that implicit default keep behaving exactly as they do today.
  • Ruby only. Node/PHP/Perl are known or suspected broken on heroku-24 (see the existing # FIXME: get php and perl examples / buildstep working for heroku-24 in Language, and openaustralia/morph#1443/#1444/#1448) and haven't been individually re-verified - happy to do the same test-first pass on each if useful.
  • default_files/python/template/platform was already heroku-24 before this PR.

Testing done

herokuish buildpack build against a real scraper (unmodified Gemfile pinning ruby "3.2.2") run directly against openaustralia/buildstep:heroku-24, pulled fresh:

-----> Ruby app detected
-----> Using Ruby version: ruby-3.2.2
Bundle complete! 3 Gemfile dependencies, 21 gems now installed.

Opening as a draft while I sanity-check whether there's anything else worth doing alongside this (e.g. whether update_docker_images! needs to run for the change to actually take effect on morph.io's build hosts).

heroku-18's buildstep image hasn't been rebuilt since 2022 (CI build/push
for it and cedar-14 was disabled 2025-05-21, see buildstep's
.github/workflows). heroku-24 is the only actively-maintained stack,
rebuilt regularly. Tested a real scraper end-to-end against the current
openaustralia/buildstep:heroku-24 image - clean bundle install, no
buildpack issues.

Only affects the template written into brand-new Ruby scrapers created
via CreateScraperWorker - existing scrapers (with or without their own
platform file) are unaffected, since this file is only read once, at
creation time. DEFAULT_PLATFORM (cedar-14, used when a scraper has no
platform file at all) is untouched.

Scoped to Ruby only for now - Node/PHP/Perl are known or suspected
broken on heroku-24 (see Language::LANGUAGES_SUPPORTED's existing FIXME
and #1443/#1444/#1448) and haven't been individually
verified.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

@benrfairless benrfairless left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can't be done while we are on the old server. It will need to wait until we are on the new server with newer docker.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants