Skip to content

MNT: compile (pin) doc build dependencies - #302

Draft
neutrinoceros wants to merge 1 commit into
idefix-code:developfrom
neutrinoceros:doc/compile_build_dependencies
Draft

MNT: compile (pin) doc build dependencies#302
neutrinoceros wants to merge 1 commit into
idefix-code:developfrom
neutrinoceros:doc/compile_build_dependencies

Conversation

@neutrinoceros

@neutrinoceros neutrinoceros commented Dec 17, 2024

Copy link
Copy Markdown
Collaborator

This is a first step towards #191 (missing the automated updates for now, see below)

Notes:

  • I created a distinction between the set of direct dependencies (requirements.in) and the complete set of dependencies (direct dependencies + their dependencies + ...) which are requirements*.txt files with everything pinned (except the Python runtime version, but that's done in .readthedocs.yaml), ususally refered to as a "compiled" requirement file
  • compiled requirements can be obtained with either of the following commands (either can also be used to upgrade requirements later)
    • uv pip compile requirements.in (via uv) (this is the tool I'm using, for reference)
    • pip-compile requirements.in (via pip-tools)
  • seeing that direct dependencies were initially pinned in python_requirements.txt (renamed to requirements.in), I produced two set of compiled requirements:
    • one with this constraint still applied (this is requirements_direct_pinned.txt)
    • one with == replaced by >= (this is requirements.txt)

I would recommend using the later (if it works), but the former is probably closer to what has been running on RTD as of late so it can serve as a fallback if the most up-to-date set doesn't fly.

I'm also keeping auto-updates via dependabot for a follow up PR because I want to first know which version of compiled requirements should be kept. This might be a little tricky to test: as far as I'm aware, RTD only runs once a patch is merged (is this right ?).
Also, I'm assuming RTD runs on commits made to master, not develop, so maybe I should target master with this PR ?

@neutrinoceros neutrinoceros added doc label for documentation related things to be done project maintenance/infrastructure labels Dec 17, 2024
@glesur

glesur commented Dec 19, 2024

Copy link
Copy Markdown
Contributor

Thanks @neutrinoceros . However, I must say I don't know how to test this... and I'm not keen on merging this straight away onto master.

@neutrinoceros

Copy link
Copy Markdown
Collaborator Author

I should be able to build docs locally with both files and send you the artifacts if you'd like.

@neutrinoceros

Copy link
Copy Markdown
Collaborator Author

So actually it's pretty simple: requirements_direct_pinned.txt is the only alternative that works at the moment. I'll replace requirements.txt with it.

@neutrinoceros
neutrinoceros force-pushed the doc/compile_build_dependencies branch from 8115080 to a007523 Compare December 19, 2024 08:24
@neutrinoceros

Copy link
Copy Markdown
Collaborator Author

I you'd like, I could add a CI job that mimicks what RTD does, so you don't have to rely on local/manual testing everytime something is upgraded. I think it's also possible to configure RTD to build on pull requests (without publishing the result, of course), but that's out of my hands.

@glesur

glesur commented Dec 19, 2024

Copy link
Copy Markdown
Contributor

I think that we definitely need a CI job to check the doc. I can have a go at this (or possibly @lih ). Let me know if you start something.

@neutrinoceros

Copy link
Copy Markdown
Collaborator Author

I don't think I will have time to work on it before next year (🥁), but I can certainly pick it up then if needed.

@neutrinoceros

Copy link
Copy Markdown
Collaborator Author

rebasing after #305 was just merged

@neutrinoceros
neutrinoceros force-pushed the doc/compile_build_dependencies branch from 67f1f60 to ad14ca4 Compare December 23, 2024 12:13
@smaret

smaret commented Dec 23, 2024

Copy link
Copy Markdown

Instead of having a CI to check that the documentation builds, followed by RTD building it on their side, you could use a CI that builds the documentation and then pushes the HTML pages to GitHub Pages (if the build succeeds).

Here is a GitHub Action that I wrote for another project. Feel free to use it if you find it helpful.

@glesur

glesur commented Feb 21, 2025

Copy link
Copy Markdown
Contributor

Thanks @smaret . For the moment nobody has time to migrate the documentation to Github Pages, so we will keep this as is.

@neutrinoceros

Copy link
Copy Markdown
Collaborator Author

I'm planning to rework this with a pylock.toml file instead of requirements.txt, in the spirit of #395 if it is accepted.

@neutrinoceros
neutrinoceros marked this pull request as draft July 30, 2026 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc label for documentation related things to be done project maintenance/infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants