From 4cc3085be2932e99d1c28eaa8e5d10049304d9c7 Mon Sep 17 00:00:00 2001 From: Rob E Date: Tue, 4 Aug 2026 16:30:52 +1000 Subject: [PATCH 1/4] Adding notes no release process --- docs/README.md | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 docs/README.md diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..bf6fe06 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,99 @@ +## Image Release Process + +**Note: Unless a semantic version build tag is set on the commit being triggered, TeamCity will generate a pre-release build of the images. This should not go out to Production** + +## 👷 Build +When a change is made to a file in this repository, TeamCity triggers the [full build chain](https://build.octopushq.com/buildConfiguration/OctopusDeploy_WorkerTools_ChainBuildTestAndRelease). +This results in a build of all the different distro's dockerfiles, as well as seperate architecture for non-windows os. + +As part of this build process, individual images are pushed to the internal artifactory image repository. + +## 🐙 Deploy +As the end of a successful build, a release is created in the [Worker Tools vNext](https://deploy.octopus.app/app#/Spaces-1163/projects/worker-tools-vnext/deployments?groupBy=Channel") Octopus Deploy Project. + +This project takes the individually built images from Artifactory, and pushes them to a target repository as well as a multi-arch image to that same registry. As we create multiple architecture images for `Ubuntu 24.04` (amd64 and arm64), an additional `ubuntu.24.04` multi-arch image is created and pushed. + + +For the `Staging` environment the target is the same Artifactory registry where they are pushed during the build chain so then individual image push essentially becomes a no-op. The multi-arch images however will still be pushed. +For `Production` the target repository is the [public DockerHub registry](https://hub.docker.com/r/octopusdeploy/worker-tools). + +## Flow Diagram + +```mermaid +flowchart TD + subgraph repo[🧑‍💻 This Git Repository] + direction LR + +RepoDirs@{ shape: docs, label: "Per Platform Docker Config + +* _ubuntu.22.04_ +* _ubuntu.24.04_ +* _windows.ltsc2022_ +* _windows.ltsc2025_"} + end + + subgraph ci[👷 Team City] + Chain[Chain: Build Test & Release] + ReleaseAll[Release:All] + Builds@{ shape: docs, label: "Per Platform Build Config + +* _ubuntu 22.04_ +* _ubuntu 22.04 x64_ +* _Windows 2022_ +* _Windows 2025_"} + Chain --> ReleaseAll + ReleaseAll --> Builds + + + + click Chain "https://build.octopushq.com/project/OctopusDeploy_WorkerTools?mode=builds#all-projects" "TeamCity" + end + + subgraph Artifactory[📦 Artifactory] + ArtifactoryWorkerTools[docker.packages.octopushq.com/octopusdeploy/workertools] + end + + subgraph DockerHub[📦 Docker Hub] + DockerHubWorkerTools[docker.io/octopusdeploy/worker-tools] + click DockerHubWorkerTools "https://hub.docker.com/r/octopusdeploy/worker-tools" + end + + subgraph Octopus[🐙 Octopus Deploy] + subgraph Project[Worker Tools vNext] + + Staging + Prod + Staging-->Prod + end + click Prod "https://deploy.octopus.app/app#/Spaces-1163/projects/worker-tools-vnext/deployments?groupBy=Channel" + click Staging "https://deploy.octopus.app/app#/Spaces-1163/projects/worker-tools-vnext/deployments?groupBy=Channel" + end + + Builds --> ArtifactoryWorkerTools + +Artifactory -.-> Staging +Artifactory -.-> Prod +Staging --> |Push MultiArch Image|ArtifactoryWorkerTools +Prod --> |Push OS Images|DockerHubWorkerTools +Prod --> |Push MultiArch Image|DockerHubWorkerTools +Prod --> |Push Latest Tag|DockerHubWorkerTools +repo moving2@==> |CI trigger|ci + +ReleaseAll moving1@==> |On Success|Staging + +moving1@{ animate: true } +moving2@{ animate: true } + +``` + +## Consumption in Octopus Deploy +When presented with the option to run an `Execution Container Image`, users can select this pre-built worker-tool image. + +image + +The list of images presented is dynamically retrieved from DockerHub and so, once the image has been pushed to `Production`, it potentially becomes the default image that customer will use when configuring a new step. + +Unfortunately if the image is not yet cached on the Dynamic Worker VM Image, cloud customers may find that the image pull process takes an unreasonable amount of time. For this reason, it is reccomended that pushing to `Production` is only done once the new image has been sucessfully cached in the available Dynamic Worker VM. + +## Dynamic Worker Images +TODO: Add To This From 6b0bf88e1df81b45e1f55274a79de75879877ca1 Mon Sep 17 00:00:00 2001 From: Rob E Date: Tue, 11 Aug 2026 09:14:28 +1000 Subject: [PATCH 2/4] Revise README for clarity and additional information Updated README to enhance clarity and add new sections regarding artifact storage and dynamic worker images. --- docs/README.md | 96 +++++++++----------------------------------------- 1 file changed, 17 insertions(+), 79 deletions(-) diff --git a/docs/README.md b/docs/README.md index bf6fe06..e2ad6ad 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,6 +1,8 @@ ## Image Release Process -**Note: Unless a semantic version build tag is set on the commit being triggered, TeamCity will generate a pre-release build of the images. This should not go out to Production** +**⚠️ Note: Unless a semantic version build tag is set on the commit being triggered, TeamCity will generate a pre-release build of the images. This should not go out to Production** + +`GitHub` Dockerfile Per Platform --> `TeamCity` [Chain:ReleaseAll](https://build.octopushq.com/project/OctopusDeploy_WorkerTools?mode=builds#all-projects) --> `Octopus (Dev)` Push to Artifactory --> `Octopus (Prod)` Push to DockerHub ## 👷 Build When a change is made to a file in this repository, TeamCity triggers the [full build chain](https://build.octopushq.com/buildConfiguration/OctopusDeploy_WorkerTools_ChainBuildTestAndRelease). @@ -8,83 +10,18 @@ This results in a build of all the different distro's dockerfiles, as well as se As part of this build process, individual images are pushed to the internal artifactory image repository. -## 🐙 Deploy -As the end of a successful build, a release is created in the [Worker Tools vNext](https://deploy.octopus.app/app#/Spaces-1163/projects/worker-tools-vnext/deployments?groupBy=Channel") Octopus Deploy Project. - -This project takes the individually built images from Artifactory, and pushes them to a target repository as well as a multi-arch image to that same registry. As we create multiple architecture images for `Ubuntu 24.04` (amd64 and arm64), an additional `ubuntu.24.04` multi-arch image is created and pushed. - - -For the `Staging` environment the target is the same Artifactory registry where they are pushed during the build chain so then individual image push essentially becomes a no-op. The multi-arch images however will still be pushed. -For `Production` the target repository is the [public DockerHub registry](https://hub.docker.com/r/octopusdeploy/worker-tools). - -## Flow Diagram - -```mermaid -flowchart TD - subgraph repo[🧑‍💻 This Git Repository] - direction LR - -RepoDirs@{ shape: docs, label: "Per Platform Docker Config - -* _ubuntu.22.04_ -* _ubuntu.24.04_ -* _windows.ltsc2022_ -* _windows.ltsc2025_"} - end - - subgraph ci[👷 Team City] - Chain[Chain: Build Test & Release] - ReleaseAll[Release:All] - Builds@{ shape: docs, label: "Per Platform Build Config - -* _ubuntu 22.04_ -* _ubuntu 22.04 x64_ -* _Windows 2022_ -* _Windows 2025_"} - Chain --> ReleaseAll - ReleaseAll --> Builds - +## 📦 Artifact Storage +* **Artifactory**: (pre-prod) _docker.packages.octopushq.com/octopusdeploy/workertools_ +* **DockerHub** (prod) [docker.io/octopusdeploy/worker-tools](https://hub.docker.com/r/octopusdeploy/worker-tools) - - click Chain "https://build.octopushq.com/project/OctopusDeploy_WorkerTools?mode=builds#all-projects" "TeamCity" - end - - subgraph Artifactory[📦 Artifactory] - ArtifactoryWorkerTools[docker.packages.octopushq.com/octopusdeploy/workertools] - end - - subgraph DockerHub[📦 Docker Hub] - DockerHubWorkerTools[docker.io/octopusdeploy/worker-tools] - click DockerHubWorkerTools "https://hub.docker.com/r/octopusdeploy/worker-tools" - end - - subgraph Octopus[🐙 Octopus Deploy] - subgraph Project[Worker Tools vNext] - - Staging - Prod - Staging-->Prod - end - click Prod "https://deploy.octopus.app/app#/Spaces-1163/projects/worker-tools-vnext/deployments?groupBy=Channel" - click Staging "https://deploy.octopus.app/app#/Spaces-1163/projects/worker-tools-vnext/deployments?groupBy=Channel" - end - - Builds --> ArtifactoryWorkerTools - -Artifactory -.-> Staging -Artifactory -.-> Prod -Staging --> |Push MultiArch Image|ArtifactoryWorkerTools -Prod --> |Push OS Images|DockerHubWorkerTools -Prod --> |Push MultiArch Image|DockerHubWorkerTools -Prod --> |Push Latest Tag|DockerHubWorkerTools -repo moving2@==> |CI trigger|ci - -ReleaseAll moving1@==> |On Success|Staging - -moving1@{ animate: true } -moving2@{ animate: true } - -``` +## 🐙 Deploy +The deployment process in [Worker Tools vNext](https://deploy.octopus.app/app#/Spaces-1163/projects/worker-tools-vnext/deployments?groupBy=Channel) performs some Image manipulation/pushing. +1. Pull each Image from Artifactory (where TC Build Pushes each) +2. Push each Image to target registry (results in a no-op for pre-prod which uses the same Artifactory registry it just pushed from +3. Create multi-arch `:` taged image across all distros and push to target registry +4. Create multi-arch `:ubuntu-` image and push to target registry (Windows does not support ARM so no need) +5. +**⚠️ Note: Until the Cloud infrastructure has been updated to use a Dynamic Worker VM that contains a cached version of the newly created image then it should not be pushed to production. See Below for more details.** ## Consumption in Octopus Deploy When presented with the option to run an `Execution Container Image`, users can select this pre-built worker-tool image. @@ -95,5 +32,6 @@ The list of images presented is dynamically retrieved from DockerHub and so, onc Unfortunately if the image is not yet cached on the Dynamic Worker VM Image, cloud customers may find that the image pull process takes an unreasonable amount of time. For this reason, it is reccomended that pushing to `Production` is only done once the new image has been sucessfully cached in the available Dynamic Worker VM. -## Dynamic Worker Images -TODO: Add To This +## ☁️ Octopus Cloud Dynamic Worker Images +Not until the +[Cloud Platform deploy space](https://deploy.octopus.app/app#/Spaces-142/projects?searchValue=Dynamic+Worker+Images&page=1&pageSize=50&hideInactiveProjects=false) contains a project per VM. From 9026ccd29878fb197f28b8de0a270579592f41a7 Mon Sep 17 00:00:00 2001 From: Rob E Date: Thu, 13 Aug 2026 11:59:17 +1000 Subject: [PATCH 3/4] Update README.md --- docs/README.md | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/docs/README.md b/docs/README.md index e2ad6ad..44d737a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -2,7 +2,11 @@ **⚠️ Note: Unless a semantic version build tag is set on the commit being triggered, TeamCity will generate a pre-release build of the images. This should not go out to Production** -`GitHub` Dockerfile Per Platform --> `TeamCity` [Chain:ReleaseAll](https://build.octopushq.com/project/OctopusDeploy_WorkerTools?mode=builds#all-projects) --> `Octopus (Dev)` Push to Artifactory --> `Octopus (Prod)` Push to DockerHub +1. `GitHub` Dockerfile Per Platform +2. `TeamCity` [Chain:ReleaseAll](https://build.octopushq.com/project/OctopusDeploy_WorkerTools?mode=builds#all-projects) +3. `Octopus (Dev)` Push to Artifactory +4. Wait For Worker Tools VM to push to Production (see below) +5. `Octopus (Prod)` Push to DockerHub ## 👷 Build When a change is made to a file in this repository, TeamCity triggers the [full build chain](https://build.octopushq.com/buildConfiguration/OctopusDeploy_WorkerTools_ChainBuildTestAndRelease). @@ -35,3 +39,16 @@ Unfortunately if the image is not yet cached on the Dynamic Worker VM Image, clo ## ☁️ Octopus Cloud Dynamic Worker Images Not until the [Cloud Platform deploy space](https://deploy.octopus.app/app#/Spaces-142/projects?searchValue=Dynamic+Worker+Images&page=1&pageSize=50&hideInactiveProjects=false) contains a project per VM. + +Go to the Image deployment (e.g [x](https://build.octopushq.com/buildConfiguration/CloudPlatform_DynamicWorkers_DynamicWorkerVmImages_Ubuntu2204/23691299?showLog=23691299_2731_39.67.645.2501.2731&logView=flowAware)), find the latest build and look for the line entry describing the cached versions, looking for the header `Cache Docker Images`. At the bottom of this you will see log entry +``` +azure-arm.ubuntu: Selecting the following images due to maximum image count of 3 : +azure-arm.ubuntu: - "6.6.5-ubuntu.22.04" +azure-arm.ubuntu: - "6.6.4-ubuntu.22.04" +azure-arm.ubuntu: - "6.6.3-ubuntu.22.04" +``` + +For each of the relevant distros, check the earliest build that had the new docker image cached. + +Once _all_ of these builds have been deployed to Production from the [Dynamic Worker Images Project](https://deploy.octopus.app/app#/Spaces-142/projects?includeProjectGroupId=ProjectGroups-1841&searchValue=Dynamic+Worker+Images&page=1&pageSize=50&hideInactiveProjects=false) Group, then the docker images can be released to production. + From 1e2204fdd21905a8800db4213ea3b1758ffff7ab Mon Sep 17 00:00:00 2001 From: Rob E Date: Thu, 13 Aug 2026 12:06:42 +1000 Subject: [PATCH 4/4] Update README.md --- docs/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/README.md b/docs/README.md index 44d737a..5c057b1 100644 --- a/docs/README.md +++ b/docs/README.md @@ -5,7 +5,7 @@ 1. `GitHub` Dockerfile Per Platform 2. `TeamCity` [Chain:ReleaseAll](https://build.octopushq.com/project/OctopusDeploy_WorkerTools?mode=builds#all-projects) 3. `Octopus (Dev)` Push to Artifactory -4. Wait For Worker Tools VM to push to Production (see below) +4. Wait For Worker Tools VM to push to Production Tenants (see below) 5. `Octopus (Prod)` Push to DockerHub ## 👷 Build @@ -50,5 +50,7 @@ azure-arm.ubuntu: - "6.6.3-ubuntu.22.04" For each of the relevant distros, check the earliest build that had the new docker image cached. +Each Octopus Project has a `Cautious Promoter` step which runs the [Cautious Promoter](https://github.com/OctopusDeploy/CautiousPromoter) tool to determin when to push to each Production Tenant. + Once _all_ of these builds have been deployed to Production from the [Dynamic Worker Images Project](https://deploy.octopus.app/app#/Spaces-142/projects?includeProjectGroupId=ProjectGroups-1841&searchValue=Dynamic+Worker+Images&page=1&pageSize=50&hideInactiveProjects=false) Group, then the docker images can be released to production.