Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@ on:
required: true
type: string
release_profile:
description: Release evidence profile; Alpha permits deferred distributed evidence
description: Release evidence profile; standard permits deferred distributed evidence
required: false
default: alpha
default: standard
type: choice
options:
- alpha
- standard
- enterprise
formal_publish:
description: Publish using an existing immutable release tag after recovery approval
Expand Down Expand Up @@ -72,7 +72,7 @@ jobs:
CANDIDATE_REF: ${{ github.event_name == 'workflow_dispatch' && inputs.candidate_sha || github.ref }}
EXPECTED_VERSION: ${{ github.event_name == 'workflow_dispatch' && inputs.expected_version || '' }}
MODE: ${{ github.event_name == 'push' && 'tag' || inputs.formal_publish == true && 'tag' || 'dry-run' }}
RELEASE_PROFILE: ${{ github.event_name == 'workflow_dispatch' && inputs.release_profile || 'alpha' }}
RELEASE_PROFILE: ${{ github.event_name == 'workflow_dispatch' && inputs.release_profile || 'standard' }}
RELEASE_TAG: ${{ github.event_name == 'push' && github.ref_name || inputs.formal_publish == true && inputs.release_tag || '' }}
EVENT_SHA: ${{ github.event_name == 'push' && github.event.after || inputs.formal_publish == true && inputs.candidate_sha || '' }}
EVENT_CREATED: ${{ github.event_name == 'push' || inputs.formal_publish == true }}
Expand Down
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ at 2 GiB by default.
The slow workflow supports manual, reusable, and scheduled full runs. It is not a required check in
the default CI workflow. A successful full run emits a schema-versioned `slow-result.json`; the
enterprise release profile requires exactly one non-expired artifact from a successful workflow run
on the exact release SHA and revalidates its workflow run ID. The Alpha release profile may defer
on the exact release SHA and revalidates its workflow run ID. The standard release profile may defer
this artifact and must not claim enterprise or stable distributed certification. The `core` profile
is diagnostic only and never produces release evidence. The dedicated self-hosted runner must be
Linux x64 with Docker Compose, at least 16 GiB available memory, at least 20 GiB free disk, and
Expand Down Expand Up @@ -126,8 +126,8 @@ not describe an untested configuration or extension point as supported behavior.

## Compatibility

Code must remain compatible with the declared `ray[data]>=2.49.2,<2.59` range. Python 3.9 is an
Alpha legacy compatibility target because it no longer receives upstream security fixes; it isn't
Code must remain compatible with the declared `ray[data]>=2.49.2,<2.59` range. Python 3.9 is a
legacy compatibility target because it no longer receives upstream security fixes; it isn't
a stable production baseline. Do not import modules below `ray.data._internal`. Add a unit test for
every independently verifiable behavior, including error paths and cleanup. Unsupported Doris types
must fail closed. Datasink callback signatures and timing differ between Ray 2.49.2–2.52.x and
Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,20 @@ or Arrow Flight SQL, and writes bounded batches through Doris HTTP Stream Load.
imports `ray.data._internal`. Ray marks `ReadTask` as DeveloperAPI, so the supported Ray window is
intentionally bounded and tested by minor release.

The project is alpha software. The tested compatibility window is:
The tested compatibility window is:

| Python | Ray | Verification |
|---|---|---|
| 3.9 | 2.49.2 | Alpha legacy compatibility; documented ReadTask and public Datasink contracts |
| 3.9 | 2.49.2 | Legacy compatibility; documented ReadTask and public Datasink contracts |
| 3.10 | 2.57.0 | Unit, documented ReadTask, and public Datasink contract tests |
| 3.11 | 2.58.0 | Unit, documented/public contracts, base and Flight package installation, dependency checks, and import tests |
| 3.12 | 2.58.0 | Unit, documented/public contracts, and Doris 4.0.6 required IT |
| 3.13 | 2.58.0 | Unit, documented/public contracts, package installation, and import tests |

Doris 4.0.6 is the fixed compatibility target for both required and opt-in distributed integration
tests. The slow distributed suite is aligned to Ray 2.58.0. Enterprise-candidate releases require
one successful full run on the exact commit; this Alpha release does not claim that distributed
evidence or stable/production readiness.
one successful full run on the exact commit; the standard release profile does not claim
that distributed evidence or stable/production readiness.
This project is not maintained or endorsed by the Ray or Apache Doris projects.

## Documentation
Expand Down Expand Up @@ -48,7 +48,7 @@ pip install "ray-doris[flight]"
```

Flight SQL requires Python 3.10 or newer because current ADBC Flight SQL releases no longer support
Python 3.9. Python 3.9 reached end of life on October 31, 2025. It remains an Alpha legacy
Python 3.9. Python 3.9 reached end of life on October 31, 2025. It remains a legacy
compatibility target for the default MySQL transport, not a stable or production profile.

The package accepts `ray[data]>=2.49.2,<2.59`. The runtime guard supports final releases in that
Expand Down Expand Up @@ -416,7 +416,7 @@ only one successful, non-expired full manifest whose commit and workflow run ID
downloaded artifact source. The dedicated runner must provide Linux x64, Docker Compose, at least
16 GiB available memory, at least 20 GiB free disk, and Actions Runner 2.327.1 or newer.
Register it with the `ray-doris-slow-it` and `ray-doris-slow-it-node24` labels.
Enterprise release profiles require the full manifest. Alpha release profiles may defer the slow
Enterprise release profiles require the full manifest. Standard release profiles may defer the slow
evidence, but must not claim enterprise or stable distributed certification. Formal recovery of the
immutable `v1.0` release remains a separate historical exception.

Expand Down
10 changes: 5 additions & 5 deletions doc/source/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Continuous integration covers these combinations:

| Python | Ray | Verification |
| --- | --- | --- |
| 3.9 | 2.49.2 | Alpha legacy compatibility; documented ReadTask and public Datasink contract tests |
| 3.9 | 2.49.2 | Legacy compatibility; documented ReadTask and public Datasink contract tests |
| 3.10 | 2.57.0 | Unit, documented ReadTask, and public Datasink contract tests |
| 3.11 | 2.58.0 | Unit, documented/public contracts, base and Flight package installation, dependency checks, and import tests |
| 3.12 | 2.58.0 | Unit, documented/public contracts, and required Doris 4.0.6 read/write integration tests |
Expand All @@ -25,8 +25,8 @@ Continuous integration covers these combinations:
The optional distributed suite uses Python 3.12, Ray 2.58.0, and Doris 4.0.6. It runs one Ray head
with no scheduling CPUs, three one-CPU Ray workers, one Doris frontend, three Doris backends, and a
TLS and Flight ingress. The workflow is now aligned to Ray 2.58.0. Enterprise-candidate releases
require one successful full run on their exact commit; the Alpha 1.1 release does not claim that
distributed evidence or stable/production readiness.
require one successful full run on their exact commit; the standard release profile does not claim
that distributed evidence or stable/production readiness.
The enterprise-candidate profile uses the minimum-privilege MySQL reader; Flight remains an
experimental regression path.

Expand Down Expand Up @@ -55,7 +55,7 @@ leader election, quorum, multi-FE failover, or an external load balancer's backe

## Review Python and Flight limits

The core package requires Python 3.9 or newer. Python 3.9 reached end of life on October 31, 2025, so it is retained only as an Alpha legacy compatibility target. It isn't part of a stable or production profile. A future stable release will require a Python line that still receives upstream security fixes at its release date.
The core package requires Python 3.9 or newer. Python 3.9 reached end of life on October 31, 2025, so it is retained only as a legacy compatibility target. It isn't part of a stable or production profile. Use a Python line that still receives upstream security fixes for production deployments.

Python 3.11 is covered by unit, public Datasink contract, base and Flight wheel installation,
dependency checks, and import tests. It doesn't have a separate real-Doris deployment
Expand Down Expand Up @@ -102,7 +102,7 @@ The required integration suite uses the default frontend HTTP query-plan endpoin
The enterprise-candidate MySQL profile requires `password_env`, strict MySQL TLS, HTTPS query
planning with hostname verification, `on_query_plan_error="error"`, and explicit query-plan and
MySQL socket timeouts. The enterprise release profile requires a successful full slow manifest bound
to the exact release SHA and workflow run. The Alpha release profile may defer that evidence and
to the exact release SHA and workflow run. The standard release profile may defer that evidence and
must state the resulting distributed-certification limitation.

Doris 4.0.6 advertises plaintext Flight `grpc` endpoints in the distributed suite. Flight stays on an isolated Compose network. The test doesn't establish a positive `grpc+tls` Doris server compatibility claim.
2 changes: 1 addition & 1 deletion doc/source/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ myst:

`ray-doris` is an independent, community-maintained Apache Doris connector for Ray Data. It plans reads with the Doris frontend query-plan endpoint, assigns tablet groups to Ray read tasks, streams rows through MySQL or Arrow Flight SQL, and writes bounded Parquet or line-delimited JSON batches through HTTP Stream Load without importing Ray Data internals.

The project is alpha software and isn't maintained or endorsed by the Ray or Apache Doris projects. See [Compatibility](compatibility.md) for the tested version matrix and [Key concepts](key-concepts.md) for the execution model.
The project isn't maintained or endorsed by the Ray or Apache Doris projects. See [Compatibility](compatibility.md) for the tested version matrix and [Key concepts](key-concepts.md) for the execution model.

MySQL is the production-candidate transport. Flight SQL and automatic transport selection remain experimental.

Expand Down
2 changes: 1 addition & 1 deletion doc/source/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ uv pip install -e ".[flight]"

Flight SQL requires Python 3.10 or newer. The default MySQL transport remains available on Python 3.9.

Flight SQL and `auto` are experimental. Use the default MySQL transport for the production-candidate profile. Python 3.9 is retained only as an Alpha legacy compatibility target because it no longer receives upstream security fixes.
Flight SQL and `auto` are experimental. Use the default MySQL transport for the production-candidate profile. Python 3.9 is retained only as a legacy compatibility target because it no longer receives upstream security fixes.

## Prepare Doris access

Expand Down
3 changes: 1 addition & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,14 @@ build-backend = "hatchling.build"

[project]
name = "ray-doris"
version = "1.1"
version = "1.2"
description = "Community-maintained Apache Doris connector for Ray Data"
readme = "README.md"
requires-python = ">=3.9"
license = "Apache-2.0"
license-files = ["LICENSE", "NOTICE"]
authors = [{ name = "jiangxt2" }]
classifiers = [
"Development Status :: 3 - Alpha",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
Expand Down
27 changes: 27 additions & 0 deletions release-notes/v1.2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# ray-doris 1.2

## Highlights

- Add Python 3.11 / Ray 2.58.0 to the tested compatibility matrix, including unit and public
contract tests, base and Flight wheel installation, dependency checks, and import tests.
- Check installed dependencies for the Python 3.12 wheel and source distribution smoke tests.
- Validate that the README and compatibility documentation match the CI Python / Ray matrix.
- Remove the project-wide Alpha classifier and current maturity label without assigning a Beta or
Stable classifier.
- Rename the default release evidence profile from `alpha` to `standard` while preserving its
existing validation requirements.

## Compatibility and release scope

- Keep the package version format at two components: `major.minor`.
- Keep the existing Python, Ray, and Doris compatibility bounds and read/write contracts.
- Retain Python 3.9 as a legacy compatibility target outside the production profile.
- Keep MySQL as the production-candidate read transport. Flight SQL, automatic transport selection,
and deployment-specific Flight TLS remain experimental.
- Required real-Doris integration tests continue to use Python 3.12 and Doris 4.0.6. Python 3.11
coverage does not add a separate real-Doris deployment certification.
- The standard release profile validates CI, package artifacts, and installation smoke tests. It
may defer exact-candidate full distributed evidence and makes no stable or enterprise distributed
certification claim. The enterprise profile continues to require that evidence.
- Write task retries remain disabled. Stream Load requests do not provide whole-Dataset atomicity
or exactly-once semantics.
2 changes: 1 addition & 1 deletion src/ray_doris/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,4 +48,4 @@
"write_doris",
]

__version__ = "1.1"
__version__ = "1.2"
31 changes: 31 additions & 0 deletions tests/it-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,3 +124,34 @@ Doris evidence remains valid under the no-runtime-change test reuse rule.
- required Doris IT, slow distributed IT, and benchmark: not run because production source,
dependencies, protocol behavior, Doris tests, Compose, and execution configuration are unchanged;
the final Python 3.12 / Ray 2.58.0 required IT result above remains reusable.

## Version 1.2 release preparation evidence

- worktree: `/Users/jiangxintong/GitHub/workspace/ray-doris-release-maturity`, branch
`release-maturity`; base revision: `a1f1f60b740d28a4dd5625577e443c7fa3c86349`;
- scope: version metadata, current maturity wording, default release profile naming, workflow policy,
and release unit tests; runtime behavior, dependency bounds, Doris tests, Compose, and distributed
execution configuration are unchanged;
- Python 3.12.12 / Ray 2.58.0 / PyArrow 25.0.1 / PyMySQL 2.2.8: final full unit and public contract
suite 318 passed with 85% aggregate coverage; the three warnings are Ray public-context
deprecations; targeted release tests passed all 66 cases;
- static and policy checks: Ruff format/lint, strict mypy for 18 source files, pre-commit, workflow
action pins, release-workflow policy, and `git diff --check` passed;
- documentation: `tools/check_docs.py`, Sphinx strict, spelling, and the separately executed external
linkcheck passed;
- package: isolated version 1.2 wheel and sdist build, Twine checks, and SHA-256 manifest validation
passed. Both distribution metadata records and embedded package versions match 1.2 and contain
no Development Status classifier. Four isolated Python 3.12 wheel/sdist base/Flight installations
passed dependency checks and imports outside the repository;
- logs: `/private/tmp/ray-doris-release-maturity.EFfzme`, including `unit-contract.xml`,
`unit-contract.log`, `release-unit.log`, `static.log`, `docs.log`, `linkcheck.log`, `package.log`,
`install-smoke.log`, and `SHA256SUMS`; package checks apply to the tested tree before this
evidence-only ledger append;
- required Doris IT: not rerun locally; the existing final Python 3.12 / Ray 2.58.0 / Doris 4.0.6
result above remains reusable because runtime behavior and infrastructure are unchanged;
- full distributed evidence: deferred under the standard release profile. Renaming that profile
does not establish stable or enterprise distributed certification; the enterprise profile still
requires a successful full manifest for the exact release candidate;
- release state: local preparation only. No 1.2 commit, push, tag, TestPyPI, PyPI, or GitHub Release
has been created. Exact candidate SHA, workflow results, and published artifact hashes must be
recorded after the separately approved external steps.
Loading
Loading