Repository navigation
chore(deps): update dependency mlflow to v3.17.2 [security] - #120
Merged
Merged
Conversation
red-hat-konflux-kflux-prd-rh02
Bot
force-pushed
the
konflux/mintmaker/main/pypi-mlflow-vulnerability
branch
from
November 1, 2026 12:02
038c594 to
fc62ac4
Compare
Signed-off-by: red-hat-konflux-kflux-prd-rh02 <190377777+red-hat-konflux-kflux-prd-rh02[bot]@users.noreply.github.com>
red-hat-konflux-kflux-prd-rh02
Bot
force-pushed
the
konflux/mintmaker/main/pypi-mlflow-vulnerability
branch
from
November 1, 2026 16:02
fc62ac4 to
c6cf4fa
Compare
Contributor
Author
Edited/Blocked NotificationRenovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR. You can manually request rebase by checking the rebase/retry box above. |
rhartuv
approved these changes
Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
3.16.0→3.17.2MLflow vulnerable to remote code execution on unauthenticated servers via third-party scorer deserialization importing caller-placed artifact modules
CVE-2026-116692 / GHSA-26p8-2jq9-3vq9
More information
Details
Summary
MLflow 3.12.0 introduced registration and deserialization support for third-party scorers in OSS MLflow. During scorer deserialization, the server accepts a request-controlled Python module path beneath one of four allowed package prefixes, imports that module, obtains a request-controlled class, and instantiates it synchronously.
An HTTP caller can combine this behavior with MLflow's built-in experiment, run, and artifact-upload endpoints:
<artifact_location>/<run_id>/artifacts/./ajax-api/3.0/mlflow/scorer/invoke.On the default no-auth server using the official MLflow v3.14.0 image, this produces unauthenticated arbitrary Python code execution with the privileges of the MLflow server process. The reproduced image runs the process as uid 0 and permits writes to its installed Python package tree.
The attached proof of concept is intentionally harmless. The uploaded constructor performs one action only: it writes a random marker beside its own uploaded run artifact. The marker is then downloaded through MLflow's run-artifact endpoint and checked for an exact match.
Technical details
1. The allow-list accepts attacker-created descendants
At release commit
86cd7f5a1dc25a1387ad07c87811bbc30f62951c,mlflow/genai/scorers/scorer_utils.pydefines these allowed roots:In
mlflow/genai/scorers/base.py, the deserializer accepts either an exact root or any descendant:It then performs the following operations using request-controlled values:
A package-name prefix is not a provenance boundary. If the server process can write below an allowed package, an attacker-created descendant can satisfy the string-prefix check and become importable Python code.
Relevant source locations at the v3.14.0 release commit:
mlflow/genai/scorers/scorer_utils.py, allowed third-party scorer modulesmlflow/genai/scorers/base.py, third-party scorer deserialization and construction2. Deserialization occurs before trace processing
In
mlflow/server/handlers.py,_invoke_scorer_handler()parses the supplied serialized scorer and calls:Only after deserialization and construction does the handler call:
The attacker-controlled constructor therefore runs before the server validates or resolves the supplied trace IDs.
The minimal proof class is deliberately not a complete MLflow scorer. After its constructor writes the marker, later trace-batching logic accesses scorer attributes that the proof class does not implement. The resulting HTTP 500 occurs after code execution and does not invalidate the proof.
No background job execution or special job-execution server option is required to reach the constructor.
3. MLflow's artifact layout places the module under the trusted prefix
The experiment creation endpoint accepts an
artifact_location. For file-backed experiments, MLflow derives each run artifact directory as:The proof of concept selects this package directory in the official v3.14.0 image:
After creating a run and uploading
audit_<token>.py, the resulting filesystem location is:The corresponding import path is:
The attack does not overwrite an existing package module. It creates a new run-scoped module in the normal file-backed artifact layout. The run ID is returned to the caller.
4. Authentication and authorization behavior
With authentication disabled, as in the default reproduced server configuration, no credentials are required.
When MLflow basic authentication is enabled, authentication still applies. However, the scorer-invocation AJAX endpoint is assigned to
validate_gateway_proxy, which returnsTruewithout performing an experiment- or scorer-level resource permission check.Practical exploitation in an authenticated deployment still requires the caller to create an experiment and run and upload artifacts to them. The key authorization issue is that scorer deserialization itself is not gated by a resource-specific permission check for the supplied experiment before the import occurs.
Proof of concept
The attached reproduction is intentionally isolated:
Run:
Expected output:
The security-relevant result is:
This proves that the attacker-controlled constructor executed in the MLflow server process before the later scorer-processing failure.
Reproduction environment
86cd7f5a1dc25a1387ad07c87811bbc30f62951cghcr.io/mlflow/mlflow:v3.14.0sha256:967de3ca34b6a5751718e380a0a383eb0ce1a6f5b484f363bd0da7559117a5f8127.0.0.1:5000sqlite:////work/mlflow.db/work/artifactsServer command inside the disposable container:
Affected range and source lineage
The vulnerable third-party scorer deserialization behavior was introduced by commit:
The feature is absent from v3.11.1 and first appears in v3.12.0.
Runtime reproduction succeeds on v3.14.0. The relevant
mlflow/genai/scorers/base.pycontent is unchanged at master commit:No fixed release was identified as of 2026-07-12.
Impact
On the reproduced stock no-auth image, any network caller who can reach the tracking server can execute Python code as the MLflow process without credentials or user interaction.
Successful exploitation provides confidentiality, integrity, and availability impact over all data, credentials, network access, and filesystem locations available to the MLflow server process.
The stock image's runtime uid 0 increases container-level impact. Host impact still depends on the container runtime configuration, mounted paths, granted capabilities, socket exposure, and other isolation controls.
In authentication-enabled deployments, exploitation requires an authenticated principal capable of creating an experiment and run and uploading artifacts. The exact chain also requires the server process to be able to write beneath one of the allowed scorer packages. This condition holds in the reproduced official image and may hold in virtual environments installed and run by the same operating-system user.
A read-only container filesystem or a scorer package directory owned by a different, non-writable user blocks this exact placement chain.
Artifacts and attachments
Remote proof-of-concept script
POC Script: new_remote_poc.py
The remote proof of concept was then executed as follows. The disposable machine’s public IP has been redacted:
The complete sanitized output was:
The proof file was also verified directly inside the disposable container:
The file contained:
{ "cwd": "/", "machine": "x86_64", "marker": "MLFLOW_REMOTE_SAFE_MARKER_1783843760683972103", "os_release": "PRETTY_NAME=\"Debian GNU/Linux 11 (bullseye)\"\nNAME=\"Debian GNU/Linux\"\nVERSION_ID=\"11\"\nVERSION=\"11 (bullseye)\"\nVERSION_CODENAME=bullseye\nID=debian\nHOME_URL=\"https://www.debian.org/\"\nSUPPORT_URL=\"https://www.debian.org/support\"\nBUG_REPORT_URL=\"https://bugs.debian.org/\"\n", "platform": "Linux-7.0.0-1006-aws-x86_64-with-glibc2.31", "python_implementation": "CPython", "python_version": "3.10.18", "server_datetime_utc": "2026-07-12T08:09:43.555055+00:00", "whoami": "root", "whoami_returncode": 0, "whoami_stderr": "" }The results demonstrate that:
rootin the reproduced official image;The authoritative reproduction indicators are:
The disposable cloud machine and container were removed after testing.
Local Reproduction Scripts:
run-official-image.sh
poc_local.py
Severity
CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:HReferences
This data is provided by OSV and the GitHub Advisory Database (CC-BY 4.0).
MLflow: SSRF on unauthenticated MLflow servers via third-party scorer kwargs in the scorer invoke, register, and GenAI evaluate endpoints
BIT-mlflow-2026-115886 / CVE-2026-115886 / GHSA-2wqp-q3c6-ccj5
More information
Details
Summary
MLflow's tracking server exposes
POST /ajax-api/3.0/mlflow/scorer/invokewith no authentication, and the handler reflectively instantiates an attacker-named class from attacker-supplied JSON with unconstrained constructor kwargs. With the optionaltrulens+trulens-providers-litellmpackages installed, an unauthenticated attacker can force the server to make an outbound HTTP request to an arbitrary host of their choosing, carrying attacker-controlled headers (SSRF).Details
Scorer.model_validate_json()reflects a class name from a JSON payload behind an import-path allowlist that never constrains the reflected attribute or its kwargs, inmlflow/genai/scorers/base.py:549-604:The allowlist (
mlflow/genai/scorers/scorer_utils.py:46-51) only gates the import path:For a
mlflow.genai.scorers.trulensclass, the forwardedmodel/kwargsreach an outbound LLM client through a routing choice the attacker controls (mlflow/genai/scorers/trulens/models.py:97-121):Only the
litellmfallback branch reaches raw**kwargs; amodelURI outside MLflow's native provider registry (e.g.openai:/gpt-4o) trivially routes there, andLiteLLM.__init__forwardsapi_base/api_keystraight intolitellm.completion().POST /ajax-api/3.0/mlflow/scorer/invoke→_invoke_scorer_handler(mlflow/server/handlers.py:7026-7098), guarded only by@catch_mlflow_exception/@_disable_if_artifacts_only— no auth decorator.Scorer.model_validate_json(serialized_scorer)(the vulnerable deserialization) runs synchronously, before any tracking-store lookup orsubmit_jobcall.trulens+trulens-providers-litellminstalled (a documented but optional, not-bundled GenAI-eval setup).PoC
The
POST .../chat/completionsrequest landing onATTACKER, carrying the attacker's ownAuthorizationheader, is the server-originated outbound request — no request was ever sent by the client to that host directly.Impact
An unauthenticated attacker forces the tracking server to issue an outbound HTTP request to an arbitrary attacker-chosen host — reachable from the server's network position, including internal services or cloud-metadata endpoints — with fully attacker-controlled request headers and body. This is SSRF with header/credential injection into the server-originated request; it requires no prior authentication and no operator misconfiguration beyond having the optional
trulens+trulens-providers-litellmpackages installed alongside MLflow.Severity
CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:NReferences
This data is provided by OSV and the GitHub Advisory Database (CC-BY 4.0).
MLflow CrewAI autologging records LLM provider API keys in logged trace spans
CVE-2026-116691 / GHSA-j7qq-vr8c-xvw3
More information
Details
Summary
mlflow.crewai.autolog()leaks CrewAI LLM provider API keys into MLflow trace telemetry.When a CrewAI
Agentis instrumented with MLflow autologging, MLflow serializes the agent's__dict__and converts its fields to strings. Thellmfield contains the CrewAI LLM object, whose string representation includes itsapi_keyin plaintext.As a result, an application that calls
mlflow.crewai.autolog()can unintentionally send its LLM provider API key to the MLflow trace backend or configured OTLP destination.The application does not need to explicitly log the credential or place it in a span attribute. The credential is extracted automatically by MLflow's instrumentation.
This was verified on MLflow 3.16.0 with CrewAI 1.15.18 using MLflow's own OTLP exporter. The fake credential appeared in 5 attribute locations across 2 spans.
Details
The primary issue is in:
mlflow/crewai/autolog.py::_get_agent_attributes()The function walks the CrewAI
Agent.__dict__and converts agent fields to strings.toolsandembedderreceive special handling, but credential-bearing objects such as the following are not sanitized:llmfunction_calling_llmagent_executorcrewFor example,
agent.llmis a CrewAILLMobject. In CrewAI 1.15.18, the underlying LLM model containsapi_keyas a normal field. Its string representation therefore contains the credential:MLflow subsequently stores this string as a span attribute.
There is an important indication that this is an unintended disclosure: MLflow's dedicated LLM attribute extraction already explicitly excludes
api_keyin_get_llm_attributes(). The LLM span therefore does not expose the credential, while the broader Agent__dict__serialization bypasses that protection.There is also a second, independent serialization path in:
mlflow/crewai/autolog.py::_construct_full_inputs()This code uses:
on call arguments. During CrewAI execution, the
Agentobject is passed as an argument, causing its__dict__to be expanded and the credential-bearing LLM object to be included inmlflow.spanInputs.Therefore, there are two independent leakage paths:
_get_agent_attributes()_construct_full_inputs()Fixing only the Agent attribute serialization would leave the
mlflow.spanInputspath vulnerable.A safer approach would be to avoid recursively serializing arbitrary object
__dict__values. Agent telemetry should use an explicit allowlist of non-sensitive scalar fields, and credential-bearing objects such asllm,function_calling_llm,agent_executor, andcrewshould either be reduced to safe metadata or omitted.Similarly,
_construct_full_inputs()should not automatically expand arbitrary objects through__dict__, or the resulting values should pass through the same sanitization mechanism before being attached to a span.PoC
Environment:
1. Direct serialization proof
The following demonstrates the vulnerability without starting a Crew, making a network request, or requiring an MLflow server:
The output contains the API key in plaintext, for example:
No provider request is required.
2. End-to-end trace disclosure
The issue can also be reproduced through MLflow's actual OTLP exporter.
Configure a local OTLP HTTP receiver and set:
Then run:
A local OTLP receiver can decode the
ExportTraceServiceRequestsent by MLflow.The provider request is expected to fail because the credential is intentionally fake. This does not affect the vulnerability: the sensitive value is added to the MLflow span when the instrumentation starts, before the provider call succeeds or fails.
In the tested environment, the fake API key appeared in the exported telemetry in 5 locations across 2 spans, including:
The captured value is produced by MLflow's actual telemetry/export path rather than being manually inserted into a span.
All credentials used in the reproduction are locally generated fake values. No real API credential is required.
Impact
Type: Sensitive Information Disclosure / Credential Disclosure
CWE: CWE-532 — Insertion of Sensitive Information into Log File
Any application using
mlflow.crewai.autolog()with a CrewAILLMconfigured with an API key can unintentionally disclose that credential through MLflow traces.The credential is written to trace telemetry even though the application does not explicitly log it.
Anyone with access to the MLflow trace storage or configured OTLP destination may therefore obtain the provider credential. Depending on the deployment, this can include MLflow users, observability operators, or third-party telemetry providers.
The leaked credential can then potentially be used against the corresponding LLM provider account, including for unauthorized API usage and associated costs.
The issue is particularly significant because the credential crosses a trust boundary: a provider API key intended for use by the application is automatically copied into observability telemetry where a broader set of users or systems may have access.
The vulnerability does not require the developer to intentionally log the secret. The only relevant prerequisite is enabling the CrewAI autologging integration.
Patches
Fixed in MLflow 3.17.0 by #25845.
mlflow.crewai.autolog()now reduces CrewAI LLM objects to their model name and drops credential-like keys before they are written to span attributes ormlflow.spanInputs.Remediation
mlflow>=3.17.0.LLMobject held (for example one passed asLLM(api_key=...)) whilemlflow.crewai.autolog()was enabled on MLflow 2.19.0 through 3.16.x.mlflow.spanInputs.Workarounds
If you cannot upgrade yet, disable CrewAI autologging with
mlflow.crewai.autolog(disable=True).Severity
CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:NReferences
This data is provided by OSV and the GitHub Advisory Database (CC-BY 4.0).
MLflow vulnerable to arbitrary file read on unauthenticated servers via prompt model version source
CVE-2026-116690 / GHSA-m65p-9j7q-6w33
More information
Details
Summary
On MLflow tracking servers running without authentication, which is the default for mlflow server, a remote attacker can read arbitrary files from the server’s filesystem by creating a model version tagged as a prompt whose source is a schemeless local path.
The fix for CVE-2026-2614 in 3.10.0 added a prompt-specific branch to the
CreateModelVersionhandler that rejectsfile://URIs and absolute paths but runs path validation only when the source carries a URL scheme. A schemeless value such as../../etc, a plain relative name,., a percent-encoded path, or a Windows drive-letter path is therefore stored unchanged as the model version’s artifact location, and themodel-versions/get-artifact endpointlater serves files from it with the privileges of the server process. The tag check also keyed on the presence of themlflow.prompt.is_prompttag rather than its value, so a false value still selected the weaker branch.Servers using
--app-name basic-authare also affected: any authenticated user can create a registered model, so a low-privilege account can use this to read the authentication database, including the admin password hash.Affected: MLflow 3.10.0 through 3.16.1. Fixed in 3.17.0 by mlflow/mlflow#25870, which accepts only MLflow’s own placeholder sources for prompt versions and requires the prompt tag to be
true.Details
When creating a model version with the
mlflow.prompt.is_prompttag set to true, the request is handled by the prompt branch in_create_model_version().mlflow/server/handlers.py:2674-2690:The prompt branch does not call
_validate_source_run(),_validate_source_model(), or_validate_non_local_source_contains_relative_paths()for schemeless relative paths.The artifact-read sink later trusts the stored source:
mlflow/server/handlers.py:2765-2782:_send_artifact()callsartifact_repository.download_artifacts(path)and serves the resulting local file withsend_file().PoC
Prerequisite: A reachable MLflow tracking server with model registry endpoints enabled.
First, create a registered model:
Next, create a prompt-tagged model version with the malicious relative source
../../../tmp:Finally, retrieve the arbitrary file via the artifact download endpoint by supplying the target filename in the
pathparameter:This returns the contents of
/tmp/sentinel-mlflow.txt. The same primitive can read sensitive files like/etc/passwdor tracking databases (e.g.mlflow.db).Impact
Direct impact is arbitrary local file read as the MLflow server process. Confirmed read targets include:
/etc/passwdmlflow-auth.dborbasic_auth.db)/home/user/.bash_historyThreat model: An attacker who can reach the MLflow tracking server HTTP port can read arbitrary files readable by the MLflow server process. On unauthenticated MLflow deployments, this is directly exploitable without credentials. On auth-enabled deployments, any authenticated user (even low-privileged or "read-only" accounts, depending on default RBAC configurations allowing model creation) can use the primitive. Leaked credential stores may enable offline password attacks or broader administrative compromise.
Severity
CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:NReferences
This data is provided by OSV and the GitHub Advisory Database (CC-BY 4.0).
MLflow FileStore trace IDs allow path traversal and arbitrary file writes
BIT-mlflow-2026-53995 / CVE-2026-53995 / GHSA-wqmh-qccw-pqwf
More information
Details
MLflow 3.17.1 and earlier contains a path traversal vulnerability in the FileStore tracking backend that allows unauthenticated attackers to control filesystem paths by submitting a crafted trace ID containing traversal sequences to the POST /api/3.0/mlflow/traces endpoint. Attackers can exploit FileStore.start_trace() to write trace_info.yaml, request_metadata, and tag files with attacker-controlled metadata outside the intended experiment trace directory, subject to the server process's filesystem permissions. FileStore is the default tracking backend before MLflow 3.7.0 and opt-in thereafter (the default changed to a SQLite database), so deployments in the earlier part of the affected range are vulnerable without special configuration while later ones are affected only when FileStore is explicitly selected.
Severity
CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:H/VA:L/SC:N/SI:N/SA:NReferences
This data is provided by OSV and the GitHub Advisory Database (CC-BY 4.0).
Release Notes
mlflow/mlflow (mlflow)
v3.17.2Compare Source
MLflow 3.17.2 is a patch release that contains a minor feature enhancement and bug fixes.
Features:
mlflow.genai.evaluateruns (#26543, @joshuawong-db)Bug fixes:
MlflowSparkStudyexecutors (#26714, @iyashk)MlflowStoragemarking Optuna trials finished before their values are saved (#26713, @iyashk)v3.17.1Compare Source
MLflow 3.17.1 is a patch release that contains some minor feature enhancements, bug fixes, and documentation updates.
Features:
make_judge()(#26578, @B-Step62)mlflow.genai.databricks.review_queuesfor managing review queues on Databricks (#26414, @harupy)Bug fixes:
Small bug fixes and documentation updates:
#26680, @joshuawong-db; #26584, @copilot-swe-agent; #26705, @kriscon-db
v3.17.0Compare Source
MLflow 3.17.0 includes several major features and improvements.
Major New Features
Database upgrade: SQL-backed servers require a coordinated schema upgrade with all writers stopped and a restorable backup, even when daily rollups remain disabled. Run
mlflow db upgrade <database-url>before restarting all replicas on the new MLflow version. Mixed-version rolling upgrades are not supported. Follow the database upgrade instructions.Other Assorted Features & Improvements:
base_urlfor direct TypeSafe judges (#26445, @B-Step62)typesafe_api_basefor TypeSafe Gateway endpoints (#26442, @B-Step62)name IN (...)filters for evaluation-dataset search (#26374, @HumairAK)@mlflow/typesafetracing integration for TypeSafe AI (#26342, @B-Step62)TraceImageto custom trace views (#26297, @chris-chieu)setup.sh(#26341, @B-Step62)uc_model_serviceUC model-version dependency support (#26140, @TomeHirata)DatabricksUCModelServiceresource type (#26137, @TomeHirata)order_byinsearch_prompts()across the client, fluent, and GenAI APIs (#25967, @avanish-garg)AmazonBedrockProvider(#25863, @rhamzatov)createPresignedUploadUrlendpoint (#24765, @henishborad)mlflow.restore_experiment()andmlflow.restore_run()to the fluent API (#25852, @avanish-garg)charts: Expose Helm liveness/readiness/startup probes as configuration variables (#25560, @creater29)MLFLOW_ENABLE_AI_GATEWAY(#25236, @DaoDaoNoCode)@mlflow.traceandmlflow.start_span(#25548, @Rohitkanithi)Bug fixes:
EvaluationDatasethashing for list-valued targets and predictions (#26433, @LE0-Lin)versionfilters rewriting quoted values (#26423, @Amazinghorseli)responsekey fortool_call_responseparts in GenAI semconv export (#26360, @Zhuoxi2000)MLFLOW_ALLOW_PICKLE_DESERIALIZATIONguard tomlflow.paddleflavor (#26428, @aaronteo-db)copy.deepcopyandpickleon Span, LazySpan, and Trace (#26307, @Rohitkanithi)tool_call_idinGeminiAdapter(#26253, @codechrl)execute_toolandinvoke_agentspans (#26394, @chrikrah)finish_reasontool_callsfor Anthropic and Gemini tool calls (#26390, @r14dd)#hash route (#26378, @aaronteo-db)_strip_markdown_code_blocksfor single-line and glued closing code fences (#26332, @lets-order-some-fries)server_namefilters for MCP access endpoints (#26327, @Amazinghorseli)invoke_model_with_response_streamchunks and GPT-6redactedContentreasoning (#26346, @kimnamu)@latestalias not resolving to the highest model version (#26291, @CTWalk)MlflowStorage.set_trial_state_valuesWAITING trial transitions (#25945, @CoralGarden52)tool_choicein Gemini and Vertex AI gateway requests (#26300, [@Vaishnavi220506](https://redirect.github.cConfiguration
📅 Schedule: (UTC)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
To execute skipped test pipelines write comment
/ok-to-test.Documentation
Find out how to configure dependency updates in MintMaker documentation or see all available configuration options in Renovate documentation.