Multimodal content blocks

[Zhuoxi2000]

Linked issue: #1059 (Phase 1 — the tracking issue stays open for the provider phase)

Purpose of change

Implements the framework part of the multimodal design agreed in #1031.

ChatMessage.content: String is replaced by blocks: List<ContentBlock> in both Java and Python. The initial block types are TextBlock, ImageBlock, AudioBlock, VideoBlock, and DocumentBlock. Media blocks share a MIME-typed shape with mime_type, exactly one of base64 data or external url, and optional metadata such as name, size_bytes, and sha256.

Existing text constructors/factories keep their signatures and create a single TextBlock. getText() / .text returns the ordered text projection.

Java and Python use the same type-discriminated JSON format, covered by the regenerated cross-language snapshots. The pemja bridge is updated to carry blocks across runtimes as well.

This PR also updates prompt handling so image-only messages are preserved, and Prompt.formatMessages applies substitutions only to text blocks while passing media blocks through unchanged.

Provider-specific multimodal conversion is intentionally left to the next #1059 phase. Existing providers continue using the text projection in this PR.

Behavior changes:

• CEL expressions using response.content need to migrate to response.blocks.
• Python ChatMessage now rejects unknown fields, so the removed content= argument fails explicitly instead of silently creating an empty message.
• toString() / __str__, equality, and hashing now use the block representation.
• data and url are mutually exclusive for media blocks.

Tests

• Added Java and Python ChatMessage serialization tests covering the wire format, mixed block ordering, media fields, round trips, source validation, and rejection of the removed content argument.
• Regenerated the Java/Python cross-language snapshots using the existing snapshot flows; cross-deserialization passes in both directions.
• Added durable-state serde coverage for a mixed text + image ChatMessage.
• Java reactor tests and Python tests pass locally, aside from environment-specific dependency failures that are covered by CI.

API

Yes. This is a Beta breaking change discussed in #1031.

ChatMessage.content is replaced by blocks in both Java and Python. Java removes getContent()/setContent; Python removes .content. Text-only access remains available through getText() / .text and setText() / set_text().

New public types in both languages are ContentBlock, TextBlock, MediaBlock, ImageBlock, AudioBlock, VideoBlock, and DocumentBlock.

ChatMessage also adds getBlocksAsMaps() / setBlocksFromMaps() for the pemja bridge, where blocks cross as plain maps.

CEL expressions referencing response.content must migrate to response.blocks.

Documentation

• doc-needed
• doc-not-needed
• doc-included

Was this patch authored or co-authored using generative AI tooling?

• Yes
• No

Generated by: Claude Code Fable 5