[python] Render min/max_partition_stats in the $manifests system table

[wombatu-kun]

Purpose

This closes a TODO in pypaimon/table/system/manifests_table.py:

# TODO: render min/max_partition_stats by casting partition
# rows to their string form. pypaimon
# has SimpleStats but no shared partition-row-to-string
# helper yet; emit NULL to preserve the column shape.
min_partition_stats.append(None)
max_partition_stats.append(None)

So $manifests returns NULL for min_partition_stats and max_partition_stats in pypaimon while Java renders them, and inspecting which manifests cover which partitions gives nothing on the Python side. The data is already parsed: ManifestListManager builds SimpleStats from the manifest list, only the rendering was missing.

This adds cast_row_to_string, a port of RowToStringCastRule and the per type *ToStringCastRule rules of paimon-common, with fields joined by ", " inside braces, the literal null for a null field, and {} for an unpartitioned table. The rendered types are BOOLEAN, TINYINT, SMALLINT, INT, BIGINT, DECIMAL, CHAR, VARCHAR, DATE, TIME and TIMESTAMP. Those needing more than str() are boolean (lower case), decimal (BigDecimal.toPlainString, never scientific, so a DECIMAL(20, 9) zero is 0.000000000 and not 0E-9), date, time and timestamp. Timestamps follow DateTimeUtils.formatTimestamp and times follow formatTimestampMillis, which generates the fraction digit by digit and stops only once the remainder is exactly zero, so neither datetime.isoformat() nor stripping trailing zeros reproduces them.

Three types are deliberately not rendered, because their Java output cannot be reproduced. A row holding one of them is not rendered at all, so the two columns keep the NULL they return today:

• FLOAT and DOUBLE go through Float/Double.toString, which up to JDK 18 is the legacy FloatingDecimal and does not produce the shortest round tripping decimal, while JDK 19+ does. Measured against JDK 17 on 200k random values per type, the shortest form differs on 10.86% of float32 and 0.28% of float64 values (2.68873286E11 against 2.6887329E11), and a JDK 21 JVM prints the shortest form instead, so there is no single Java rendering to target. Matching pre JDK 19 exactly would mean porting FloatingDecimal itself.
• TIMESTAMP WITH LOCAL TIME ZONE is formatted in TimeZone.getDefault(), so the same manifest reads 2024-01-02 03:04:05 on a UTC JVM and 2024-01-02 06:04:05 on a Europe/Moscow one.
• BINARY, VARBINARY and BYTES are decoded as UTF-8, and on malformed input the JDK decoder and Python disagree on how many replacement characters to emit: ED A0 80 is one in Java and three in Python.

One point for a maintainer opinion, deliberately not changed here: pypaimon renders the partition column of $files and $buckets as pt=v/pt2=v2, while Java renders those two with the brace form and keeps pt=v only for $partitions. Aligning them would change values these tables already return, so it is out of scope for this PR.

Tests

New pypaimon/tests/row_to_string_test.py covers the rendering per type, a null field, an empty row, the timestamp fraction at precision 0/3/6, the TIME digit generation (.10 and not .1 for 101 ms at precision 2), the decimal plain form, and that a row holding a FLOAT, DOUBLE, TIMESTAMP_LTZ or BINARY field renders as None while cast_value_to_string rejects those types outright.

pypaimon/tests/system/manifests_table_test.py replaces the assertions that pinned the NULL placeholder: an unpartitioned table yields {}, a table partitioned by INT and STRING yields {1, 2024-01-01} and {2, 2024-01-02}, a DECIMAL(20, 9) partition yields {0.000000000}, and DOUBLE and BYTES partitions keep both columns NULL.

Everything still rendered was verified against RowToStringCastRule itself, driving the real CastExecutors from a built paimon-common on JDK 8 and comparing whole rows rather than single values: every DATE a Python date can hold, the full TIME precision and millisecond space, all DECIMAL precision and scale pairs, TIMESTAMP precision 0 to 9 over epochs from year 1 to 9999, integer boundaries, strings containing braces and the , separator, and randomly generated multi field rows with nulls. No differences remain, and the same harness does flag the ones the previous commit of this PR had.

[JingsongLi]

• Scientific notation does not match Java’s Float/Double.toString.
• Local regression: Java returns 1.0E7 / 1.0E-5, while Python currently outputs 1e+07 / 1e-05.

[wombatu-kun]

Done 9df763e - FLOAT/DOUBLE now render via Java's Float/Double.toString format (uppercase E, no + sign, no zero-padded exponent; e.g. 1.0E7 / 1.0E-5).

[JingsongLi]

The new floating-point format implementation does not fully match Java 8’s Float/Double.toString. When comparing the bit patterns of 300,000 float values and 300,000 double values, there are also significant differences in ordinary finite values; for example, Java outputs 2.68873286E11, while Python outputs 2.6887329E11. Furthermore, a valid Float.MAX_VALUE triggers an OverflowError.

[wombatu-kun]

The float formatter is removed rather than fixed. Both points reproduce, and matching pre-JDK-19 output would mean porting the legacy FloatingDecimal; JDK 19+ changed it in the other direction, so there is no single Java rendering to target. FLOAT and DOUBLE partition columns leave min/max_partition_stats NULL again.

Auditing the rest the same way, TIMESTAMP WITH LOCAL TIME ZONE is excluded for the same reason (Java formats it in TimeZone.getDefault()), and so is BINARY (the JDK UTF-8 decoder and Python disagree on the replacement characters for malformed input). Two real bugs are fixed: TIME at precision 2 dropped a zero the truncated tail sat behind (.1 instead of .10 for 101 ms), and DECIMAL rendered 0E-9 where Decimal.toString gives 0.000000000. What is still rendered is now checked against RowToStringCastRule itself: BOOLEAN, TINYINT, SMALLINT, INT, BIGINT, DECIMAL, CHAR, VARCHAR, DATE, TIME and TIMESTAMP.