Repository navigation
Check resolved locale when performing date range formatter fallback - #8692
Draft
Manishearth wants to merge 6 commits into
Draft
Manishearth wants to merge 6 commits into
Manishearth wants to merge 6 commits into
Conversation
…ck in range selection Previously, DateRangePatternSelectionData and TimeRangePatternSelectionData used DataPayloadOr<ErasedPackedRangePatterns, ()>, where None represented a fieldset without a date or time component. In preparation for discarding inherited range patterns when their resolved locale mismatches the single pattern (unicode-org#8359), change the Other type parameter to bool: - Self::none() (Err(false)) indicates the fieldset does not include a date/time component. - Self::fallback() (Err(true)) indicates the fieldset includes a date/time component, but range formatting should fall back to the glue pattern. Use is_present() in RangeFormatterCore::format and contains_difference so fieldset presence continues to be observed directly from range_selection.
… selection constructors In preparation for checking that range patterns resolve to the same locale as their corresponding single patterns (unicode-org#8359), introduce PatternSelectionDataMetadata holding the resolved DataLocale for the date and time pattern loads. Thread this metadata out of DatePatternSelectionData::try_new_with_skeleton, TimePatternSelectionData::try_new_with_skeleton, TimePatternSelectionData::try_new_overlap_with_skeleton, and DateTimeZonePatternSelectionData::try_new_with_skeleton via try_new_internal_with_metadata on DateTimeFormatter and FixedCalendarDateTimeFormatter. No behavior change yet.
…e-org#8359) Pass PatternSelectionDataMetadata into DateTimeZoneRangePatternSelectionData::try_new_with_skeleton and compare the resolved DataLocale of DateRangePatternSelectionData and TimeRangePatternSelectionData against the single pattern's resolved locale. Because DateTimeRangeFormatter reuses the field names loaded for the single pattern and expects range patterns to match the single pattern's field order and separators, a range pattern that fell back to an ancestor locale (such as und) while the single pattern resolved at a more specific locale cannot be used safely. When the single pattern resolved at a non-root locale and the range pattern's resolved locale differs, return Self::fallback() so range formatting falls back to the glue pattern ({0} – {1}).
…vider_source Extract SemanticSkeletonsContext::new and make PatternsWithDistance pub(super) so that range pattern datagen can construct single-pattern contexts for locales and their fallback ancestors when checking inheritance consistency.
…overridden (unicode-org#8359) Runtime locale comparison alone does not catch all inconsistent inherited range patterns for two reasons: 1. cldr-json pre-flattens inherited intervalFormats from parent locales and und into every locale's ca-*.json file, and make_packed_range_data indexes them using the child locale's date_skeletons. When a locale overrides date_skeletons (e.g. de Buddhist and Hebrew using medium GyMMdd instead of und's GyMMMd), the packed header differs from und, preventing datagen from deduplicating the range payload to und. 2. GenericPackedPatterns packs all 9 (length, variant) slots into a single marker payload, so a regional locale that overrides only Short single patterns while customizing Medium/Long range patterns (e.g. en-CA or en-ZA ym0d) resolves both single and range markers at the child locale. To handle both without bloating data size: - In PatternsByGreatestDifference::match_fields, walk the locale fallback chain to find the highest ancestor from which the matched range skeleton was inherited unchanged. If the single pattern for those fields was overridden along the way (single_here != single_at_origin), clear that slot to header = 0. - In make_packed_range_data, if all slots in the resulting PackedRangePatterns are empty (header == 0), delegate to the parent locale's make_packed_range_data so datagen deduplicates the entry to its ancestor instead of emitting a tombstone payload, relying on the runtime locale check to fall back to glue.
Regenerate icu_datetime_data baked range pattern payloads and fingerprints.csv following the range pattern inheritance consistency changes in icu_provider_source (unicode-org#8359).
Manishearth
force-pushed
the
fix-8359
branch
from
November 13, 2026 06:38
f79c92e to
0d37a18
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #8359
This does two things:
This is largely agent-driven and I'm not super happy with the result, but I can't think of a way to do it better with the approach we settled on. Curious to here what sffc thinks, and feel free to take over on this since a lot of this touches gnarly datetimeformat code.
This should be reviewed commit-by-commit.
(opening as a draft since i gtg)
Changelog
icu_datetime: Don't allow range formatting to fall back across a language boundary