Repository navigation
Bug-hunt fixes (culture-safe numbers, cancellation, model mapping) and four measured Core speedups - #187
Merged
Conversation
XlsbSheetWriter.WriteRow had no try/finally around the cell loop, so a rejected value (NaN, oversized text) left the row active and every later StartRow, WriteRow or End threw, losing the whole package. The row is now finished in a finally, keeping the cells written before the rejection, which matches what disposing an XlsbRowWriter after a rejected Write does. XLSX and XLS have no bulk WriteRow, so only XLSB was affected.
TypeMap bound every public instance property, so a POCO with an indexer made Read<T> throw NotSupportedException for 'Item' even though no column could ever match it. The writer already filtered indexers.
…ulture XLSX numbers with more than 15 significant digits keep only their text, and Cell.TryParse parsed that text with the caller's culture: under pt-BR 35.840000000000003 became 35840000000000004, under fr-FR it failed. Number and Date cells now always parse invariantly (also in the ISpanParsable path); string and CSV cells still honor the culture.
Parse(IExcelSheet) always used the Excel type map, so a CSV sheet reached through IExcelWorkbook read text dates as default, accepted integer TimeOnly cells as midnight, and turned blank lines into all-default models (or a required-value failure). It now picks the CSV map for a CsvSheet and the generic projector skips CSV blank records like the CSV-specialized one.
GetProperties returns both members when a derived class redeclares a property with a different type, so typed reads filled both and typed writes emitted two columns with the same header. Reader and writer now share one property walk that keeps only the most-derived member per name, as Core's generator does.
…d on Fail kept whichever callback exception happened first in wall-clock time, but the native pipeline reports the earliest failing chunk in sheet order and may discard and re-read later chunks. With failures in several chunks the rethrown exception varied run to run. Each failure now returns its own positive code, which native passes back verbatim, so the exception is looked up by the status the run returned.
…LSB sheets OpenSheetAsync took a shortcut to the sync OpenSheet when the zip was memory-backed (ReadOnlyMemory overloads, or a writable MemoryStream exposing its buffer), which built the enumerator with a default token, so a cancelled token never stopped the read.
…ted write XlsxRowWriter consumed the pending explicit cell reference (set by Skip) and the pending Style() as call arguments, before CellFormatter validated the value. A rejected NaN, infinite or pre-epoch date therefore dropped both, so the next cell lost its r= attribute and landed in the wrong column. Row state is now committed only after the cell is written. XLSB and XLS had the same pending-style loss: EffectiveStyle ran before the non-finite and epoch checks. Those checks now run first.
ParallelCsvEnumerable only saw the token handed to GetAsyncEnumerator, so the ct given to SheetParallel.ParseAsync was dropped on the partitioned path and a cancelled parse read every row. It now links both tokens, as the XLSX pipeline does.
GetText formatted with "R", which on .NET Framework x64 can drop the last bit (0.84551240822557006 came back one ulp off from XLSB and XLS). The "R" text is kept when it parses back to the value, otherwise G17 is used.
…again When the data after the header fit in one chunk, the factory opened a private CsvReader and handed it to the CsvEnumerable, whose first enumeration disposed it; a second await foreach threw ObjectDisposedException. The fallback now reads the caller's reader sheet, which opens a fresh cursor per enumeration, so there is no owned reader left to dispose.
…sets BuildGlobals summed sheet lengths in int, so a workbook stream over 2 GiB wrapped the BOUNDSHEET lbPlyPos, INDEX and DBCELL offsets negative and the container came out corrupt. BIFF8 stores those as 32-bit offsets and a CFB v3 stream must stay under 2 GiB, so the sum is now checked and throws ExcelLimitExceededException (WorkbookStreamBytes). The OLE writer's sector rounding of the stream size is also done in long, since a size within 511 bytes of int.MaxValue overflowed it.
…ed with another GetAsyncEnumerator replaced the token captured at creation (the SheetParallel.ParseAsync ct) with any cancelable one passed to it, so .WithCancellation(other) dropped the original. Both are now linked when both can cancel, and the async enumerator disposes the linked source.
DetectFileFormat only caught InvalidDataException, so a password-protected ZIP (NotSupportedException) or one with more entries than MaxZipEntries (ExcelLimitExceededException) escaped instead of returning the documented Unknown. Opening such a file still throws NotSupportedException, which the error guide documents for features the reader does not support.
CreateContinuation cut the base name by UTF-16 code unit to fit the 31-char limit with its " (n)" suffix, so a name with an astral character at the cut point produced a continuation sheet ending in a lone high surrogate, which the AddSheet name validation would have rejected. The cut now drops the whole pair.
…e the header row CsvHeaderBinder threw InvalidOperationException when the source had fewer rows than HeaderRow, while the sequential parser yields an empty sequence. The binder now reports no data start, so the factory returns the empty result.
…prefix A tag inside a leading comment or processing instruction was taken for the root element, so a prefixed sheet, workbook, styles or shared-strings part read as unprefixed and yielded nothing.
…egative offset The time-only anchor was fixed at 1899-12-31, which is before the 1904 epoch and so had no serial. A trailing '-hh:mm' offset also made the value look like a date and picked up today's date. Anchor at the workbook epoch and detect time-only by the absence of a date part before the first ':'.
HasEntryWithPrefix compared bytes exactly while TryGetEntry ignores ASCII case, so a package storing xl/Tables/table1.xml made table discovery bail out early and Tables came back silently empty.
GetProperties includes indexers, and binding one threw ArgumentException from Expression.Property inside the cached Lazy, so any model with a public indexer could never be parsed. Indexers are now filtered out, as the writer and the source generator already do.
…adable typed cells GetValue falls back to the cell's text when a Number, Date or Boolean cell cannot be read as that type (e.g. a date-styled serial past DateTime's range), but GetFieldType and GetSchemaTable still reported double/DateTime/bool. The type now follows the same check, so it always matches GetValue's value.
…ne with new Reflection returns both the hiding and the hidden property, so FromAttributes threw "Ambiguous header binding" and the record writer emitted the column twice. Both now walk the type hierarchy most-derived first and keep one property per name, as the source generator does.
xl_parse_typed, xl_parse_arrow, xl_write_typed_to_memory, xl_encrypt_package_to_memory, xl_open_write_handle, xl_add_style and xl_rows_next_cells returned InvalidArgument without touching a valid out pointer, so a caller freeing its buffer afterwards freed stack garbage.
A NULL name pointer reached Encoding.UTF8.GetString and surfaced as XL_ERROR 'Value cannot be null' instead of XL_INVALID_ARGUMENT.
ctypes.string_at takes a C int, so a typed column or buffer past 2 GiB was silently truncated or raised SystemError. Route every computed-size copy through _native.copy_bytes, which sizes a c_char array from the address.
…error C fixed over an empty array yields NULL, which an empty in-memory CSV should open from. Rejected open requests now also report why.
…writing only its first chunk combine_chunks leaves binary columns split past 2 GiB, and taking batches[0] silently dropped the rest. The int32-offset ABI cannot take that column anyway, so raise a ValueError.
…eclaration MappableProperties checked [ExcelIgnore] before claiming the name, so a derived property marked [ExcelIgnore] let the base declaration of the same name bind, and for an override that calls the very setter the attribute was meant to skip.
…ngth out_of_range read every column with a fits check as row_count i64 values, so a hand-written binding with fits on a bool column read 8 bytes per row from a 1-byte-per-row buffer from safe code, and an empty names slice panicked on names[0]. Check only I64 columns, stop at the column length, and tolerate no names.
ExcelColumn, ExcelIgnore, ExcelRequired, ExcelConverter and ExcelStyle are Inherited, and reflection resolves them up the override chain; the generated map only read the overriding symbol. An ignore on the most-derived declaration now also suppresses the name instead of binding the base declaration.
…rence sample Inference samples 100 rows, so a column without a blank in the sample was non-nullable and a later blank threw. Inferred columns are now all nullable; an explicit schema keeps its own nullability.
…s tail lazily convert: an error after progress text landed on the same line; end the progress line first and drop the blank line printed on failure. preview: --rows allocated a rows/2 tail array before reading anything, which threw OutOfMemoryException for huge values; it now grows with the rows actually read.
store_panic kept the first payload forever and finish resumed it whenever the status was i32::MAX, so a panic from a discarded CSV chunk replaced a later panic or a legitimate Err(i32::MAX). Each panic now gets a status below i32::MAX, a caller code inside that window is recorded the same way, and finish maps the status the run ended on back to that exact failure. A discarded panic still cannot fail a run that returns XL_OK.
A non-seekable input was copied through a doubling MemoryStream (256 B start), allocating ~3x the input in zeroed LOH arrays plus a copy per doubling. Now pooled 64K..1M chunks are assembled into a single uninitialized exact-size array. Reading the 6.7 MB XLSX through a non-seekable stream was 8-9% slower than seekable.
… the array limit The check ran only after the whole stream was read, so an oversized input was held in pooled chunks before failing; MemoryStream used to fail as it crossed the limit.
A blank cell in a nullable typed column ran the full failing parse (FastDate, double.TryParse, DateTime.TryParseExact x4) before recording a null: 56 ns per blank date versus 11.5 ns for a valid one, 45 ns for a blank double. TypedTableBuilder now appends the null directly for a missing or empty cell in a nullable column, writing the same default and validity bit as before. Cell.TryGetDateTime and Cell.TryParse<T> return false at once for an empty text cell of a type whose parser never accepts empty. Model parsing already skips CellType.Empty cells before calling the parser.
…er row The merge thread of a partitioned SheetParallel.ParseAsync<T> handed out every row through an async iterator, paying a full state-machine resume per row (21-22 ns/row in isolation). A hand-written enumerator now returns rows of the current chunk synchronously and only goes async at chunk boundaries (5.5 ns/row). Chunk taking, failure row numbers, cancellation per chunk, list pooling and teardown are unchanged; DisposeAsync is idempotent and also runs on completion and on a fault.
…r entry Every DeflateEncoder (one per ZIP entry and one per parallel segment) rented and cleared a 64K-slot hash table, about 15 us of a 49 us 10-row XLSX write. Each thread now keeps one table, and each encoder starts its positions past the previous encoder's, so stale slots fail the distance check exactly as zero slots did. Output is byte-identical. The table is still cleared once its base passes 2^30 or after a stream that slid its window. Measured: writing a 10-row XLSX is 31-45% faster.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #187 +/- ##
==========================================
+ Coverage 92.32% 92.42% +0.10%
==========================================
Files 302 304 +2
Lines 25602 25856 +254
Branches 4219 4273 +54
==========================================
+ Hits 23636 23898 +262
+ Misses 1345 1334 -11
- Partials 621 624 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
GabrielMarquezMatte
added a commit
that referenced
this pull request
Nov 18, 2026
Bug-hunt fixes (culture-safe numbers, cancellation, model mapping) and four measured Core speedups
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
develop is 38 commits ahead of master: 33 fixes from a third bug hunt and 5 performance changes from a Core audit. Every fix has a regression test that failed before it. Every perf change was measured with an interleaved A/B against the pre-perf develop.
Reader
<v>text of 15 digits or fewer. Longer text, such as0.30000000000000004or any computed double Excel writes, was parsed withExcelParserOptions.Culture. Under pt-BR,35.840000000000003became35840000000000004, and under fr-FR it failed to parse.Cell.TryParseandTryParseSpanParsablenow use the invariant culture forNumber/Datecells, and string/CSV cells keep the caller's culture.OpenSheetAsyncdroppedctwhen the ZIP index was backed by memory, which coversFromXlsx(ReadOnlyMemory)and any writableMemoryStream.t="d"cells now read in 1904 workbooks and with a negative offset (10:30:00-03:00). They used to lose their serial, or pick up today's date.<Tag>inside a leading comment used to make a prefixed (x:) sheet read zero rows.HasEntryWithPrefixis ASCII case-insensitive likeTryGetEntry. Before,xl/Tables/madeworkbook.Tablessilently empty.DetectFileFormatreturnsUnknownfor encrypted ZIPs and ZIPs past the entry limit instead of throwing.SheetDataReader.GetFieldType/GetSchemaTablereportstringwhenGetValuewill return a string (e.g. an out-of-range date serial).Parser and generator
Parse(IExcelSheet)on aCsvSheetuses the CSV map. Text dates read correctly, integers are no longer read asTimeOnly, and blank lines no longer produce phantom models.ParseAsynctoken;WithCancellationlinks tokens instead of replacing them;HeaderRowyields nothing, like the sequential parser.FromAttributes, record writer):new;[ExcelIgnore]on the most-derived declaration suppresses the name. It no longer binds through the base.override(ExcelColumn,ExcelIgnore,ExcelRequired,ExcelConverter,ExcelStyle), matching reflection.Writer
WriteRowfinishes the row when a cell is rejected. Before, the sheet and the whole package could never be ended.SkipthenWrite(NaN)). The next cell used to land in column A. XLSB and XLS also kept a pending style on rejected numbers and dates.ExcelLimitExceededExceptioninstead of wrapping its offsets;Arrow, native ABI, CLI
ToArrowRecordBatchwithout a schema makes inferred columns nullable. A blank past the 100-row sample used to throw.xl_bufferno longer frees stack garbage;XL_INVALID_ARGUMENT;xl_open_memoryaccepts(NULL, 0), and every rejection sets a last error.converterrors start on their own line;preview --rows <huge>no longer allocates the whole tail buffer up front.Bindings
newproperties;Aggregaterethrows the failure the native run ended on, not the first one in wall-clock time;GetTextfalls back toG17when .NET Framework's"R"doesn't round-trip.out_of_rangecheck only readsI64columns, bounded by the column length. A hand-writtenExcelMappercould read past a native buffer from safe code;ctypes.string_at, whose size is a Cint(silent truncation past 2 GiB);ValueErrorinstead of writing only its first chunk.Performance
SheetParallel.ParseAsyncstruct model, 300k-row CSV, dop 4TryAppendNull, empty guards inCell)ToArrowRecordBatch, 150k-row XLSX, 30% blanksThe controls (typed XLSX parse, 100k-row XLSX write) are neutral.
Evidence
Repro tests, before → after:
dotnet test tests/ExcelReader.Testson the merged develop: 4632 passed, 0 failedcargo test -p excelreadergreen, Python 272 passed (1 xfail)Merge Danger
Door: two-way. There are no public API, C header or
XL_ABI_VERSIONchanges, and every commit reverts on its own.Blast Radius: behavior
ExcelParserOptions.Culture/Cell.TryParse(provider)no longer apply to workbookNumber/Datecells. Callers who relied on a culture to parse workbook numbers get invariant results now (the correct ones).[ExcelColumn]) or skip columns (inherited[ExcelIgnore]). This brings them in line withFromAttributes.ToArrowRecordBatch()without a schema reports every inferred field as nullable.SheetDataReader: a column whose first row holds an unreadable value is typedstring.PrefetchStream's documented chunk copy is still open.