Repository navigation
Bug-hunt fixes: bounded OLE chains, XLSB/XLSX writer output Excel opens cleanly, CSV date mapping, native close crash - #186
Merged
Merged
Conversation
A chain walking FAT entries past the end of the file sized its buffer before any sector was checked, allocating about 128x the file and, past about 4M sectors, throwing OverflowException instead of InvalidDataException.
The token only reached the refill, so rows already buffered kept coming after cancellation and a small sheet never observed it.
Rows parsed from the buffer, batched or not, returned without checking the token, so cancellation was only seen at the next refill.
The merged map was built from the serial-date type map and reused for CSV, so attribute-fallback DateTime/DateOnly/TimeOnly properties lost the text readers FromAttributes uses: culture-formatted CSV dates stayed default.
A non-generic UnsafeAccessor cannot bind a member of a constructed generic type, so every row threw MissingMethodException. Declare the accessor in a generic holder that mirrors the open type's parameters and constraints.
The first chunk forced DetectEncodingFromByteOrderMark on, so an index-based map stripped a BOM the sequential parser keeps.
… hiding or narrowing them
One of them was silently dropped, and CSV kept the last while the other formats kept the first. Throw like the header-name ambiguity check does.
convert aborted on a date-formatted cell whose serial has no DateTime (Excel shows ####); write the number instead, as preview shows the text.
…tches Parallel CSV against sequential on stray quotes, text after a closing quote and mixed terminators; the failure row across quoted newlines at every chunk size; header-only and all-blank text columns through ToArrowRecordBatch.
…cess xl_close_write_handle disposed the handle in a finally outside any catch, and StagedFile.Dispose's temp-file delete could throw. When the commit's move and that delete both failed, the exception left the [UnmanagedCallersOnly] export and the runtime failed fast. The delete is now best effort and the dispose cannot throw, so close reports XL_ERROR for the failed commit.
xl_open_source and xl_open_stream called release when the options or format were invalid, but not when the struct named a NULL callback, a negative length or a NULL out_handle, so a caller could not tell from XL_INVALID_ARGUMENT whether it still owned user_data. Once the struct has the right struct_size, the library now always owns it; the header says so.
…ashing PublishBuffer allocated and copied the finished workbook outside a try, so an OutOfMemoryException left the export and failed the process.
… first sheet hidden
…dth and XLSB pane fixes
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #186 +/- ##
==========================================
- Coverage 92.38% 92.30% -0.08%
==========================================
Files 302 302
Lines 25526 25602 +76
Branches 4194 4219 +25
==========================================
+ Hits 23583 23633 +50
- Misses 1319 1347 +28
+ Partials 624 622 -2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
develop is 21 commits ahead of master. All of them come from a bug hunt across the reader, the parser layer, the writer and the native ABI. Each fix has a regression test that failed before the fix.
Reader
ChainByteLengthnow checks every sector against the container's sector count and sums lengths as along, before anything is allocated. Previously a crafted FAT made a 1 MB file allocate about 135 MB, and a 17 MB file madeDetectFileFormatthrowOverflowExceptioninstead of returningUnknown.MoveNext/MoveNextAsync. Rows already buffered used to keep coming after cancel, and a small sheet never threw. XLS and XLSB already did this.Parser, generator, CLI
BuildWithAttributeFallbackreads CSV with the CSV type map. Text dates such as01/15/2024came back as0001-01-01with no error.class M : Base<int>). It now emits a genericUnsafeAccessorholder. Before, every row threwMissingMethodException.DetectEncodingFromByteOrderMark = false, matching the sequential parser.PropertyAtbindings on the same column now throw at build time. Before, one binding was dropped silently, and CSV and the generic path dropped different ones.convertwrites the serial number for a date-formatted cell outside theDateTimerange, aspreviewalready did. Before, it aborted the whole conversion.Writer (each fix checked by opening the output in Excel 16)
-0.0is written as a real cell. Its sign-bit-only RK made Excel refuse to open the whole workbook. That RK comes fromMath.Round(-0.004, 2),-0.0mand similar values.BrtPaneblob was written on every sheet.widthreads as 0), and XLSB and XLS made it narrower than default.activeTab/firstSheetat the first visible sheet, as the XLS writer already did. It is written only when that sheet isn't sheet 0, so ordinary output is unchanged.Native ABI
xl_close_write_handleno longer crashes the host process when the commit move and the temp-file cleanup both fail. Cleanup inStagedFileis now best-effort, and close still returnsXL_ERROR.xl_open_source/xl_open_streamcallreleaseexactly once whenever the struct is readable, failures included. Before, it depended on which validation failed. The header now documents this.xl_write_handle_bytesreports a failed buffer copy asXL_ERRORinstead of letting the exception escape the export.Evidence
Repro tests, before → after:
dotnet build ExcelReader.slnx: 0 warnings, 0 errorsdotnet test tests/ExcelReader.Tests: 4484 passed, 0 failedcargo test --all-featuresgreen, native namespace 557 passedBehavior changes
PropertyAtcolumn indexes, CSV writer delimiter or quote bytes ≥ 0x80, and unpaired surrogates in sheet names, number formats and font names. Each is anArgumentException, except the duplicatePropertyAtcolumns, which throwInvalidOperationException.OperationCanceledExceptionon the nextMoveNextafter cancel, even when rows are buffered.BrtPane, and a workbook view is written when the first sheet is hidden. The native typed-write golden digests were updated for these.XL_ABI_VERSIONis unchanged. Raw C callers that freeduser_datathemselves after a failedxl_open_source/xl_open_streamwith a NULLout_handle, NULLread/read_ator negative length must stop doing so. The bundled C++, Rust, Python and .NET Framework bindings never did.Merge Danger
Door: two-way
Every fix is a local change with its own commit and can be reverted on its own. No persisted data or file format version is involved.
Blast Radius: writer output
The widest effect is byte-level changes to XLSX, XLSB and XLS files the writer produces. That breaks anyone who pins output hashes, and it changes how Excel displays styled columns and hidden first sheets. Readers only change on hostile CFB input and on cancellation timing.