Repository navigation
Conversation
- DataBlock: print source IP as dotted quad; %lu with an unsigned int argument was undefined behaviour on 64-bit. - python_parser: pass nIP to parse_next_data_block, whose signature gained the parameter. - CUdpDevice: initialise _nSrcIP and clear it at the start of each Read() so a stale address is never reported. - CBaseDevice::getLastSourceIP: return 0 instead of false. - Regenerate the data outputs in install/sample_output; the only differences are the new source IP lines. filter.txt is left as is: "Test filter" already fails on master (see trail learning). Co-Authored-By: Claude Opus 5.5 <[email protected]>
New output format ASTERIX_JSONX (EJSONX). It has the same structure as
-j (one record per line, no trailing comma, so lines can be parsed
independently), but each field is an object with its value and the hex
of the bytes that hold it, plus the bit mask when the field is not a
whole number of bytes (the same rule as -je):
"SAC":{"val":25,"hex":"19"}
"V":{"val":0,"hex":"02","mask":"10000000"}
Unlike -je it is valid JSON (':' instead of '='), single line, and has
no name/meaning. -j, -jh and -je are unchanged. EJSONX is appended after
EGPS so existing format numbers do not move.
Checked on the PCAP, ORADIS and GPS samples: every line parses on its
own, and removing hex/mask gives exactly the -j line. New test
"Test jsonx CAT_062_065"; valgrind clean.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
SonarCloud reported "unannotated fall-through between switch labels" after EJSONX was added below the "/* no break */" comment. Sonar does not accept the comment, and [[fallthrough]] needs C++17 (the project is C++11). Instead, all four JSON labels now share one block, and the "\n\t\t" prefix is added with an if for EJSONH/EJSONE only. No behaviour change: the -j, -jh, -je and -jx outputs of the four sample files are byte-identical before and after. Co-Authored-By: Claude Opus 5.5 <[email protected]>
SonarCloud flagged the explicit delete[] in appendJsonHexMask. The hex and mask strings are now held in std::unique_ptr<unsigned char[]>. For that to be safe, the error paths of the bit-field text getters (six-bit, hex, full-byte hex, mask, octal, ASCII) now return "???" allocated with new[] (newErrorString<T>) instead of strdup. Callers release these results with delete[], so the old malloc-based error result was undefined behaviour for every text output, not only -jx. No output change: -j, -jh, -je and -jx are byte-identical on the four sample files; valgrind clean. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Source IP in decoded output and new -jx (compact JSON with hex/mask) format
In the ORADIS branches of the raw, GPS and PCAP subformats: - The temporary AsterixData whose blocks are spliced into the packet's data was never deleted (32 bytes leaked per extra block). - The 6-byte ORADIS header was read before checking 6 bytes remained. - A byte count below 6 was not rejected: byteCount - 6 became negative (huge length to parsePacket), and a count of 0 never advanced the loop. Co-Authored-By: Claude Opus 5.5 <[email protected]>
item_name is 128 bytes but was read with %128s, which writes up to 129 (the terminating NUL). Triggered by -LF with BDS entries. Co-Authored-By: Claude Opus 5.5 <[email protected]>
The sub-format of a Repetitive item can be DataItemFormatBDS (e.g. Mode S MB data), so the C-style casts to DataItemFormatFixed were undefined behaviour (reported by UBSan). Only virtual methods of DataItemFormat are used, so the base pointer is enough. Co-Authored-By: Claude Opus 5.5 <[email protected]>
The printable test looked at *pData (the first byte of the item) instead of the character being copied. Control characters were copied into the output whenever the first byte was printable, and a NUL byte cut the string short. Now each non printable character becomes a space. A NULL result from getBits is also handled. Visible in the output only for ASCII fields holding such bytes (e.g. the 0x01 trailing byte of some CAT048 callsigns or a NUL in CAT062 RDS). Co-Authored-By: Claude Opus 5.5 <[email protected]>
The failure is only a warning (as in Init()), but the socket was closed and then used by bind() and connect(). Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The headers were read without comparing to incl_len: a short packet read past the buffer, an IP header length below 5 moved the pointer backwards, a UDP length below 8 underflowed, and a UDP length larger than the captured bytes (small snaplen) made parsePacket read past the buffer. Such packets are now rejected with an error. Co-Authored-By: Claude Opus 5.5 <[email protected]>
DataItemFormat::getLength(pData) had no limit, so with truncated or malformed data the length calculation itself read past the end of the buffer (ASan heap-buffer-overflow in Repetitive::getLength and in Fixed::isLastPart called from Variable). DataItem::parse compared the length with the remaining bytes only after it had been computed. Variable::getLength could also run without limit when the FX chain continued past the parts defined for the item. getLength now takes nMaxLength and never reads more than that many bytes. When the item does not fit it returns a value greater than nMaxLength, so the existing "needed length > available" checks in DataItem::parse and DataRecord reject it as before. - Variable: checks each part fits before reading its FX bit, and a zero-length part cannot loop forever. - Compound: primary and each secondary part are bounded by what is left; getText stops and closes the output if a part does not fit. - Repetitive / Explicit: the repetition / length byte must exist. - Fixed, BDS: unchanged (no data is read). - getText of Compound, Explicit and Repetitive, and the Wireshark and Python wrappers, pass the item length as the bound. No output change on valid data: the 81 sample outputs (10 inputs x 8 formats + -L) are byte-identical. With 960 mutated inputs, master has sanitizer errors on 522 and this branch on none. The Wireshark wrapper was compiled with -DWIRESHARK_WRAPPER. The Python wrapper was only syntax-checked against a stub Python.h, because the headers are not installed. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- DoneWithFile: sfxFormat was 10 bytes but sprintf wrote ".ci<pid>_ %y%m%d%H%M%S" into it (up to 24 bytes), a stack overflow with any pid. It is now 32 bytes; strftime uses sizeof(suffix). - NextFileName: newName is static, so strncpy without a terminator let strcat append after the previous call's name, and the extension strcat was unbounded. Built with snprintf now, and truncation is logged. - DoneWithFile / DoneAll: strncpy + strcat replaced with snprintf sized to the destination. These paths are only used by the MARKDONE, PACKETFILE and disk output modes, which the CLI does not build. They removed the -Wstringop-truncation warnings: diskdevice.cxx now builds clean with -Wall -O2. The new NextFileName gives the same names as before for normal paths (checked on 21 cases), and the 81 sample outputs are unchanged. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Fix buffer overflows, leaks and invalid casts in the decoder
The original CroatiaControlLtd/asterix has not been updated since
December 2025, and its decoder had memory-safety problems that are
fixed in this fork.
- version.h: version 2.8.11-ifsnop.1 ("<upstream>-ifsnop.<n>"), plus
the upstream version and the fork URL. asterix -V / -h now print the
fork origin under the version line. The Croatia Control copyright
notice is kept.
- README.md and README: fork notice with the reasons and the changes.
README.md also says that the PyPI package and the badges belong to
the original project.
- HISTORY: entry for 2.8.11-ifsnop.1.
- The Python module version (0.7.11) is unchanged.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Mark as fork: version 2.8.11-ifsnop.1 and fork notice in README
-jx is meant for changing a value and writing it back to ASTERIX at minimal cost. Until now it lost information and gave no position: - Unique keys: Variable items repeated FX (and some items repeated spare) in the same object, so standard JSON parsers kept only the last one (110 of 281 sample records). FX bits are now numbered by extent (FX0, FX1, ...) and other repeated names get the first free number (spare, spare1). - mask whenever the field does not fill its hex bytes. The -je rule (mask only when the bit length is not a multiple of 8) dropped the mask of 8*n-bit fields that start mid-byte (BDS res1), so the byte they share with the next field could not be told apart. - "fspec" (bytes while the last bit is 1) is split from "hexdata", which now holds only the data items: the record is fspec + hexdata. - "pos" on every field: byte offset of its hex in hexdata. A value is written back with hexdata[pos..] = (hex & ~mask) | (value & mask), with no knowledge of the item structure (Compound, Repetitive, Explicit, BDS). DataRecord stores each item's offset (m_nOffset), and DataItemBits::JsonPosition carries it while the item is written. -j, -jh and -je are unchanged (byte-identical on all samples). Checked on all samples (451 records, 18850 fields): every pos points at its hex, fspec + hexdata equals the old hexdata, values are identical to -j, and there are no duplicate keys. Round trip: each of 10408 fields inverted inside its mask, re-encoded and decoded again. 10268 change only that field, 0 are wrong, and 140 change the record structure (BDS register number, CAT001 I020 TYP UAP selector) as expected. ASan/ UBSan and 960 mutated inputs are clean. The trail learning on -jx is replaced, since its mask rule and the missing-structure limits no longer hold. Co-Authored-By: Claude Opus 5.5 <[email protected]>
- "fspec" is now printed from the FSPEC bytes stored at parse time (DataRecord::m_pFSPECData / m_nFSPECLength) instead of being cut from the hexdata string. The FSPEC is never recomputed. -jx output is byte-identical. - -j gets the same record header as -jx: "fspec" apart, and "hexdata" with only the data items after the FSPEC. In -jx, "pos" is a byte offset in this hexdata (its first byte is pos 0). Breaking change for -j, decided by the user: in the original project the hexdata of -j was the whole record, and now the whole record is fspec + hexdata. Documented in README.md and README. The expected outputs of the two -j tests were regenerated (header lines only). -jh, -je, XML and TXT are unchanged. trail check reports a conflict with the -jx learning on master (l-ce031d62), which says -j is unchanged. This branch replaces that learning (now l-5bf7ab58), but the replacement only counts once it is merged, so the conflict is expected until then. Co-Authored-By: Claude Opus 5.5 <[email protected]>
The -j record header change (fspec split from hexdata) is incompatible with 2.8.11-ifsnop.1, so the fork revision goes up. HISTORY lists the -jx write-back changes (fspec, pos, full masks, unique keys) and the -j header change. trail check still reports the conflict on cat_062_065_json.txt: the learning on master says -j does not change, and this branch replaces it (l-5bf7ab58), which only counts once PR #4 is merged. Co-Authored-By: Claude Opus 5.5 <[email protected]>
-jx: edit a value and write it back to ASTERIX (fspec, pos, full masks, unique keys); -j header with fspec; version 2.8.11-ifsnop.2
The rules did generate header dependencies but they never matched. gcc wrote "../../obj/<dir>/release/X.o: X.cpp X.h ..." and a sed then rewrote the target as "$(OBJ_DIR_SED)/X.o $(OBJ_DIR_SED)/X.d :". OBJ_DIR_SED was never defined, so the included .d files held rules for /X.o at the filesystem root and the real objects only depended on their source file. Editing a header (version.h, a class layout in UAP.h) left stale objects: -V kept the old version, and objects built against two layouts of the same class crashed at run time. The five compile rules now use -MMD -MP -MF $(OBJ_DIR)/X.d: gcc names the target after -o, so no sed is needed; -MP adds empty rules for the headers, so deleting or renaming one does not break the build. System headers are left out of the .d files. Checked: second make does nothing; touching asterix/UAP.h, main/version.h, engine/basedevice.hxx or asterix/Utils.h rebuilds exactly the objects that include them, the libraries and the executable; removing a header and its include builds; make debug gets the same .d files; -jx output and install/test unchanged. Run make clean once after updating so old .d files are replaced. Replaces the learning on version bumps, whose forced-rebuild warning this fixes (l-b9b7da47 -> l-949a9c1c). Co-Authored-By: Claude Opus 5.5 <[email protected]>
make: rebuild objects when an included header changes
format() called vsnprintf twice (once to measure, once to write) and
copied the result through a heap buffer, for every fragment of output:
about 45% of the -jx run time. It now formats into a 512-byte stack
buffer and only takes the old two-pass path for longer results.
New helpers write exactly what the matching printf conversion would:
appendHexByte/appendHexBytes ("%02X"), writeLong/appendLong ("%ld") and
appendDouble7 ("%.7lf", through std::to_chars when built as C++17,
which is specified to match printf; snprintf under C++11).
-jx of a 51 MB GPS file: 68 s -> 50 s, output identical.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Every -jx bits field went through getBits (new[]), sprintf("%02X") per
byte and three format() calls for hex, mask and pos, plus getBits again
in getUnsigned.
- getUnsigned reads the (at most 5) bytes of the field and shifts the
value out; the old getBits path stays for out-of-range requests.
- appendJsonHexMask builds ,"hex":"..."[,"mask":"..."],"pos":N in a
stack buffer and appends it once. The hex is the bytes holding the
field, the mask depends only on m_nFrom/m_nTo and is built once per
field (rebuilt if they change).
- Key, "val" and closing text are appended directly.
Same text as getHexBitStringFullByte/getHexBitStringMask and the
format() calls they replace. -jx of a 51 MB GPS file: 50 s -> 25 s,
output identical.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Each level built its text in a temporary string that the level above
copied again: DataItem -> DataRecord -> AsterixData, and the parts of
Variable and Compound items (Variable also through substr).
Now each level appends to the caller's string and, when the item prints
nothing, cuts it back to where it started. Separators and braces are
the same: Variable parts are written as {...} and the braces trimmed in
place, the record header is removed when no item prints.
The record header writes the long fspec/hexdata strings directly
instead of through one format() call, and the record hexdata is built
with a table instead of snprintf per byte.
All output formats identical on the samples (with and without -LF).
-jx of a 51 MB GPS file: 25 s -> 20 s.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
jsonUniqueKey filled a std::set<std::string> for every field of every
record (a new set per JSON object) to number repeated names (spare,
spare1...) and FX bits (FX0, FX1...). The keys depend only on the
format: for a Fixed item on its own, and for each extent of a Variable
item (the parts before an extent are always the same ones).
- DataItemFormatFixed::makeJsonPrefixes runs the same jsonUniqueKey
algorithm and stores each key as its ready-made "key":{ prefix.
- A Fixed item keeps its prefixes; a Variable item keeps them per
extent (each extent its own object when the item has one sub item,
keys accumulated across parts otherwise, as before).
- DataItemBits::getText takes the prefix instead of the bare key.
-jx of a 51 MB GPS file: 20 s -> 15 s, output identical.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Fix CodeQL alerts: format strings and publish workflow permissions
-j and -jx already printed "fspec" (the FSPEC bytes) and a "hexdata" holding only the data items after it, so the record is fspec + hexdata. -jh, -je and the default text output now use the same record header: - text: a "Fspec: ..." line before "HexData: ...", which loses the FSPEC bytes; - -jh / -je: "fspec":"..." before "hexdata":"...", same layout as the rest of their header. Only XML (-x/-xh) keeps the whole record in hexdata. The data block "HexData:" line of the text output (CAT+LEN of the block) is unchanged. Incompatible for readers of hexdata in -jh, -je and text: use fspec + hexdata to get the whole record. The FSPEC hex and the items hex are computed once per record for the text and JSON formats (DataRecord::getText). Expected outputs regenerated for the affected tests only (filter.txt left as it is). Checked by rebuilding the old output from the new one (old hexdata == new fspec + new hexdata, all other lines equal) on all samples and a slice of a 51 MB GPS file, with every FSPEC well formed; -l, -x, -xh, -j and -jx are byte-identical; ASan/UBSan clean. Replaces the learning on the hex JSON output (l-5bf7ab58 -> l-565293c9): it said -jh, -je and text kept the whole record. Co-Authored-By: Claude Opus 5.5 <[email protected]>
- -h: a "(default)" entry under "Output format" describes the text output used when no output format option is given (data block and record headers, Fspec + HexData, items with values, units and meanings); -j, -jh, -je and -jx note the fspec/hexdata record header; the usage line lists -G and -xh, which were only in the option list. - README usage block mirrors -h (its usage line had the same gaps). - README.md: the record header item now covers -j, -jh, -je and text. - HISTORY: 2.8.11-ifsnop.4, including the changes merged after 2.8.11-ifsnop.3 (format strings, CodeQL and publish workflows). The fork revision goes up because the -jh/-je/text hexdata change is incompatible with 2.8.11-ifsnop.3. Co-Authored-By: Claude Opus 5.5 <[email protected]>
FSPEC apart in -jh, -je and text output; document the default output in -h; 2.8.11-ifsnop.4
"Test filter" compares `asterix -L` (every configured item) with
install/sample_output/filter.txt, which was never regenerated after the
XML definitions changed:
- CAT015 was added (148 items missing from filter.txt);
- CAT001 I100 QB1/QD1 were renamed QB4/QD4.
filter.txt is regenerated with -L. It is also the -LF input of the
filtered CAT001/002 and CAT034/048 tests; their expected outputs were
regenerated too and did not change, because the samples contain no
CAT001 I100 and no CAT015 records.
install/test/test.sh now passes completely ("All tests OK").
Replaces the learning that kept this test failing on purpose
(l-2ef9bc59 -> l-5bec4220).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Tests: refresh filter.txt so "Test filter" passes
The HDLC reader always asked the device for MAX_RXBUF (4096) bytes, and a file device only returns complete reads: an HDLC file under 4 KB decoded nothing, and longer files lost their last partial buffer (a 5 KB file with 200 records gave 155). When the device knows how many bytes are left (files), the reader asks for at most that. Devices that do not know (serial, stdin: 0) keep reading MAX_RXBUF as before. Read() itself is unchanged: the ORADIS and GPS readers rely on it failing when a block is incomplete. Found by the synthetic HDLC sample of the new test suite. Co-Authored-By: Claude Opus 5.5 <[email protected]>
A <Category> is handed to the definitions at </Category>. When a definitions file failed to parse inside a category (syntax error, unknown element), that Category and its data items were leaked. ~XMLParser() now deletes it; after a successful handover the pointer is already NULL. Found by LeakSanitizer in the new unit tests (definitions file with a syntax error). Co-Authored-By: Claude Opus 5.5 <[email protected]>
proc_hdlc_frame() walks the Asterix blocks of a frame by their length field. A block length of 0 never advanced, so a frame with a valid FCS and such a block looped forever (reachable from serial input and from HDLC files of 4 KB or more; smaller files did not get that far before the previous fix). The frame is now rejected when a block length is under 3 (its own header) or when no room is left for a header. Found by the mutated-input run of the new test suite; the regression input is tests/data/bad/hdlc_block_len_zero.hdlc. Co-Authored-By: Claude Opus 5.5 <[email protected]>
The packet buffer was allocated with incl_len straight from the file. A corrupt value asked for up to 4 GB: depending on the memory available the program aborted with an uncaught std::bad_alloc, or read on with a huge buffer. incl_len must now be between 1 and libpcap's MAXIMUM_SNAPLEN (262144) and, for files, not more than what is left; otherwise the packet is reported and skipped like any other read error. Found by the mutated-input run of the new test suite; the regression input is tests/data/bad/pcap_huge_incl_len.pcap. Co-Authored-By: Claude Opus 5.5 <[email protected]>
The record length was computed as byte count - header - padding in an unsigned variable, so a byte count under 12 wrapped around to almost 4 GB and the "neededLen <= 0" check could never fire: the buffer allocation then aborted with std::bad_alloc (or read on with a huge buffer when memory allowed). The byte count must now be larger than header + padding and, for files, fit in what is left. Found by the robustness run of the new test suite with a memory limit; the regression input is tests/data/bad/final_count_too_small.final. Co-Authored-By: Claude Opus 5.5 <[email protected]>
tests/run_tests.sh [binary] [sections] checks the command line tool: - golden: 13 inputs (tests/cases.txt) x 8 output formats x with and without -LF, against tests/golden (UPDATE=1 rewrites them). Wall-clock timestamps of raw/ORADIS input are masked. - consistency (tools/check_outputs.py): every -jx record has a well formed fspec, fspec+hexdata of "length" bytes and a matching crc; every field's hex is at pos in hexdata, its mask fits and an integer val equals the masked bits; keys are unique; fspec/hexdata agree in -j, -jh, -je, text and XML; -j values equal -jx val. - roundtrip (tools/roundtrip.py): data fields are edited with hexdata[pos..] = (hex & ~mask) | (value & mask), the records rebuilt and decoded again: exactly the edited values change. - inputs: stdin, UDP multicast on loopback, loop mode -o. - robustness: malformed inputs (tests/data/bad), optional mutated inputs (MUTANTS=dir, tools/mutate.py), a % in the definitions file name; runs must end in time, without signal or sanitizer report (MEMLIMIT_KB caps memory so huge allocations fail everywhere). - cli: -h, -V, option errors, -L against filter.txt. tests/tools/gen_samples.py builds the synthetic inputs from the real samples: the CAT001/002 blocks re-wrapped as raw, ORADIS, FINAL and HDLC files, a stream with every category, and malformed inputs (each of them a regression of a fixed bug). tests/data/filter_subset.txt is an -LF filter that really filters (filter.txt lists every item). Co-Authored-By: Claude Opus 5.5 <[email protected]>
… as a library CMake (minimum 3.10, as CMake 4 refuses older minimums): - asterix_decoder: static library with the decoder (src/asterix without the packet formats), linked by the application, the unit tests and the fuzz target. - -DASTERIX_TESTS=ON: tests/unit (asterix_unit_tests) and ctest, which also runs tests/run_tests.sh with the CMake binary. - -DASTERIX_FUZZ=ON: tests/fuzz/fuzz_parse, libFuzzer with clang (the decoder gets -fsanitize=fuzzer-no-link), a plain program that runs the given files with other compilers. - -DASTERIX_COVERAGE=ON: gcov build; tests/tools/gcov_flush.c turns SIGTERM into exit() so stopped runs keep their data. Unit tests (tests/unit, own single-header harness, C++11): bit fields and encodings through getText, the -jx hex/mask/pos text, the getLength contract (no over-read, > nMaxLength when it does not fit, same length on its own bytes) on every item of every category, parser edge cases, CAT001 UAP selection, record headers of every format, format()/append helpers against printf, regressions (Tracer %, XML errors, truncated BDS). tests/tools/coverage_report.py reports line coverage of src/ and fails under tests/coverage_min.txt (63%; 64.6% measured, 48% before). Co-Authored-By: Claude Opus 5.5 <[email protected]>
.github/workflows/tests.yml: - make: release and debug build, install/test (with valgrind), tests/run_tests.sh, and a check that tests/data matches gen_samples.py. - cmake: C++11 build with gcc and clang, ctest (unit tests and run_tests.sh with that binary). - sanitizers: ASan+UBSan build, unit tests with leak detection, run_tests.sh with 520 mutated inputs, the decoder on every input. - fuzz: libFuzzer for 60 s on each push, 30 min weekly; the crashing input is kept as an artifact. - coverage: gcov build, all tests, report in the job summary, fails under tests/coverage_min.txt. tests/README.md describes the layers, how to run them locally and what to update when an output changes on purpose. install/test/test.sh now prints the diff of a failed case. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…memory Co-Authored-By: Claude Opus 5.5 <[email protected]>
Text fields (callsign, aircraft type, airports, runway, stand, SID/STAR, CAT004 designators...) printed every printable character as is, so a quote or backslash in damaged data made -j/-jx invalid JSON (12 lines over 1950 mutated inputs), and '<' or '&' would break XML. Per the ASTERIX definitions these fields are upper-case alphanumeric padded with spaces (e.g. I062/390: "up to seven upper-case alphanumeric characters ... padded with space characters"; ICAO aircraft identification: alphanumeric, no symbols). The radio channel name of I205/090 also has a decimal point (121.100). Every other character, control characters included, is now printed as '@', which no output format has to escape. -jx "hex" keeps the raw bytes. Golden outputs: 582 characters changed in 18 files, all of them a control character used as padding in the samples (e.g. callsign DLHEGN + 0x01, runway NU2 0x00) that was printed as a space and is now '@'. Co-Authored-By: Claude Opus 5.5 <[email protected]>
The output is unchanged: after a malformed record, decoding goes on from where that record stopped (most records after it decode correctly, some may be wrong, as ASTERIX records carry no length). DataBlock now reports it once per block, so the data can be checked later: WARNING: ... malformed record 4 in data block CAT001 (59 bytes), 0 record(s) decoded after it may be wrong. Block: 01003BF7C6... The block is printed in hex as received (CAT, LEN and data). Tracer and the application's debug_trace accept messages of any length (they were cut at 1024 characters), and debug_trace prints decoder messages starting with "Warning: " as WARNING instead of ERROR. Tests: unit tests for the warning (whole block, no warning on a good block, messages over 1024 characters); run_tests.sh checks the warning on tests/data/bad/raw_broken_record_in_block.raw and that -j/-jx output of malformed and mutated inputs is valid JSON. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…cisions Co-Authored-By: Claude Opus 5.5 <[email protected]>
Test suite run on every push and pull request; 5 decoder fixes it found
Text fields keep only A-Z, 0-9, space and '.'; warning with the whole block on malformed records
PR #12 was merged into its base branch test_suite after PR #11 had already been merged into master, so its commits never reached master. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…fsnop.3 2.8.11-ifsnop.4 was set in version.h by PR #9 but never tagged; this release also includes everything merged after it: - record header with the FSPEC apart in -jh, -je and text (PR #9); - format-string fixes and workflow updates (PRs #7, #8); - the test suite run on every push and pull request (PR #11) and the five decoder fixes it found (HDLC files and block length 0, PCAP packet length, FINAL byte count, XMLParser leak); - text fields limited to A-Z, 0-9, space and '.', and the warning with the whole block on malformed records (PR #12). README.md lists the test suite and the fixes and explains how to run the tests; README gets a TESTS section and the CMake options; HISTORY details every change of 2.8.11-ifsnop.4. Co-Authored-By: Claude Opus 5.5 <[email protected]>
- build: state of the Tests workflow on master (shields.io, label "build"). - coverage: on each push to master, the coverage job writes the line coverage as a shields.io endpoint JSON (COVERAGE_BADGE in coverage_report.py) to badges/coverage.json. The badges branch holds that one file in a single commit, force-pushed every time. That step is the only one with write permission (job-level contents: write). The README now says which badges are this fork's and which belong to the original project. Co-Authored-By: Claude Opus 5.5 <[email protected]>
README: build and coverage badges
The -jx path of DataItemBits::getText was rewritten with direct appends in the speed-up work, but the field key, the numeric value and the text value of -j, -jh, -x and -xh still went through format() (vsnprintf): 61% of the run time of -j. They now use the existing helpers (appendLong for %ld, appendDouble7 for %.7lf, direct appends for keys, quoted strings, XML tags and the separator), writing the same text. 51 MB GPS file (asterix_enaire.ini, -G), output identical (md5): -j 15.4 s -> 7.7 s (39.9 s before any optimization) -jh 15.6 s -> 8.5 s -x 17.6 s -> 8.8 s -xh 22.0 s -> 10.8 s The 208 golden outputs and install/test pass unchanged; unit tests and the ASan run with 520 mutated inputs are clean. Co-Authored-By: Claude Opus 5.5 <[email protected]>
-j, -jh, -x and -xh about twice as fast, same output
Includes since 2.8.11-ifsnop.4: - -j, -jh, -x and -xh about twice as fast, same output (PR #14); - build and coverage badges in the README (PR #13). No output change, so it is compatible with ifsnop.4; the revision goes up to mark the new release, as requested. Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 43fede5)
The upstream fix (previous commit, CroatiaControlLtd#254) only touched asterix/config, used by the Python module. Copy it to install/config so the C++ binary decodes CAT048 RE too, and regenerate install/sample_output/filter.txt with -L for the new CAT048:IRE:VAL. Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




2.8.11-ifsnop.1 (python_v0.7.11 unchanged)
First version of the fork https://github.com/ifsnop/asterix, based on CroatiaControlLtd/asterix 2.8.11.
The original project has not been updated since December 2025.
Security fixes (found with AddressSanitizer/UBSan and fuzzing):
Out-of-bounds reads when computing item lengths of truncated or malformed records
Out-of-bounds reads in PCAP link/IP/UDP headers and ORADIS headers, endless loop with ORADIS byte count 0
Stack overflow parsing BDS entries of -LF filter files
Memory leak with ORADIS input
Invalid casts of Repetitive sub-formats (BDS) to Fixed
Non printable characters in ASCII fields copied to the output
Stack and static buffer overflows building file names in the disk device
TCP client socket used after close, definition object leaked when asterix.ini is missing
Source IP of each record in TXT, JSON and XML outputs (UDP multicast input and GPS files)
New output format -jx / --json-hex: compact JSON, one record per line, with hex and mask of each field
2.8.11-ifsnop.2 (python_v0.7.11 unchanged)
-jx / --json-hex made to change a value and write it back to ASTERIX:
"fspec" (FSPEC bytes) and "hexdata" (data items after the FSPEC): the record is fspec+hexdata
"pos" in every field: byte offset of its hex in hexdata (first byte of hexdata is pos 0)
"mask" whenever a field does not fill its hex bytes, so hexdata[pos..] = (hex & ~mask) | (value & mask)
Unique keys: FX bits numbered by extent (FX0, FX1, ...), other repeated names numbered (spare, spare1)
-j record header changed (incompatible with the original): "fspec" printed separately and "hexdata" holds
only the data items after the FSPEC (in the original, hexdata was the whole record)
2.8.11-ifsnop.3 (python_v0.7.11 unchanged)
Faster decoding with byte-identical output: -jx about 8x faster (51 MB GPS file: 68 s -> 8.5 s),
other output formats 2-2.6x faster:
format() formats once into a stack buffer instead of twice through the heap
-jx fields written without allocations (hex, mask and pos built in place, mask built once per field)
Items written straight into the output string instead of copied through temporary strings
-jx unique keys computed once per format instead of a std::set per field
Cached FRN -> item and Compound presence-bit lookups
Disk input: no ftell/fstat/select system calls per packet
make rebuilds objects when an included header changes (dependency targets were never matched);
run make clean once after updating
2.8.11-ifsnop.4 (python_v0.7.11 unchanged)
Record header with the FSPEC apart in -jh, -je and the default text output, as in -j and -jx (incompatible):
"fspec" (JSON) / "Fspec:" (text) has the FSPEC bytes and "hexdata" / "HexData:" only the data
items after the FSPEC: the record is fspec+hexdata. XML keeps the whole record in hexdata.
-h documents the default text output and lists -G and -xh in the usage line
Format strings: text from input data or file names is never used as a printf format
(a definitions file name with % made the program crash); format/argument type mismatches fixed
Python publish workflow runs only by hand; CodeQL workflow updated (codeql-action v4, make build)
Text fields (ASCII encoding: callsign, aircraft type, airports, runway, stand, SID/STAR, CAT004
designators...) only print A-Z, 0-9, space and '.' (radio channel names, I205/090); any other
byte is printed as '@'. A quote or backslash in damaged data made -j/-jx invalid JSON.
Visible on captures that pad these fields with control characters: they used to print as
spaces and now print as '@' (e.g. callsign DLHEGN + 0x01 -> DLHEGN@). -jx "hex" keeps the raw bytes.
Malformed records: decoding goes on after them as before, and a WARNING on stderr gives the record
number and the whole data block in hex (CAT, LEN, data) for later analysis. Decoder messages
are no longer cut at 1024 characters.
Fixes found by the new test suite:
HDLC input from files: files under 4 KB decoded nothing and longer files lost their last
partial 4 KB buffer
HDLC: a frame with an Asterix block length of 0 looped forever
PCAP: a corrupt packet length (incl_len) allocated up to 4 GB and could abort (std::bad_alloc);
lengths over 262144 bytes or past the end of the file are rejected
FINAL: a byte count smaller than the record header wrapped around to almost 4 GB (same abort)
XMLParser: a definitions file that failed inside leaked that category
Test suite (tests/, see tests/README.md), run by GitHub Actions on every push and pull request:
golden outputs: 13 inputs x 8 output formats x with and without -LF, including synthetic FINAL,
HDLC and ORADIS files generated from the real samples (tests/tools/gen_samples.py)
-jx consistency (fspec, length, crc, hex/mask/pos/val, unique keys) and agreement of -j, -jh,
-je, text and XML; write-back of edited values through hex/mask/pos
stdin, UDP multicast and loop mode input; command line options
malformed inputs (one per fixed bug) and mutated inputs under AddressSanitizer/UBSan, valid JSON
whatever the input; unit tests (bit fields, getLength contract of every item of every category,
parser, UAP selection, record headers); libFuzzer; line coverage floor (65%, from 48%)
CMake: minimum version 3.10; the decoder is a static library; options ASTERIX_TESTS, ASTERIX_FUZZ and
ASTERIX_COVERAGE. install/test/test.sh prints the diff of a failed case.
2.8.11-ifsnop.5 (python_v0.7.11 unchanged)
-j, -jh, -x and -xh about twice as fast as in ifsnop.4, output byte for byte the same: the field key,
numeric value and text value no longer go through format() (51 MB GPS file: -j 15.4 s -> 7.7 s,
39.9 s before any optimization; -jh 15.6 -> 8.5 s; -x 17.6 -> 8.8 s; -xh 22.0 -> 10.8 s)
README: build and coverage badges of this fork; the coverage job publishes the line coverage to the
badges branch on each push to master