Symptom#

A multi-member gzip file cut short (a trace still being written, or a copy that stopped) loses all of its last member: every event in it is missing from the index and the reads, and reads may throw. In one case a View returned 0 rows instead of 3,594.

Cause#

libdeflate decodes a whole member into one buffer. It has no streaming mode and gives no partial output. For a cut member it returns BadData, the same code it returns for a member that is not fully read yet, so the reader kept reading more bytes, retried, and at end of file dropped the member.

Fix#

• Keep libdeflate for every complete member; it is the fast path.
• Only when libdeflate returns BadData at end of file, decode that last member with zlib (zlib-ng) streaming inflate(), which gives all output up to the cut. This runs at most once per file.
• Drop the partial last line, mark the member as truncated (not checksum-verified), and let readers stop cleanly at the cut. The readers need the same fallback as the index build.
• Report truncated files (dftracer-utils: IndexStatus::truncated).

Evidence#

• dftracer-utils stage 2b: on the laghos papi_set1 trace cut at 5%, 50%, 90% and 99%, the indexed line counts matched zlib's prefix decode exactly (80,526; 796,253; 1,344,664; 1,488,574).
• The read-path tests failed without the fix (the reads threw, and the View returned 0 rows instead of 3,594) and passed with it; complete members still use libdeflate, and the benchmark stayed within 5% of the baseline.