Skip to content

feat(errortracking): attach the tombstone text to native crash events - #868

Draft
turnipdabeets wants to merge 2 commits into
mainfrom
feat/native-crash-report
Draft

turnipdabeets wants to merge 2 commits into
mainfrom
feat/native-crash-report

Conversation

@turnipdabeets

@turnipdabeets turnipdabeets commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

This is the Android pair of PostHog/posthog-ios#957 and addresses PostHog/posthog#117880, which came from a customer support ticket. Native (NDK) crash events only carry the crashing thread's stack. Customers can't see the full report Android itself produces for a native crash, the debuggerd tombstone: every thread, the registers, the signal details.

The SDK already reads the tombstone on the next launch (ApplicationExitInfo, Android 12+). It now renders it as text in debuggerd's own format, following AOSP's tombstone_proto_to_text.cpp, and attaches it as $exception_crash_report. That's the same property iOS sends. $exception_list and $debug_images don't change, so symbolication and issue grouping work exactly as before.

What's in the report:

  • the process header
  • the signal, its cause and the abort message
  • the crashing thread's registers
  • every thread's backtrace

What's left out, and why:

  • Memory dumps, memory maps, open files and logcat: they can hold user data, and the maps alone run to thousands of lines.
  • Other threads' registers: about a sixth of the size. debuggerd's logcat output and the iOS report leave them out too.

Size, which needs a decision from error tracking: a native crash event grows from about 70 KB to about 300 KB. A typical app has about 50 threads, and ART's frames are long. That's much more than iOS, about +10 KB.

  • Like iOS, the SDK measures the serialized event before sending. The check is in the core capture path and runs on the fully built event, after super properties, context and beforeSend, which is why the core posthog package changes too. If the event is over 900 KB, the SDK drops only the report, so the crash event is never lost.
  • At about 4.5 KB per thread, apps with more than about 185 threads won't get the report.
  • A possible follow-up for both SDKs: fall back to a report with only the crashing thread instead of dropping it.

Also in this PR:

  • Report-only fields can't lose the crash. Fields that only the report uses (causes, registers) are parsed leniently, and formatting errors are caught. A malformed value drops just that field, or the report, never the crash event.
  • A pre-existing parser bug, fixed in the report only. proto3 leaves out a fault address of 0, so null-dereference crashes lost their fault address. The report now prints fault addr 0x0… like debuggerd. I deliberately didn't change the $exception value, which still says SEGV_MAPERR instead of SEGV_MAPERR at 0x0, because changing it would regroup existing issues. That should be tracked separately.

For error tracking (@PostHog/team-error-tracking):

  • Same asks as #957: confirm the property name and the size trade-off.
  • No sdk-spec covers this property yet. If you're happy with the name, we'll add it to the exception-event-metadata spec.
  • Today the report only shows when you turn on the Properties tab's JSON toggle.

💚 How did you test it?

  • Unit tests:
    • Parser tests use hand-encoded protobuf: header fields, all threads, a negative signal code, a zero fault address, and a malformed cause or register that must not fail the crash.
    • A golden-text test for the formatter, plus a test for crashes with several causes.
    • Size-check tests: one counts JSON escaping, and one goes through the real capture path with a large registered property.
    • The full suite passes: 765 posthog-android and 1,036 core tests. spotlessCheck and apiCheck are clean, and there's no public API change.
  • End to end on a Pixel 9 emulator (API 37, arm64). I triggered a real SIGSEGV in the sample app and relaunched it to send the crash. In PostHog:
    • The event arrived with the report: about 230–260 KB and 51–53 threads, varying by run.
    • The crashing thread's text matches what debuggerd printed to logcat for all 58 backtrace lines. This newer Android image prints a few extra header lines (kernel release, ppid, executable, pointer-auth keys) that AOSP main's proto doesn't carry.
    • A baseline crash from unmodified main landed in the same issue, with an identical $exception_list, so grouping is unchanged.
  • Not tested: API 31–34 devices, and real MTE/HWASan tombstones (the multi-cause path is covered by unit tests only).

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

  • Tools: Claude Code (Opus 5.5) did the implementation, the emulator end-to-end test and the comparison against debuggerd output. I directed the work and reviewed it.
  • Format: the text follows AOSP's tombstone formatter (main, October 2026), with field numbers checked against AOSP's tombstone.proto. It's Android's own crash format, which Android developers already read in logcat and the Play Console.
  • Review: two rounds of independent review sub-agents ran before this PR. Fixes from them:
    • the missing changeset
    • report-only fields could have lost the crash event
    • several causes were dropped from the report
    • the size check is now fully inside its error handling

🤖 Generated with Claude Code

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed 8007fd0: 1 Should fix.

This is the only automatic review while this pull request is a draft. Comment @posthog review to review new changes sooner. Otherwise, a review of the new changes runs when you mark it ready for review.

@turnipdabeets
turnipdabeets marked this pull request as ready for review October 20, 2026 23:53
@turnipdabeets
turnipdabeets requested a review from a team as a code owner October 20, 2026 23:53
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

posthog-android Compliance Report

Date: 2026-10-27 16:09:41 UTC
Duration: 118012ms

✅ All Tests Passed!

46/46 tests passed


Capture Tests

✅ 29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 307ms
Format Validation.Event Has Uuid ✅ 22ms
Format Validation.Event Has Lib Properties ✅ 22ms
Format Validation.Distinct Id Is String ✅ 20ms
Format Validation.Token Is Present ✅ 22ms
Format Validation.Custom Properties Preserved ✅ 20ms
Format Validation.Event Has Timestamp ✅ 19ms
Retry Behavior.Retries On 503 ✅ 7025ms
Retry Behavior.Does Not Retry On 400 ✅ 4025ms
Retry Behavior.Does Not Retry On 401 ✅ 4021ms
Retry Behavior.Respects Retry After Header ✅ 7023ms
Retry Behavior.Implements Backoff ✅ 17031ms
Retry Behavior.Retries On 500 ✅ 7019ms
Retry Behavior.Retries On 502 ✅ 7014ms
Retry Behavior.Retries On 504 ✅ 7014ms
Retry Behavior.Max Retries Respected ✅ 17024ms
Deduplication.Generates Unique Uuids ✅ 30ms
Deduplication.Preserves Uuid On Retry ✅ 7013ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 12023ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 7017ms
Deduplication.No Duplicate Events In Batch ✅ 29ms
Deduplication.Different Events Have Different Uuids ✅ 19ms
Compression.Sends Gzip When Enabled ✅ 15ms
Batch Format.Uses Proper Batch Structure ✅ 13ms
Batch Format.Flush With No Events Sends Nothing ✅ 9ms
Batch Format.Multiple Events Batched Together ✅ 26ms
Error Handling.Does Not Retry On 403 ✅ 4015ms
Error Handling.Does Not Retry On 413 ✅ 4019ms
Error Handling.Retries On 408 ✅ 5022ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 29ms
Request Payload.Flags Request Uses V2 Query Param ✅ 31ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 19ms
Request Payload.Flags Request Omits Authorization Header ✅ 20ms
Request Payload.Token In Flags Body Matches Init ✅ 16ms
Request Payload.Groups Round Trip ✅ 17ms
Request Payload.Groups Default To Empty Object ✅ 16ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 16ms
Request Payload.Disable Geoip Omitted Defaults To False ✅ 15ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 14ms
Request Lifecycle.No Flags Request On Init Alone ✅ 6ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 21ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 32ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 19ms
Retry Behavior.Retries Flags On 502 ✅ 320ms
Retry Behavior.Retries Flags On 504 ✅ 317ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 18ms

@hpouillot

Copy link
Copy Markdown

🤖 Bot review: I’d fix the async MTE warning before release. For SEGV_MTEAERR, debuggerd explains that the crash is delayed and the stack is the first subsequent system call/context switch—not necessarily where corruption happened. This looks like a small formatter change plus a test, and omitting it could mislead someone reading $exception_crash_report.

I wouldn’t block release on GWP-ASan/Scudo heap allocation/deallocation backtraces: parseCause currently keeps only human_readable, so those debuggerd traces are missing, but supporting them needs nested parsing and tests. Could we explicitly list them among the report’s exclusions and track a follow-up?

@turnipdabeets
turnipdabeets marked this pull request as draft October 21, 2026 14:16
@turnipdabeets
turnipdabeets force-pushed the feat/native-crash-report branch from 6ec0288 to 89adf00 Compare October 26, 2026 17:11
@turnipdabeets
turnipdabeets force-pushed the feat/native-crash-report branch from 89adf00 to f8d2d53 Compare October 27, 2026 16:01

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants