Skip to content

[BUG] Checksum-keyed reconciliation can never repair an entity row whose type is wrong — sync reports it “unchanged” forever #1538

Description

@tsobczynski

Bug Description

If an entity row's type fields are wrong but its checksum matches the bytes on disk, no amount of re-syncing or re-indexing will repair it. Change detection compares checksums only, so the row is reported unchanged and is never re-parsed. There appears to be no mechanism anywhere that notices a row whose type disagrees with its file.

How we hit it

A markdown note's entity row was replaced by a plain-file row — note_type='file', content_type='text/plain', permalink=NULL, no note_content row, title taken from an Emacs autosave (filed separately as #1537).

The important part for this issue is the reason it was unrepairable. The autosave's bytes were byte-identical to what the note became 16 seconds later, so the corrupt row's checksum and size already matched the real file exactly. Every subsequent scan therefore said:

[CHANGE] Move detection: found 0 moved files
[CHANGE] Delete detection: found 0 deleted files
[CHANGE] Detection complete: 0 new, 0 modified, 0 deleted, 0 moved, N unchanged

Repeated incremental rebuilds changed nothing. This has now happened twice, weeks apart, and on the first occurrence the rebuild attempts were abandoned as ineffective without the cause ever being found.

Steps To Reproduce

  1. Obtain an entity row whose file_path is an existing .md note, but with note_type='file', content_type='text/plain', permalink=NULL, no note_content row — and a checksum that matches the file on disk. (Either via [BUG] Emacs autosave #note.md# is indexed as a non-markdown resource and can destroy the real note's entity row #1537, or by hand.)
  2. Run any sync / rebuild, however many times.
  3. Observe the row reported as unchanged; edit_note <permalink> still does not resolve.
  4. Now append a single byte to the file, and it repairs itself completely.

Expected Behavior

A sync or rebuild should be able to repair a row that disagrees with its own file. At minimum, a row whose file_path is a markdown path but whose content_type is not text/markdown — or a markdown entity with no note_content row — should be treated as dirty regardless of checksum.

Actual Behavior

The row is permanently invisible to sync. The only repair is to change the file's bytes so the checksum differs.

Worth noting that the repair machinery itself is entirely healthy — it just cannot be triggered. Once the checksum differed, index_markdown_file ran and _apply_markdown_entity_fields() rewrote title, note_type, permalink and content_type on the existing row, in place:

title         #note.md#   ->  note
note_type     file        ->  note
content_type  text/plain  ->  text/markdown
permalink     NULL        ->  proj/notes/note
observations  0           ->  N
note_content  absent      ->  present, synced

Same entity id, graph intact, no delete and no cascade. A one-byte append did what an unbounded number of rebuilds could not.

Possible Solution

  1. Add a consistency predicate to change detection alongside the checksum comparison — mark a row dirty when runtime_file_path_is_markdown_note(file_path) is true but content_type != 'text/markdown', or when a markdown entity has no note_content row.
  2. Offer a bm sync --force / --repair that ignores checksums for a project or path. Either of these would have let us fix this without resorting to modifying the user's note to provoke a reindex.
  3. Consider a bm doctor-style read-only consistency check, since this class of corruption is silent: read_note kept working the whole time, so nothing surfaced until a write was attempted.

Environment

  • OS: macOS 26.6.2 (Darwin 25.6.0), arm64
  • Python: 3.14.7
  • basic-memory: 0.23.2
  • Install method: pipenv virtualenv; MCP servers launched by Claude Desktop and Claude Code
  • Config: local SQLite, semantic_search_enabled: true, index_changes: true, kebab_filenames: false

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions