Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion godbrain_core/cpp_memory_store/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# C++ Alexandria Memory Store (cut 10)
# C++ Alexandria Memory Store (cut 11)

Same stdin JSON door as Go `memory-store.exe`. Go under `godbrain_core/memory_store/`
stays. This tree is the C++ replacement; Start/Heal still launch Go until this
Expand Down Expand Up @@ -49,6 +49,10 @@ Cut 10: `rag-eval.exe` offline hybrid fixture (`-corpus` path, default Go
testdata). `-live` still uses Go. Thresholds match Go (Recall/MRR/nDCG ≥ 0.90,
citation 1.0, no leakage).

Cut 11: ingest `document` + `chunks` (Local-Document-Adapter). Same gates as Go:
metadata and chunks together, `content_sha256` of `raw_transcript`, contiguous
UTF-8 byte ranges, forbidden-secret scan, immutable `chunks` collection.

Start/Heal still launch Go.

```powershell
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ bool embedding_cosine(const std::vector<float>& left, const std::vector<float>&
bool embedding_embed(
const EmbeddingRuntime& runtime, const std::string& content, std::vector<float>* vector, std::string* err);
bool embedding_embed_fake(int dimension, const std::string& content, std::vector<float>* vector, std::string* err);
bool sha256_hex(const std::string& bytes, std::string* hex, std::string* err);

int run_embedding_self_test();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,30 @@ struct Claim {
std::vector<std::string> evidence_spans;
};

struct DocumentMetadata {
std::string source_label;
std::string display_name;
std::string file_sha256;
std::string content_sha256;
std::string extraction_method;
std::vector<std::string> languages;
std::string backend;
std::string backend_version;
int chunk_count = 0;
bool has_ocr_confidence = false;
double ocr_confidence = 0;
};

struct SourceChunk {
int index = 0;
int count = 0;
int start_byte = 0;
int end_byte = 0;
std::string text;
bool has_confidence = false;
double confidence = 0;
};

struct SkillExtracted {
std::string name;
std::string content;
Expand All @@ -77,8 +101,12 @@ struct DistillationPayload {
std::vector<std::string> opsec_candidates;
std::vector<SkillExtracted> skills_extracted;
bool has_document = false;
DocumentMetadata document;
std::vector<SourceChunk> chunks;
};

bool validate_document_payload(const DistillationPayload& p, std::string* err);

std::string json_escape(const std::string& s);
std::string skill_procedure_text(const SkillExtracted& skill);
std::string skill_stable_id(const std::string& name, const std::string& content);
Expand Down
4 changes: 4 additions & 0 deletions godbrain_core/cpp_memory_store/src/embedding.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ std::wstring fields_join(const std::wstring& in) {
return out;
}

} // namespace

bool sha256_hex(const std::string& bytes, std::string* hex, std::string* err) {
BCRYPT_ALG_HANDLE alg = nullptr;
NTSTATUS st = BCryptOpenAlgorithmProvider(&alg, BCRYPT_SHA256_ALGORITHM, nullptr, 0);
Expand Down Expand Up @@ -174,6 +176,8 @@ bool sha256_hex(const std::string& bytes, std::string* hex, std::string* err) {
return true;
}

namespace {

std::string wide_lower(const std::wstring& w) {
if (w.empty()) return "";
std::wstring copy = w;
Expand Down
Loading