Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,22 @@ Protocol) stdio server. No AI / vector / LLM anywhere in the binary — output i
where the unit struct produced NO node before — plus the Rust half of PR #1516's
first-class `union` kind, byte-for-byte; it deliberately claims NO `instantiates`
edge, because that edge fires only for the call-expression construction form and
a Rust union is constructible only as `U { … }`).
a Rust union is constructible only as `U { … }`), and `reference/golden/go/`
(corpus `crates/codegraph-bench/fixtures/go/`; the repo's only Go corpus, and the
only corpus where `files.generated` is anything other than `0` — it byte-pins
#1500 content-header detection at BOTH values, three each way: `payroll.go`
(Go's codified `Code generated … DO NOT EDIT.`), `worker_types.go` (pattern 6 /
CG-25, Wrangler's TWO `by` clauses) and `api.pb.go` (the PATH signal, no banner)
at `1`; `payroll_usecase.go` (hand-written, and the must-not-demote side),
`nightly.go` (ONE `by` clause — ordinary prose) and `generator.go` (the banner is
a `const` in the function BODY, so the comment-line fence rejects it) at `0`).
The Python cross-file edge is resolved by Gate 3a's unique-name fallback;
import Gate 3b is deliberately unreachable for real Python nodes because the
extractor does not mark them exported. Regen recipe: `docs/equivalence.md`
"Godot fixture" / "Ruby fixture" / "Python fixture" / "Kotlin fixture" /
"TypeScript resolution fixture" / "C++ fixture" / "Rust fixture" sections.
"TypeScript resolution fixture" / "C++ fixture" / "Rust fixture" / "Go fixture"
sections. `mini` has no re-indexable provenance and a schema migration cannot
normalise it in place, so it has its own "Mini schema rebuild" recipe there.
- **node-id formula**: `{kind}:{sha256("{filePath}:{kind}:{name}:{line}").hex[:32]}`; file nodes are the
literal `file:{relpath}`; lines are 1-based; paths relative with `/`.
- **No AI / vector / LLM crates** — enforced by `scripts/guardrail.sh` (CI gate):
Expand Down
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,19 @@ Project configuration remains in `.codegraph/config.toml` and
fields `legacyIndexPresent` and `legacyIndexPaths` remain in `status --json`, but
always report `false` and `[]` respectively.

### Re-index required for content-header generated-file detection

Generated-file detection now also reads a file's **content banner**, not only its
filename, so a machine-written `payroll.go` carrying
`// Code generated … DO NOT EDIT.` is demoted in ranking like a `.pb.go` file
already was. The verdict is computed at index time and stored on the file row.

An index built by an earlier release therefore reports every file as
non-generated for the content half until it is re-extracted. Nothing breaks in the
meantime — the filename signal keeps working exactly as before — and the next
`codegraph index` or `codegraph sync` picks the new detection up automatically,
since the extraction-version bump classifies the old index as outdated.

---

## Installation
Expand Down
14 changes: 14 additions & 0 deletions crates/codegraph-bench/fixtures/go/api.pb.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
package payroll

type PayrollRequest struct {
ID string
Amount int64
}

func (m *PayrollRequest) Reset() {
*m = PayrollRequest{}
}

func (m *PayrollRequest) GetAmount() int64 {
return m.Amount
}
14 changes: 14 additions & 0 deletions crates/codegraph-bench/fixtures/go/generator.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
package payroll

// Emitter writes CRUD scaffolding for a table.
type Emitter struct {
Table string
}

// Emit renders the file body, prefixing it with the standard banner.
// The banner literal lives in the BODY as a string constant, so this
// generator source must NOT be classified as generated itself.
func (e *Emitter) Emit() string {
const banner = "// Code generated by fkit-crud. DO NOT EDIT."
return banner + "\\n\\npackage " + e.Table + "\\n"
}
9 changes: 9 additions & 0 deletions crates/codegraph-bench/fixtures/go/nightly.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
package payroll

// RunNightly kicks off the ETL job. Note the wording below: the report is
// generated by running the ETL job, which is ONE "by" clause and therefore
// ordinary prose rather than a generation banner.
func RunNightly(jobID string) error {
_ = jobID
return nil
}
12 changes: 12 additions & 0 deletions crates/codegraph-bench/fixtures/go/payroll.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions crates/codegraph-bench/fixtures/go/payroll_usecase.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package payroll

// ComputePay applies the real business rules for a payroll run.
func ComputePay(p *Payroll, bonus int64) int64 {
return p.Amount + bonus
}
12 changes: 12 additions & 0 deletions crates/codegraph-bench/fixtures/go/worker_types.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
// Generated by Wrangler by running `wrangler types` (hash: 6f3a1c9d2b)

package worker

type Env struct {
AccountID string
Bucket string
}

func NewEnv(account string, bucket string) *Env {
return &Env{AccountID: account, Bucket: bucket}
}
8 changes: 6 additions & 2 deletions crates/codegraph-bench/src/oracle/canonicalize.rs
Original file line number Diff line number Diff line change
Expand Up @@ -146,8 +146,9 @@ fn canonical_refs(conn: &Connection) -> Result<Vec<CanonicalRow>> {
}

fn canonical_files(conn: &Connection) -> Result<Vec<CanonicalRow>> {
let mut stmt =
conn.prepare("SELECT path, content_hash, language, size, node_count, errors FROM files")?;
let mut stmt = conn.prepare(
"SELECT path, content_hash, language, size, node_count, errors, generated FROM files",
)?;
let rows = stmt.query_map([], |row| {
Ok(RawFile {
path: row.get(0)?,
Expand All @@ -156,6 +157,7 @@ fn canonical_files(conn: &Connection) -> Result<Vec<CanonicalRow>> {
size: row.get(3)?,
node_count: row.get(4)?,
errors: row.get(5)?,
generated: row.get(6)?,
})
})?;

Expand Down Expand Up @@ -387,6 +389,7 @@ struct RawFile {
size: i64,
node_count: i64,
errors: Option<String>,
generated: i64,
}

impl RawFile {
Expand All @@ -399,6 +402,7 @@ impl RawFile {
row.insert("size".to_string(), json!(self.size));
row.insert("node_count".to_string(), json!(self.node_count));
row.insert("errors".to_string(), json_column(self.errors, "errors")?);
row.insert("generated".to_string(), json!(self.generated));
Ok(row)
}
}
28 changes: 28 additions & 0 deletions crates/codegraph-bench/tests/equivalence.rs
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,26 @@ fn rust_db_is_self_equivalent_to_rust_golden() {
assert_equivalent(&rust_db(), &rust_golden_dir()).unwrap();
}

#[test]
fn generated_golden_matches_committed_go_fixture() {
// Guards #1500 content-header detection: `files.generated` is byte-pinned at
// BOTH values — 1 for the Go banner, the Wrangler double-`by` banner, and the
// path-only `.pb.go`; 0 for the hand-written sibling, the single-`by` prose
// negative, and the generator whose banner is a `const` in its body.
let tempdir = TestDir::new("generated-golden-go");
write_golden(&go_db(), tempdir.path()).unwrap();

let expected = load_golden(&go_golden_dir()).unwrap();
let actual = load_golden(tempdir.path()).unwrap();

diff_canonical(&expected, &actual, None).unwrap();
}

#[test]
fn go_db_is_self_equivalent_to_go_golden() {
assert_equivalent(&go_db(), &go_golden_dir()).unwrap();
}

#[test]
fn generated_golden_matches_committed_python_fixture() {
let tempdir = TestDir::new("generated-golden-python");
Expand Down Expand Up @@ -438,6 +458,14 @@ fn rust_golden_dir() -> PathBuf {
workspace_root().join("reference/golden/rust")
}

fn go_db() -> PathBuf {
workspace_root().join("reference/golden/go/colby.db")
}

fn go_golden_dir() -> PathBuf {
workspace_root().join("reference/golden/go")
}

fn python_db() -> PathBuf {
workspace_root().join("reference/golden/python/colby.db")
}
Expand Down
3 changes: 3 additions & 0 deletions crates/codegraph-cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ use anyhow::{Context, Result, anyhow, bail};
use clap::{CommandFactory, Parser, Subcommand, ValueEnum};
use clap_complete::{Shell, generate};
use codegraph_core::config::Config;
use codegraph_core::generated_header::detect_generated_file;
use codegraph_core::logger::{LoggerConfig, init_logger};
use codegraph_core::node_id::hash_content;
use codegraph_core::types::{ExtractionResult, FileRecord, Language, Node, NodeKind};
Expand Down Expand Up @@ -4866,6 +4867,7 @@ fn index_project_inner(
.filter(|n| n.file_path == *relative)
.count() as i64,
errors: result.errors.clone(),
generated: detect_generated_file(relative, &source),
};
tx.send((i, relative.clone(), file, result))
.map_err(|_| anyhow!("parse result channel disconnected"))?;
Expand Down Expand Up @@ -6957,6 +6959,7 @@ mod formatter_and_env_tests {
indexed_at: 0,
node_count,
errors: Vec::new(),
generated: false,
}
}

Expand Down
44 changes: 44 additions & 0 deletions crates/codegraph-cli/tests/parallel_index.rs
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ fn mini_fixture() -> PathBuf {
workspace_root().join("crates/codegraph-bench/fixtures/mini")
}

fn go_fixture() -> PathBuf {
workspace_root().join("crates/codegraph-bench/fixtures/go")
}

fn copy_tree(src: &Path, dst: &Path) {
fs::create_dir_all(dst).unwrap();
for entry in fs::read_dir(src).unwrap() {
Expand Down Expand Up @@ -258,6 +262,46 @@ fn oversized_file_size_skips_with_exact_file_record() {
);
}

/// WRITE PATH 1 (`index --force`, the rayon producer closure): the persisted
/// `files.generated` must carry the real per-file verdict, not a hardcoded false.
/// This is the CLI counterpart to the two `codegraph-watch` tests that cover the
/// incremental-sync and Outdated-rebuild write paths — three independent sites
/// build their own `FileRecord`, so each needs its own end-to-end assertion.
#[test]
fn index_force_writes_the_generated_flag_per_file() {
let dir = TestDir::new("generated-flag");
let project = dir.path().join("go");
copy_tree(&go_fixture(), &project);

init_and_force_index(&project);

let store = Store::open(&db_path(&project)).unwrap();
let mut rows: Vec<(String, i64)> = store
.connection()
.prepare("SELECT path, generated FROM files ORDER BY path")
.unwrap()
.query_map([], |row| {
Ok((row.get::<_, String>(0)?, row.get::<_, i64>(1)?))
})
.unwrap()
.map(Result::unwrap)
.collect();
rows.sort();

assert_eq!(
rows,
vec![
("api.pb.go".to_string(), 1),
("generator.go".to_string(), 0),
("nightly.go".to_string(), 0),
("payroll.go".to_string(), 1),
("payroll_usecase.go".to_string(), 0),
("worker_types.go".to_string(), 1),
],
"index --force must persist the real generated verdict for every file"
);
}

/// SYNC PRAGMA GUARD (end-to-end): the bulk-index path runs under
/// `synchronous=OFF` (T5) but must restore NORMAL before returning. After a real
/// `index --force` via the CLI, a FRESH `Store::open` on the produced DB must
Expand Down
Loading
Loading