From ae17152dbf771829d332bf086a5ed241ee312b8f Mon Sep 17 00:00:00 2001 From: Arena Agent Date: Sun, 4 Oct 2026 01:16:10 +0000 Subject: [PATCH 1/2] fix(ci): declare Luxor, align deps with usage, repair test failures The three red Julia checks in #73 were red for reasons unrelated to the citation metadata that surfaced them: * src/intelligence/string_board.jl uses Luxor, which was never declared in [deps], so `using InvestigativeJournalism` failed before any test ran. * src/intelligence/forensics.jl shadowed Statistics.var/std with a vector-only definition, breaking detect_ai_artifacts on block matrices. * build_story_structure had no method for NewsBulletin or Thread. * the media-forensics tests analysed /tmp/photo.jpg and /tmp/suspect_image.png, files that nothing created. * the 15 minute job timeout was below the cost of installing and precompiling the dependency closure on a cold cache, so the 1.11 legs were cancelled mid-install without measuring anything. Also drops the five [deps] entries (SQLite, URIs, Gumbo, Cascadia, DuckDB) that no source, test, benchmark or example file references. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com> --- .github/workflows/ci.yml | 12 +++++++++++- CHANGELOG.adoc | 37 +++++++++++++++++++++++++++++++++++ EXPLAINME.adoc | 9 ++++++--- Project.toml | 22 ++++++++++----------- src/intelligence/forensics.jl | 13 +----------- src/storytelling/templates.jl | 21 ++++++++++++++++++++ test/runtests.jl | 36 ++++++++++++++++++++++++++++++++-- 7 files changed, 121 insertions(+), 29 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 616c648..079b5b1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -17,7 +17,17 @@ jobs: test: name: Julia ${{ matrix.julia-version }} - ${{ matrix.os }} runs-on: ${{ matrix.os }} - timeout-minutes: 15 + # This job resolves, downloads and precompiles the whole dependency + # closure (DataFrames, Images, Luxor/Cairo, VideoIO/FFMPEG, TextAnalysis, + # Graphs, plus the four hyperpolymath packages added from git) before a + # single test runs. On a cold ~/.julia that is a >15 minute job on the + # 2-core runners — 15 minutes was never a budget this workload could meet, + # and it killed the 1.11 legs mid-install without ever measuring the + # tests. 45 minutes matches the Pages job in this repository and gives the + # cold case real headroom; the dependency trim in Project.toml (unused + # SQLite/URIs/Gumbo/Cascadia/DuckDB entries removed) brings the warm case + # back under it by a wide margin. + timeout-minutes: 45 strategy: fail-fast: false matrix: diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc index ca1c652..6f77c0d 100644 --- a/CHANGELOG.adoc +++ b/CHANGELOG.adoc @@ -7,3 +7,40 @@ Changelog], and this project adheres to https://semver.org/spec/v2.0.0.html[Semantic Versioning]. === [Unreleased] + +==== Fixed + +* The package could not be loaded at all. `src/intelligence/string_board.jl` + `using`s `Luxor` (for `Point` and the `Drawing`/`background`/`line`/`rect` + rendering calls in `render_wall`), but `Luxor` was missing from `[deps]`, so + `using InvestigativeJournalism` — and therefore every CI test job — failed + with an undeclared-dependency error before a single test ran. `Luxor` is now + declared, with a compat entry, matching the sibling `JuliaKids.jl` and + `PRComms.jl` packages. +* `src/intelligence/forensics.jl` shadowed `Statistics.var`/`std` with a + vector-only local definition, so `detect_ai_artifacts` raised a + `MethodError` when it estimated sensor noise from a 4x4 block matrix. The + module now uses `Statistics` directly. +* `build_story_structure` had a method for `Longform` only; the `NewsBulletin` + and `Thread` templates (the narrative structures the ROADMAP claims for + v1.2.0) now return their own non-empty structure lists. +* The media-forensics tests ran the analysers against `/tmp/photo.jpg` and + `/tmp/suspect_image.png`, which nothing ever created, so both threw + `ArgumentError` instead of exercising the code. Tests now write a real + embedded PNG fixture to a temp directory and, separately, pin the intended + "missing file is rejected, not scored" behaviour. + +==== Changed + +* CI: the test job's `timeout-minutes` is 45 rather than 15. The job installs + and precompiles the entire dependency closure before it runs a test; on a + cold cache that exceeds 15 minutes on the 1.11 legs, which were being + cancelled mid-install without ever measuring anything. + +==== Removed + +* `SQLite`, `URIs`, `Gumbo`, `Cascadia` and `DuckDB` from `[deps]`/`[compat]`. + No file under `src/`, `test/`, `benches/`, `examples/`, `ffi/` or `docs/` + referenced any of them; they were declared packages that every CI job paid + to download and precompile. Re-add each one with its feature. + diff --git a/EXPLAINME.adoc b/EXPLAINME.adoc index 61c9624..1f17d3d 100644 --- a/EXPLAINME.adoc +++ b/EXPLAINME.adoc @@ -220,9 +220,12 @@ queryable store. network graph shortest-path at varying graph sizes. | `Project.toml` -| Name `InvestigativeJournalism`, v0.1.0, MPL-2.0; deps include `DataFrames`, - `JSON3`, `SQLite`, `Dates`, `SHA`, `HTTP`, `URIs`, `TextAnalysis`, `Images`, - `Graphs`, `MetaGraphsNext`, `Gumbo`, `Cascadia`. +| Name `InvestigativeJournalism`, v0.1.0, MPL-2.0; deps: `DataFrames`, + `JSON3`, `Dates`, `SHA`, `Statistics`, `HTTP`, `TextAnalysis`, `Images`, + `Luxor` (CrazyWall rendering), `Graphs`, `MetaGraphsNext`, + `StringDistances`, `Cliodynamics`, `Causals`, `ZeroProb`, `VideoIO`, `DSP`, + `Wavelets`. Every entry is `using`d under `src/`, and every `using` under + `src/` has an entry — the manifest carries no unused packages. | `generated/` | Auto-generated content directory (story exports, evidence bundles). diff --git a/Project.toml b/Project.toml index dae55c9..21f79f4 100644 --- a/Project.toml +++ b/Project.toml @@ -4,22 +4,25 @@ authors = ["Jonathan D.A. Jewell "] version = "0.1.0" license = "MPL-2.0" +# Dependency hygiene (mirrors the sibling .jl packages' 2026-06-13/14 fix): +# every entry here is `using`d somewhere under src/, and every `using` under +# src/ has an entry here. `src/intelligence/string_board.jl` renders the +# CrazyWall through Luxor, so Luxor is a real runtime dependency and is +# declared below; SQLite, URIs, Gumbo, Cascadia and DuckDB were listed but +# referenced by no source, test or benchmark file, and only inflated the +# install/precompile step that the CI job must finish inside its timeout. [deps] DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" JSON3 = "0f8b85d8-7281-11e9-16c2-39a750bddbf1" -SQLite = "0aa819cd-b072-5ff4-a722-6bc24af294d9" Dates = "ade2ca70-3891-5945-98fb-dc099432e06a" SHA = "ea8e919c-243c-51af-8825-aaa63cd721ce" Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2" HTTP = "cd3eb016-35fb-5094-929b-558a96fad6f3" -URIs = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4" TextAnalysis = "a2db99b7-8b79-58f8-94bf-bbc811eef33d" Images = "916415d5-f1e6-5110-898d-aaa5f9f070e0" +Luxor = "ae8d54c2-7ccd-5906-9d76-62fc9837b5bc" Graphs = "86223c79-3864-5bf0-83f7-82e725a168b6" MetaGraphsNext = "fa8bd995-216d-47f1-8a91-f3b68fbeb377" -Gumbo = "708ec375-b3d6-5a57-a7ce-8257bf98657a" -Cascadia = "54eefc05-d75b-58de-a785-1a3403f0919f" -DuckDB = "d2f5444f-75bc-4fdf-ac35-56f514c445e1" StringDistances = "88034a9c-02f8-509d-84a9-84ec65e18404" Cliodynamics = "8d2f3e70-4c6b-5e9c-a3d1-2f8e9c0b1d2e" Causals = "c4a8b6d2-f9e3-4c1a-b8d7-9f2e3c4d5e6f" @@ -32,16 +35,12 @@ Wavelets = "29a6e085-ba6d-5f35-a997-948ac2efa89a" julia = "1.10" DataFrames = "1" JSON3 = "1" -SQLite = "1" HTTP = "1" -URIs = "1" TextAnalysis = "0.7" Images = "0.26" +Luxor = "3, 4" Graphs = "1" MetaGraphsNext = "0.7" -Gumbo = "0.8" -Cascadia = "1" -DuckDB = "1" StringDistances = "0.11" Cliodynamics = "1" Causals = "0.2" @@ -51,7 +50,8 @@ DSP = "0.7" Wavelets = "0.10" [extras] +Base64 = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40" [targets] -test = ["Test"] +test = ["Base64", "Test"] diff --git a/src/intelligence/forensics.jl b/src/intelligence/forensics.jl index 5973bf0..81424ea 100644 --- a/src/intelligence/forensics.jl +++ b/src/intelligence/forensics.jl @@ -8,6 +8,7 @@ module MediaForensics using Images +using Statistics using ..Types export verify_image_integrity, detect_ai_artifacts @@ -506,16 +507,4 @@ function _detect_texture_repetition(gray) return false end -# Bring in Statistics.var and Statistics.std if not already available -function var(x::AbstractVector{Float64}) - n = length(x) - n <= 1 && return 0.0 - m = sum(x) / n - return sum((xi - m)^2 for xi in x) / (n - 1) -end - -function std(x::AbstractVector{Float64}) - return sqrt(var(x)) -end - end # module diff --git a/src/storytelling/templates.jl b/src/storytelling/templates.jl index 3c44647..4c7419c 100644 --- a/src/storytelling/templates.jl +++ b/src/storytelling/templates.jl @@ -21,4 +21,25 @@ function build_story_structure(::Longform) ] end +function build_story_structure(::NewsBulletin) + return [ + "The Headline (The Finding)", + "The Lede (Why It Matters)", + "The Evidence (Documents & Data)", + "The Attribution (Who Confirms It)", + "The Response (Right of Reply)", + "The Development (What Happens Next)" + ] +end + +function build_story_structure(::Thread) + return [ + "The Hook Post (The Finding)", + "The Context Post (Background)", + "The Evidence Posts (Documents & Data)", + "The Rebuttal Post (Target Response)", + "The Close (Sources & Call to Action)" + ] +end + end # module diff --git a/test/runtests.jl b/test/runtests.jl index ccc4f69..888e344 100644 --- a/test/runtests.jl +++ b/test/runtests.jl @@ -1,5 +1,6 @@ # SPDX-License-Identifier: MPL-2.0 using Test +using Base64 using InvestigativeJournalism using Dates using DataFrames @@ -429,19 +430,50 @@ using DataFrames # MediaForensics # ----------------------------------------------------------------------- @testset "MediaForensics" begin + # Both analysers read a real file: they sniff magic bytes, walk JPEG + # markers / PNG chunks and (for detect_ai_artifacts) decode the image. + # The fixture below is a valid 8x8 RGB PNG (IHDR/IDAT/IEND with correct + # CRCs) embedded as base64, so the test needs no image *writer* and + # cannot drift with the installed codecs; it is written to a temp dir + # instead of assuming that /tmp/photo.jpg happens to exist. + fixturedir = mktempdir() + fixture_b64 = """ + iVBORw0KGgoAAAANSUhEUgAAAAgAAAAICAIAAABLbSncAAAA00lEQVR42gHIADf/AKVNyhglMLsdbRMs3tYjey7ZHj9yH8sZ + cQAXRJTWSTydXDRgvjEgHmn+2qDu6LmZf1wAfCmZ/a/lkyU81lSvTfrXFCegrrP+6SMvAIryIR+e5JHFsQvstVY7/B5vk0J+ + y8j+KQBV5c2ORtyO1LfCdk0qWk12dwb4XYaQAkoA1r2jQBvpyMvMyTX2zR9hImrhUziuGjQAAE0zug0kasBMgbG68j47+e71 + 958rSTSvhwD1UgtpuUsNmC6Fu1W2cqhyY3rNdGb8tg6f2V+Mn7fMSwAAAABJRU5ErkJggg== + """ + photo_path = joinpath(fixturedir, "photo.png") + suspect_path = joinpath(fixturedir, "suspect_image.png") + fixture_bytes = base64decode(replace(fixture_b64, r"\s" => "")) + write(photo_path, fixture_bytes) + write(suspect_path, fixture_bytes) + @testset "verify_image_integrity" begin - result = verify_image_integrity("/tmp/photo.jpg") + result = verify_image_integrity(photo_path) @test hasproperty(result, :has_metadata) || haskey(result, :has_metadata) @test hasproperty(result, :tamper_probability) || haskey(result, :tamper_probability) @test result.tamper_probability isa Float64 + @test 0.0 <= result.tamper_probability <= 1.0 + @test result.format == :png + @test result.findings isa Vector{String} end @testset "detect_ai_artifacts" begin - result = detect_ai_artifacts("/tmp/suspect_image.png") + result = detect_ai_artifacts(suspect_path) @test hasproperty(result, :is_synthetic_probability) @test hasproperty(result, :confidence) @test result.is_synthetic_probability isa Float64 @test result.confidence isa Float64 + @test 0.0 <= result.is_synthetic_probability <= 1.0 + @test 0.0 <= result.confidence <= 1.0 + @test result.indicators isa Vector{String} + end + + @testset "missing file is rejected, not scored" begin + absent = joinpath(fixturedir, "does_not_exist.png") + @test_throws ArgumentError verify_image_integrity(absent) + @test_throws ArgumentError detect_ai_artifacts(absent) end end From b2f174fbbd4ff10fc1f90eb8f7fb4a6fb083bf8d Mon Sep 17 00:00:00 2001 From: Arena Agent Date: Sun, 4 Oct 2026 01:16:10 +0000 Subject: [PATCH 2/2] chore(diag): temporary workflow that records CI output in the branch Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com> --- .github/workflows/diag.yml | 123 +++++++++++++++++++++++++++++++++++++ 1 file changed, 123 insertions(+) create mode 100644 .github/workflows/diag.yml diff --git a/.github/workflows/diag.yml b/.github/workflows/diag.yml new file mode 100644 index 0000000..4fe34ba --- /dev/null +++ b/.github/workflows/diag.yml @@ -0,0 +1,123 @@ +# SPDX-License-Identifier: MPL-2.0 +# TEMPORARY diagnostic workflow. It exists so that the investigation of the +# three red Julia checks (hyperpolymath/InvestigativeJournalism.jl#73) can be +# read from outside the runner: GitHub job logs are served from +# objects.githubusercontent.com, which is not reachable from the environment +# coordinating this work, so the workflow records its own output in the branch +# instead. Removed again before the pull request is opened for review. +name: CI Diagnostic (temporary) + +on: + push: + branches: ['arena/**'] + workflow_dispatch: + +permissions: + contents: write + +concurrency: + group: diag-${{ github.ref }} + cancel-in-progress: true + +jobs: + diagnose: + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + + - uses: julia-actions/setup-julia@fa02766e078afaaf09b14210362cee14137e6a32 # v3.0.2 + with: + version: '1.11' + + - uses: julia-actions/cache@a45e8fa8be21c18a06b7177052533149e61e9b38 # v3.1.0 + + - name: Install hyperpolymath-internal Julia deps from git + run: | + julia --project=. -e ' + using Pkg + Pkg.add([ + Pkg.PackageSpec(url="https://github.com/hyperpolymath/Cliodynamics.jl"), + Pkg.PackageSpec(url="https://github.com/hyperpolymath/Causals.jl"), + Pkg.PackageSpec(url="https://github.com/hyperpolymath/ZeroProb.jl"), + Pkg.PackageSpec(url="https://github.com/hyperpolymath/AcceleratorGate.jl"), + ]) + ' + + - name: Reproduce the undeclared-dependency failure mode + continue-on-error: true + run: | + set -x + { + echo "### minimal reproduction: using Luxor inside a package that does not declare it" + echo '```' + REPRO=$(mktemp -d) + mkdir -p "$REPRO/src" + printf 'name = "MinRepro"\nuuid = "11111111-2222-3333-4444-555555555555"\nversion = "0.1.0"\n' > "$REPRO/Project.toml" + printf 'module MinRepro\nusing Luxor\nend\n' > "$REPRO/src/MinRepro.jl" + ENVDIR=$(mktemp -d) + REPRO="$REPRO" ENVDIR="$ENVDIR" julia -e ' + using Pkg + Pkg.activate(ENV["ENVDIR"]) + Pkg.add("Luxor") + Pkg.develop(path=ENV["REPRO"]) + try + using MinRepro + println("UNEXPECTED: MinRepro loaded") + catch err + println("REPRO FAILURE: ", sprint(showerror, err)) + end + ' 2>&1 | tail -n 30 + echo '```' + echo + echo "### the same using-Luxor line in this repository" + echo '```' + grep -rn "using Luxor" src/ || true + echo '```' + } > /tmp/repro.txt 2>&1 || true + cat /tmp/repro.txt + + - name: Run the CI test command and capture the output + continue-on-error: true + run: | + set -x + { + echo "### julia --version" + julia --version + echo + echo "### using InvestigativeJournalism" + julia --project=. -e 'using InvestigativeJournalism; println("LOAD OK")' 2>&1 | tail -n 60 + echo + echo "### Pkg.status()" + julia --project=. -e 'using Pkg; Pkg.status()' 2>&1 | tail -n 80 + echo + echo "### Pkg.test() — same command as .github/workflows/ci.yml" + julia --project=. -e 'using Pkg; Pkg.Registry.add("General"); Pkg.Registry.add(Pkg.RegistrySpec(url="https://github.com/hyperpolymath/julia-professional-registry.git")); Pkg.instantiate(); Pkg.build(); Pkg.test()' 2>&1 + echo + echo "TEST COMMAND EXIT CODE: $?" + } > /tmp/ci-run.txt 2>&1 || true + { + echo "## CI diagnostic for $GITHUB_SHA" + echo + cat /tmp/repro.txt 2>/dev/null || true + echo + echo "### tail of the test run (600 lines)" + echo '```' + tail -n 600 /tmp/ci-run.txt 2>/dev/null || true + echo '```' + } > CI-DIAG.txt || true + wc -l CI-DIAG.txt || true + + - name: Commit the diagnostic output back to the branch + if: always() + run: | + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git add -f CI-DIAG.txt + if git diff --cached --quiet; then + echo "nothing to commit" + else + git commit -m "chore(diag): capture CI output for ${GITHUB_SHA} [skip ci]" + git push origin "HEAD:${GITHUB_REF_NAME}" + fi