Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

## Unreleased

- Genetic map download, 2026-11-15 (K653, owner's N124 A). A Genetic map card on the Kits screen offers the HapMap-derived GRCh37 map (about 24 MB) from the Beagle site at the University of Washington. It says where the map comes from and that no license is stated, downloads only after a yes, checks the pinned size and SHA-256, and can remove the map again. Comparisons use it when both raw files are on GRCh37; the demo always uses the approximate rule. Tested with made-up archives, and once by hand with the real archive (chromosome 1 comes out at 286 cM). The card hasn't been clicked in the built app.

- The cross-kit check (S7) is enforced, 2026-11-15 (K655). Phasing a kit with a parent now stops unless the two share parent-child cM, and rebuilding someone also needs the children to share full-sibling cM. `fittingRelationships` and `RELATIONSHIP_WORDS` say which relationships a cM fits, and the Kits screen lists them for each pair compared on the Mac. Tested on simulated kits; the screen text hasn't been seen in the built app.

- The app carries its own Node, 2026-11-15 (K338). `pnpm fetch:node` downloads Node v26.13.0 from nodejs.org, checks the pinned SHA-256 and puts it where Tauri bundles it, so the Mac app no longer needs Node installed. A release build never falls back to another Node on the Mac. Also fixed two faults in the packaged helper that kept a built app from opening a library: its shared code was split into chunk files the app didn't include, and the build dropped the `node:` prefix from `node:sqlite`. Checked by building the app and running its helper with no PATH; the app window wasn't clicked through.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,15 @@ DNA-first genetic genealogy for Mac. OneFamily groups your DNA matches before yo
- a simulator of made-up families with planted answers
- importers for Genealogy Assistant-style match lists you export yourself (allowed by the owner's reading of the terms, pending a legal review), and for raw DNA in five companies' layouts
- match grouping from the DNA alone (ADR 0006)
- kit-to-kit comparison, triangulation and segment painting
- kit-to-kit comparison, triangulation and segment painting, with an approximate map or the GRCh37 map the app downloads after a yes
- sibling phasing, half-kits and rebuilt relatives (made-up data only)
- the hypothesis engine with odds (ADR 0008)
- the library with checked, encrypted backups
- one universal ID per person, and proposed cross-service links
- OneFamily for Mac, v0: runs on a made-up demo family. It's been built and opened locally, not released. It carries its own copy of Node, so nothing extra needs installing (ADR 0007).
- the Chrome extension the app collects with (`apps/extension`, ADR 0009). It stores nothing, talks to the app only through Chrome's native messaging, and asks for your OK in the app. It only works with a local mock site: every DNA service's terms say no to automated page visits. It's been tested with Vitest against the mock site, and hasn't been loaded into Chrome yet.

270 tests pass locally. The raw layouts haven't been checked against a real downloaded file. The register's verdicts are a careful reading of public pages, not legal advice, and a legal review is still to come.
275 tests pass locally. The raw layouts haven't been checked against a real downloaded file. The register's verdicts are a careful reading of public pages, not legal advice, and a legal review is still to come.

## What it will do

Expand Down
144 changes: 144 additions & 0 deletions apps/desktop/host/genetic-map.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
/**
* The GRCh37 genetic map, downloaded onto this Mac when the person asks (owner, 2026-11-11, N124 A; K653).
*
* OneFamily doesn't bundle it: no license is stated where it's published. The app names the source, downloads
* the file from there only after a yes, checks it against the size and SHA-256 pinned here, and keeps the
* autosomes and X as one PLINK file in the app's own folder, with a note saying where it came from. Without
* it, comparisons use the approximate 1 cM per Mb rule and say so.
*/
import { createHash } from 'node:crypto'
import { existsSync, mkdirSync, readFileSync, renameSync, rmSync, writeFileSync } from 'node:fs'
import { join } from 'node:path'
import { inflateRawSync } from 'node:zlib'
import { parsePlinkMap, type GeneticMap } from '@extantape/onefamily-core'

export const GRCH37_MAP = {
name: 'HapMap Phase II, GRCh37 (Beagle PLINK files)',
url: 'https://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh37.map.zip',
bytes: 23_913_120,
sha256: '8e88141345844571bc63efac6c043ee29e8ca41f26087dc51b7f1918943a077b',
publisher: 'the Beagle genetic maps page at the University of Washington',
madeBy: 'The HapMap Phase II map (2007), lifted to GRCh37 by Adam Auton (2010), in PLINK format by Brian Browning.',
license: 'No license is stated on the download page.',
} as const

const CHROMOSOMES = [...Array.from({ length: 22 }, (_, i) => String(i + 1)), 'X']

export class MapError extends Error {}

const dir = (stateDir: string) => join(stateDir, 'genetic-maps', 'GRCh37')
const mapFile = (stateDir: string) => join(dir(stateDir), 'plink.GRCh37.map')
const noteFile = (stateDir: string) => join(dir(stateDir), 'SOURCE.md')

/** The files in a zip archive, by name. Only stored and deflated entries, which is all this archive uses. */
export function readZip(buf: Buffer): Map<string, Buffer> {
let end = -1
for (let i = buf.length - 22; i >= Math.max(0, buf.length - 65_557); i--) if (buf.readUInt32LE(i) === 0x06054b50) { end = i; break }
if (end < 0) throw new MapError('The download isn’t a zip archive.')
const count = buf.readUInt16LE(end + 10)
let p = buf.readUInt32LE(end + 16)
const files = new Map<string, Buffer>()
for (let n = 0; n < count; n++) {
if (buf.readUInt32LE(p) !== 0x02014b50) throw new MapError('The zip archive’s directory is damaged.')
const method = buf.readUInt16LE(p + 10)
const packed = buf.readUInt32LE(p + 20)
const size = buf.readUInt32LE(p + 24)
const nameLen = buf.readUInt16LE(p + 28)
const extraLen = buf.readUInt16LE(p + 30)
const commentLen = buf.readUInt16LE(p + 32)
const local = buf.readUInt32LE(p + 42)
const name = buf.subarray(p + 46, p + 46 + nameLen).toString('utf8')
const start = local + 30 + buf.readUInt16LE(local + 26) + buf.readUInt16LE(local + 28)
const data = buf.subarray(start, start + packed)
if (method === 0) files.set(name, Buffer.from(data))
else if (method === 8) files.set(name, inflateRawSync(data))
else throw new MapError(`${name} uses a compression this app doesn’t read.`)
if (files.get(name)!.length !== size) throw new MapError(`${name} didn’t unpack to its stated size.`)
p += 46 + nameLen + extraLen + commentLen
}
return files
}

/** Checks a downloaded archive and installs the map. Nothing is written unless every check passes. */
export function installMap(stateDir: string, archive: Uint8Array, today: string, expected: { bytes: number; sha256: string } = GRCH37_MAP): void {
const buf = Buffer.from(archive)
if (buf.length !== expected.bytes) throw new MapError(`The download is ${buf.length.toLocaleString()} bytes, not the expected ${expected.bytes.toLocaleString()}. Nothing was installed.`)
const sha = createHash('sha256').update(buf).digest('hex')
if (sha !== expected.sha256) throw new MapError('The download doesn’t match the checksum this app expects. Nothing was installed.')
const files = readZip(buf)
const parts = CHROMOSOMES.map((c) => {
const f = files.get(`plink.chr${c}.GRCh37.map`)
if (!f) throw new MapError(`The archive has no map for chromosome ${c}. Nothing was installed.`)
return f.toString('utf8').trimEnd()
})
const text = parts.join('\n') + '\n'
parsePlinkMap(GRCH37_MAP.name, text) // refuses a malformed map
mkdirSync(dir(stateDir), { recursive: true })
const tmp = `${mapFile(stateDir)}.partial`
writeFileSync(tmp, text)
writeFileSync(
noteFile(stateDir),
[
'# Genetic map (GRCh37)',
'',
`Downloaded on ${today} from ${GRCH37_MAP.publisher}, at the person's request:`,
GRCH37_MAP.url,
'',
GRCH37_MAP.madeBy,
GRCH37_MAP.license,
'',
`Archive SHA-256: ${sha}`,
'Kept here: chromosomes 1 to 22 and X, joined into plink.GRCh37.map. Delete this folder to go back to the approximate rule.',
'',
].join('\n'),
)
renameSync(tmp, mapFile(stateDir))
}

export type MapStatus = {
installed: boolean
name: string
publisher: string
url: string
megabytes: number
madeBy: string
license: string
installedOn: string | null
}

export function mapStatus(stateDir: string): MapStatus {
const installed = existsSync(mapFile(stateDir))
const note = installed && existsSync(noteFile(stateDir)) ? readFileSync(noteFile(stateDir), 'utf8') : ''
return {
installed,
name: GRCH37_MAP.name,
publisher: GRCH37_MAP.publisher,
url: GRCH37_MAP.url,
megabytes: Math.round(GRCH37_MAP.bytes / 1_000_000),
madeBy: GRCH37_MAP.madeBy,
license: GRCH37_MAP.license,
installedOn: /Downloaded on (\d{4}-\d{2}-\d{2})/.exec(note)?.[1] ?? null,
}
}

let cached: { file: string; map: GeneticMap } | null = null

/** The installed map, or null. */
export function loadMap(stateDir: string): GeneticMap | null {
const file = mapFile(stateDir)
if (!existsSync(file)) return null
if (cached?.file !== file) cached = { file, map: parsePlinkMap(GRCH37_MAP.name, readFileSync(file, 'utf8')) }
return cached.map
}

export function removeMap(stateDir: string): void {
rmSync(dir(stateDir), { recursive: true, force: true })
cached = null
}

/** Downloads the archive from its publisher. */
export async function downloadArchive(fetchImpl: typeof fetch = fetch): Promise<Uint8Array> {
const res = await fetchImpl(GRCH37_MAP.url)
if (!res.ok) throw new MapError(`The map’s publisher answered ${res.status}. Nothing was installed.`)
return new Uint8Array(await res.arrayBuffer())
}
50 changes: 45 additions & 5 deletions apps/desktop/host/handlers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {
import type { PedigreeDocument } from '@extantape/onepedigree-core'
import { withCandidateFathers, type TreePerson } from '../src/pedigree'
import { createDemoLibrary, DEMO_NOTE } from './demo'
import { downloadArchive, installMap, loadMap, mapStatus, removeMap } from './genetic-map'
import { createExtensionBridge, type ExtensionData } from './extension/bridge'
import { extensionRegister } from './extension/terms'
import { PUBLISHED_EXTENSION_ID } from './extension/wire'
Expand Down Expand Up @@ -53,6 +54,9 @@ export function createHost(opts: {
now?: () => Date
/** The real services' terms files (name → text) for the extension's register. Without them every real service is refused. */
termsFiles?: Readonly<Record<string, string>>
/** Where the genetic map comes from. Tests pass a made-up archive and its own size and checksum. */
downloadMap?: () => Promise<Uint8Array>
mapExpected?: { bytes: number; sha256: string }
}) {
const keys = opts.keys ?? noKeyStore
const now = opts.now ?? (() => new Date())
Expand All @@ -66,6 +70,17 @@ export function createHost(opts: {
return s.current
}

/**
* The map for comparing two kits: the downloaded GRCh37 map when it's installed and both raw files are on
* GRCh37, otherwise the approximate rule. Ancestry and FamilyTreeDNA files don't state a build; both are
* GRCh37. The demo's positions are made up, so it always uses the approximate rule.
*/
const mapFor = (lib: string, ...kits: LoadedKit[]) => {
if (existsSync(join(lib, DEMO_NOTE))) return ONE_CM_PER_MB
const on37 = (k: LoadedKit) => k.raw?.details.build === '37' || (k.raw?.details.build == null && ['ancestry', 'ftdna'].includes(k.raw?.details.service ?? ''))
return (kits.every(on37) && loadMap(opts.stateDir)) || ONE_CM_PER_MB
}

// --- Library and decisions

const decisions = (lib: string) => readDecisions(decisionsPath(lib))
Expand Down Expand Up @@ -429,6 +444,29 @@ export function createHost(opts: {
return status()
},

// --- Genetic map (N124 A, K653)

async geneticMap() {
return mapStatus(opts.stateDir)
},

/** Downloads the GRCh37 map from its publisher. Only after a yes: `confirm` must be true. */
async downloadGeneticMap({ confirm }) {
if (confirm !== true) throw new HostError('Downloading the genetic map needs a yes first.')
const archive = await (opts.downloadMap ?? downloadArchive)()
try {
installMap(opts.stateDir, archive, now().toISOString().slice(0, 10), opts.mapExpected)
} catch (e) {
throw new HostError((e as Error).message)
}
return mapStatus(opts.stateDir)
},

async removeGeneticMap() {
removeMap(opts.stateDir)
return mapStatus(opts.stateDir)
},

// --- Screens

async kits(): Promise<KitsResult> {
Expand All @@ -454,7 +492,7 @@ export function createHost(opts: {
const a = data.kits[i]!
const b = data.kits[j]!
if (!a.raw || !b.raw) continue
const c = compareKits(a.raw.calls, b.raw.calls, { map: ONE_CM_PER_MB, errorTolerance: 50 })
const c = compareKits(a.raw.calls, b.raw.calls, { map: mapFor(lib, a, b), errorTolerance: 50 })
comparisons.push({ a: a.result.kit, b: b.result.kit, totalCm: Math.round(c.totalCm), segments: c.segments.length, longestCm: Math.round(c.longestCm), map: c.map, fits: fittingRelationships(c.totalCm).map((r) => RELATIONSHIP_WORDS[r]) })
}
return { kits, comparisons }
Expand Down Expand Up @@ -487,18 +525,20 @@ export function createHost(opts: {
},

async segments({ a, b }) {
const data = await load(current())
const lib = current()
const data = await load(lib)
const ka = data.kits.find((k) => k.result.kit === a)
const kb = data.kits.find((k) => k.result.kit === b)
if (!ka || !kb) throw new HostError('That kit isn’t in this library.')
if (!ka.raw || !kb.raw) throw new HostError('Both kits need a raw DNA file to be compared.')
const c = compareKits(ka.raw.calls, kb.raw.calls, { map: ONE_CM_PER_MB, errorTolerance: 50 })
const map = mapFor(lib, ka, kb)
const c = compareKits(ka.raw.calls, kb.raw.calls, { map, errorTolerance: 50 })
const chromosomes = [...ka.raw.calls.keys()].filter((chr) => /^\d+$/.test(chr) && kb.raw!.calls.has(chr)).map((chr) => {
const pos = ka.raw!.calls.get(chr)!.position
return {
chromosome: chr,
startCm: ONE_CM_PER_MB.toCm(chr, Number(pos[0] ?? 0)),
endCm: ONE_CM_PER_MB.toCm(chr, Number(pos[pos.length - 1] ?? 0)),
startCm: map.toCm(chr, Number(pos[0] ?? 0)),
endCm: map.toCm(chr, Number(pos[pos.length - 1] ?? 0)),
segments: c.segments.filter((s) => s.chromosome === chr).map((s) => ({ startCm: s.startCm, endCm: s.endCm, cm: s.cm, markers: s.markers })),
}
})
Expand Down
7 changes: 7 additions & 0 deletions apps/desktop/host/protocol.ts
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,8 @@ export type GedmatchStatus = {
kits: { kit: string; name: string; hasRaw: boolean; gedmatchId: string | null; setting: 'opt-in' | 'opt-out' | 'research' | 'private' | null; minor: boolean; recordedAt: string | null; warnings: string[]; recommended: boolean }[]
}

export type MapStatus = { installed: boolean; name: string; publisher: string; url: string; megabytes: number; madeBy: string; license: string; installedOn: string | null }

export type HostApi = {
status: { params: Record<string, never>; result: Status }
chooseLibrary: { params: { path: string }; result: Status }
Expand Down Expand Up @@ -188,5 +190,10 @@ export type HostApi = {
/** Trust a developer build of the extension, by its ID. Recorded in the decisions log; `confirm` must be true. */
trustDeveloperExtension: { params: { id: string; confirm: boolean }; result: ExtensionView }
untrustDeveloperExtension: { params: { id: string }; result: ExtensionView }
/** The GRCh37 genetic map: whether it's on this Mac, and where it comes from. */
geneticMap: { params: Record<string, never>; result: MapStatus }
/** Download it from its publisher, checked against a pinned checksum. `confirm` must be true. */
downloadGeneticMap: { params: { confirm: boolean }; result: MapStatus }
removeGeneticMap: { params: Record<string, never>; result: MapStatus }
}
export type HostMethod = keyof HostApi
84 changes: 84 additions & 0 deletions apps/desktop/src/screens/GeneticMap.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
import { useEffect, useState } from 'react'
import type { MapStatus } from '../../host/protocol'
import { call } from '../api'

/**
* The GRCh37 genetic map card (owner, 2026-11-11, N124 A). Names the source before anything is downloaded,
* downloads only after a yes, and can remove the map again.
*/
export function GeneticMapCard({ onChange }: { onChange?: () => void }) {
const [status, setStatus] = useState<MapStatus | null>(null)
const [confirming, setConfirming] = useState(false)
const [busy, setBusy] = useState(false)
const [error, setError] = useState<string | null>(null)

useEffect(() => {
call('geneticMap').then(setStatus, (e: Error) => setError(e.message))
}, [])

const run = async (f: () => Promise<MapStatus>) => {
setBusy(true)
setError(null)
try {
setStatus(await f())
onChange?.()
} catch (e) {
setError((e as Error).message)
} finally {
setBusy(false)
setConfirming(false)
}
}

if (!status) return error ? <p role="alert">{error}</p> : null
return (
<section className="ol-card" aria-labelledby="genetic-map">
<h2 id="genetic-map" style={{ marginTop: 0 }}>
Genetic map
</h2>
{status.installed ? (
<>
<p>
Using the {status.name} map, downloaded {status.installedOn ? `on ${status.installedOn} ` : ''}from {status.publisher}. Kits whose raw files are on
GRCh37 are measured with it. The made-up demo always uses the approximate rule, because its positions are invented.
</p>
<button type="button" disabled={busy} onClick={() => void run(() => call('removeGeneticMap'))}>
Remove the map
</button>
</>
) : (
<>
<p>
Comparisons use an approximate rule of 1 cM per million base pairs. A published genetic map measures segments more accurately. OneFamily doesn’t
include one, but it can download the usual GRCh37 map onto this Mac for you.
</p>
{!confirming ? (
<button type="button" disabled={busy} onClick={() => setConfirming(true)}>
Download the map…
</button>
) : (
<div className="ol-confirm" role="group" aria-label="Confirm the map download">
<p>
This downloads about {status.megabytes} MB from {status.publisher}:
<br />
<code style={{ wordBreak: 'break-all' }}>{status.url}</code>
</p>
<p className="ol-hint">
{status.madeBy} {status.license} The file is checked against a fixed checksum before it’s used, and nothing about you or your DNA is sent.
</p>
<div className="ol-row">
<button type="button" className="ol-primary" disabled={busy} onClick={() => void run(() => call('downloadGeneticMap', { confirm: true }))}>
{busy ? 'Downloading…' : 'Yes, download it'}
</button>
<button type="button" autoFocus disabled={busy} onClick={() => setConfirming(false)}>
Not now
</button>
</div>
</div>
)}
</>
)}
{error && <p role="alert">{error}</p>}
</section>
)
}
Loading
Loading