Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,11 @@ npm run typecheck --workspace=packages/app
npm run dist --workspace=packages/app # build + electron-builder (anche dist:win / dist:mac / dist:linux)

# CLI dopo la build del core
node packages/core/dist/cli.js load <path> [--filter-date A B] [--filter-area s,w,n,e] [--filter-activity X,Y] [--stats] [--export out.json|out.kml]
node packages/core/dist/cli.js load <path> [--filter-date A B] [--filter-area s,w,n,e] [--filter-activity X,Y] [--stats] [--quality] [--sanitize] [--export out.json|.kml|.geojson|.csv|.e7.json] [--export-format F]
```

## Struttura
- `packages/core/src/` — `loaders/` (un loader per formato Google + auto-detect), `filters.ts`, `stats.ts`, `transforms.ts`, `exporters/` (JSON, KML), `activity-mapping.ts`, `types.ts`, `cli.ts`, `index.ts`
- `packages/core/src/` — `loaders/` (un loader per formato Google + auto-detect), `filters.ts`, `stats.ts`, `quality.ts`, `transforms.ts`, `exporters/` (JSON, KML, GeoJSON, CSV, E7 + dispatcher `exportDataset`), `activity-mapping.ts`, `types.ts`, `cli.ts`, `index.ts`
- `packages/core/tests/` — test Vitest, con la stessa struttura di `src/`
- `packages/app/src/main/` — main process Electron (dataset in memoria, dialog, IPC)
- `packages/app/src/preload/` — espone `window.api` tipizzato via `contextBridge`
Expand Down Expand Up @@ -67,5 +67,5 @@ node packages/core/dist/cli.js load <path> [--filter-date A B] [--filter-area s,
- I filtri sono immutabili (restituiscono un nuovo dataset); `filterByActivityType` mantiene sempre le visite; il filtro date tiene le entry che si sovrappongono al range
- Packaging: l'app pacchettizzata non contiene `node_modules`. Il main include `mappit-core` nel bundle dai sorgenti (alias in `electron.vite.config.ts`) e electron-builder ha `npmRebuild: false`, perché altrimenti lancia `npm install --production` alla radice del monorepo e cancella le devDependencies
- Electron: `contextIsolation: true`, `nodeIntegration: false`, `sandbox: false` (necessario per il preload). Il main tiene `originalDataset` (mai modificato) e `currentDataset`
- Formato condiviso con il blog: il blog si aspetta `{ locations: [{ latitudeE7, longitudeE7, timestamp }] }`, mentre l'export di mappit produce `lat`/`lng` decimali. Oggi la conversione è manuale (vedi `PLAN.md` nella repo blog)
- Formato condiviso con il blog: il blog si aspetta `{ locations: [{ latitudeE7, longitudeE7, timestamp }] }`. Lo produce l'exporter E7 (`exportToE7`, CLI `--export route.e7.json`): punti grezzi + punti dei path delle activity, ordinati per tempo; è l'unico punto in cui mappit usa coordinate E7
- Prettier: virgolette singole, `printWidth` 100. TODO: `.prettierrc` dice `tabWidth: 2` ma il codice usa 4 spazi
12 changes: 7 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@

MappIt is a privacy-first toolkit for exploring your Google Location History locally:

- **Electron desktop app** with interactive map (deck.gl + MapLibre), timeline sidebar, activity filters, area search, and KML/JSON export
- **Electron desktop app** with interactive map (deck.gl + MapLibre), timeline sidebar, activity filters, area search, and KML/JSON/GeoJSON/CSV/E7 export
- **Node.js CLI** for headless processing, filtering, and exporting
- **Core library** (`mappit-core`) usable in your own scripts
- Auto-detects all Google Takeout location formats (Records.json, Timeline.json standard/semantic/iOS, monthly Semantic Location History files)
- Filters by date range, bounding box, or activity type
- Computes stats: distance per activity, monthly summaries, yearly breakdowns
- Exports to JSON and KML
- Exports to JSON, KML, GeoJSON, CSV and E7 locations (`latitudeE7`/`longitudeE7`)

</details>

Expand Down Expand Up @@ -134,6 +134,7 @@ npx mappit-core load ./data --stats
# Esportazione
npx mappit-core load ./data --filter-date 2024-01-01 2024-06-30 --export output.json
npx mappit-core load ./data --filter-date 2024-01-01 2024-06-30 --export output.kml
npx mappit-core load ./data --filter-date 2024-01-01 2024-06-30 --export route.e7.json # E7 per il blog
```

Tutte le opzioni:
Expand All @@ -152,7 +153,8 @@ Options:
-a, --filter-area Filtra per bounding box: south,west,north,east [string]
--filter-activity Filtra per tipo di attività (separati da virgola)
[string]
-e, --export Esporta su file (.json o .kml) [string]
-e, --export Esporta su file (.json, .kml, .geojson, .csv, .e7.json) [string]
--export-format Forza il formato (json, kml, geojson, csv, e7) [string]
-s, --stats Mostra statistiche di riepilogo
[boolean] [default: false]
```
Expand Down Expand Up @@ -194,7 +196,7 @@ App desktop per esplorare i dati di Google Location History. Usa `mappit-core` c
- Auto-detect del formato (tutti e 5 i formati supportati dal core)
- Visualizzazione summary: statistiche generali, distanza per attività, breakdown annuale
- Filtro per intervallo date con ricalcolo statistiche
- Export dataset filtrato in JSON o KML
- Export dataset filtrato in JSON, KML, GeoJSON, CSV o E7

### Architettura

Expand Down Expand Up @@ -234,7 +236,7 @@ dist/
- **Vite** v7 come bundler per il renderer
- **Vanilla TypeScript** per la UI (nessun framework)

> L'app include mappa interattiva (deck.gl + MapLibre), sidebar timeline, filtri attività, ricerca luoghi, area search, summary con grafici, e export KML/JSON.
> L'app include mappa interattiva (deck.gl + MapLibre), sidebar timeline, filtri attività, ricerca luoghi, area search, summary con grafici, e export KML/JSON/GeoJSON/CSV/E7.

### Distribuzione

Expand Down
13 changes: 4 additions & 9 deletions packages/app/src/main/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ import {
computeSummary,
computeYearlySummary,
computeMonthlySummary,
exportToJson,
exportToKml,
exportDataset,
exportFormatFromPath,
} from 'mappit-core';
import type { MappitDataset, DataSource, BoundingBox, PlaceVisit } from 'mappit-core';
import type { PlaceSearchHit } from '../shared/ipc-channels';
Expand Down Expand Up @@ -315,13 +315,8 @@ function registerIpcHandlers(): void {
throw new Error('No dataset loaded');
}

const ext = path.extname(args.filePath).toLowerCase();
const content =
ext === '.kml'
? exportToKml(currentDataset)
: exportToJson(currentDataset);

fs.writeFileSync(args.filePath, content, 'utf-8');
const format = exportFormatFromPath(args.filePath);
fs.writeFileSync(args.filePath, exportDataset(currentDataset, format), 'utf-8');
return true;
},
);
Expand Down
3 changes: 3 additions & 0 deletions packages/app/src/renderer/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,9 @@ async function handleExport(): Promise<void> {
defaultName: 'mappit-export.kml',
filters: [
{ name: 'KML files', extensions: ['kml'] },
{ name: 'GeoJSON files', extensions: ['geojson'] },
{ name: 'CSV files', extensions: ['csv'] },
{ name: 'E7 locations (travel blog)', extensions: ['e7.json'] },
{ name: 'JSON files', extensions: ['json'] },
{ name: 'All files', extensions: ['*'] },
],
Expand Down
33 changes: 18 additions & 15 deletions packages/core/src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
* mappit-core load <path> --filter-activity WALKING,CYCLING
* mappit-core load <path> --export output.json
* mappit-core load <path> --export output.kml
* mappit-core load <path> --export output.geojson # also .csv
* mappit-core load <path> --export route.e7.json # E7 locations (travel blog)
* mappit-core load <path> --export out.json --export-format e7
* mappit-core load <path> --stats
* mappit-core load <path> --quality # validation + quality metrics
* mappit-core load <path> --sanitize --export clean.json # drop invalid items
Expand All @@ -22,7 +25,7 @@
import ora from 'ora';

import { VERSION } from './index';
import { detectFormat, parseAuto } from './loaders/auto-detect';

Check warning on line 28 in packages/core/src/cli.ts

View workflow job for this annotation

GitHub Actions / Lint

'detectFormat' is defined but never used
import { parseRecords } from './loaders/records';
import { parseTimelineStandard } from './loaders/timeline-standard';
import { parseTimelineSemantic } from './loaders/timeline-semantic';
Expand All @@ -32,8 +35,8 @@
import type { BoundingBox } from './filters';
import { computeSummary, computeYearlySummary } from './stats';
import { validateDataset, sanitizeDataset, computeQualityMetrics } from './quality';
import { exportToJson } from './exporters/json-exporter';
import { exportToKml } from './exporters/kml-exporter';
import { exportDataset, exportFormatFromPath, EXPORT_FORMATS } from './exporters';
import type { ExportFormat } from './exporters';
import type { MappitDataset, DataSource } from './types';

// ---------------------------------------------------------------------------
Expand Down Expand Up @@ -326,7 +329,13 @@
})
.option('export', {
alias: 'e',
describe: 'Export to file (.json or .kml)',
describe:
'Export to file; format from the extension: .json, .kml, .geojson, .csv, .e7.json (E7 locations)',
type: 'string',
})
.option('export-format', {
describe: 'Force the export format instead of using the file extension',
choices: EXPORT_FORMATS,
type: 'string',
})
.option('stats', {
Expand Down Expand Up @@ -394,18 +403,12 @@

// 4 — Export
if (cmdArgv.export) {
spinner.start(`Exporting to ${cmdArgv.export}…`);
const ext = path.extname(cmdArgv.export).toLowerCase();
let content: string;

if (ext === '.kml') {
content = exportToKml(ds);
} else {
content = exportToJson(ds);
}

fs.writeFileSync(cmdArgv.export, content, 'utf-8');
spinner.succeed(`Written to ${cmdArgv.export}`);
const format =
(cmdArgv.exportFormat as ExportFormat | undefined) ??
exportFormatFromPath(cmdArgv.export);
spinner.start(`Exporting ${format} to ${cmdArgv.export}…`);
fs.writeFileSync(cmdArgv.export, exportDataset(ds, format), 'utf-8');
spinner.succeed(`Written ${format} to ${cmdArgv.export}`);
}
} catch (err: unknown) {
const msg = err instanceof Error ? err.message : String(err);
Expand Down
106 changes: 106 additions & 0 deletions packages/core/src/exporters/csv-exporter.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
/**
* Export a MappitDataset to CSV, one row per location.
*
* Rows, in dataset order:
* - `point`: a raw point
* - `visit`: a visit (`timestamp` = start, `end_time` = end)
* - `path`: a GPS point of an activity path (`timestamp` falls back to the
* activity start when the point has none)
* - `activity`: for activities without a path, two rows with the start and
* end location (at `startTime` and `endTime`), so no activity is lost
*
* Coordinates are decimal degrees. Values are quoted per RFC 4180 when needed.
*/

import type { MappitDataset } from '../types';

/** Options for {@link exportToCsv}. */
export interface CsvExportOptions {
/** Field separator (default `,`). */
delimiter?: string;
/** Line ending (default `\n`). */
newline?: string;
}

/** Column order of the CSV output. */
export const CSV_COLUMNS = [
'kind',
'timestamp',
'end_time',
'lat',
'lng',
'activity_type',
'accuracy',
'name',
'place_id',
] as const;

type Row = Partial<Record<(typeof CSV_COLUMNS)[number], string | number | undefined>>;

function cell(value: string | number | undefined, delimiter: string): string {
if (value === undefined || value === null) return '';
const s = String(value);
return s.includes(delimiter) || /["\r\n]/.test(s) ? `"${s.replace(/"/g, '""')}"` : s;
}

/**
* Serialise a dataset as CSV text, with a header row.
*/
export function exportToCsv(dataset: MappitDataset, options: CsvExportOptions = {}): string {
const { delimiter = ',', newline = '\n' } = options;
const rows: Row[] = [];

for (const p of dataset.points) {
rows.push({
kind: 'point',
timestamp: p.timestamp,
lat: p.lat,
lng: p.lng,
activity_type: p.activityType,
accuracy: p.accuracy,
});
}

for (const e of dataset.timeline) {
if (e.type === 'visit') {
rows.push({
kind: 'visit',
timestamp: e.startTime,
end_time: e.endTime,
lat: e.lat,
lng: e.lng,
name: e.name,
place_id: e.placeId,
});
} else if (e.path.length === 0) {
for (const [loc, t] of [
[e.startLocation, e.startTime],
[e.endLocation, e.endTime],
] as const) {
rows.push({
kind: 'activity',
timestamp: t,
lat: loc.lat,
lng: loc.lng,
activity_type: e.activityType,
});
}
} else {
for (const p of e.path) {
rows.push({
kind: 'path',
timestamp: p.timestamp ?? e.startTime,
lat: p.lat,
lng: p.lng,
activity_type: e.activityType,
});
}
}
Comment on lines +88 to +98
}

const lines = [CSV_COLUMNS.join(delimiter)];
for (const r of rows) {
lines.push(CSV_COLUMNS.map((c) => cell(r[c], delimiter)).join(delimiter));
}
return lines.join(newline) + newline;
}
70 changes: 70 additions & 0 deletions packages/core/src/exporters/e7-exporter.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
/**
* Export a MappitDataset to the E7 "locations" JSON used by the travel blog
* (and by the legacy Google Records.json):
*
* ```json
* { "locations": [{ "latitudeE7": 454642000, "longitudeE7": 91900000, "timestamp": "…" }] }
* ```
*
* E7 = decimal degrees × 10^7, rounded to an integer. This is the only place
* where mappit produces E7 coordinates: the internal model stays decimal.
*
* Locations come from raw points and from activity GPS paths (a path point
* without a timestamp takes the activity start). Visits are excluded by
* default, matching the blog's route extraction. Output is sorted by time.
*/

import type { MappitDataset } from '../types';

/** Options for {@link exportToE7}. */
export interface E7ExportOptions {
/** Pretty-print with indentation (default `2`). Set `0` for compact. */
indent?: number;
/** Also emit one location per visit, at its start time (default `false`). */
includeVisits?: boolean;
}

/** One location in E7 format. */
export interface E7Location {
latitudeE7: number;
longitudeE7: number;
timestamp: string;
}

const toE7 = (deg: number): number => Math.round(deg * 1e7);

/**
* Build the `{ locations }` object from a dataset.
*/
export function toE7Locations(
dataset: MappitDataset,
options: Omit<E7ExportOptions, 'indent'> = {},
): { locations: E7Location[] } {
const { includeVisits = false } = options;
const locations: E7Location[] = [];
const push = (lat: number, lng: number, timestamp: string) =>
locations.push({ latitudeE7: toE7(lat), longitudeE7: toE7(lng), timestamp });

for (const p of dataset.points) push(p.lat, p.lng, p.timestamp);

for (const e of dataset.timeline) {
if (e.type === 'activity') {
for (const p of e.path) push(p.lat, p.lng, p.timestamp ?? e.startTime);
} else if (includeVisits) {
push(e.lat, e.lng, e.startTime);
}
}

// Stable sort by instant (handles mixed UTC offsets)
const indexed = locations.map((l, i) => ({ l, i, t: Date.parse(l.timestamp) }));
indexed.sort((a, b) => (a.t || 0) - (b.t || 0) || a.i - b.i);
return { locations: indexed.map((x) => x.l) };
}

/**
* Serialise a dataset as E7 locations JSON.
*/
export function exportToE7(dataset: MappitDataset, options: E7ExportOptions = {}): string {
const { indent = 2, ...rest } = options;
return JSON.stringify(toE7Locations(dataset, rest), null, indent);
}
Loading
Loading