Personal relationship management of your ego social network using plain-text Markdown files. Think of it like a Personal CRM on steroids. You'll need steroids and patience to create the Markdown files but they'll be yours forever.
This is a collection of templates with instructions and over time it will evolve. The approach relies heavily on a Personal Knowledge Management (PKM) tool like Obsidian but could work with any text editor.
Getting here has been a decades-long journey which you can read about in The Long and Winding Road.
This will take a long time to build out and require some attention to detail.
Here's the visualization of my social network in Obsidian. This is using a filter of the Markdown files that have tags: [person] in their frontmatter – the meta-data at the top of my note files. The colors represent which category I have the person in such as the bright green for my "A-listers" where I have the tags person and alist. Red is for flist, i.e., the people that I don't want to keep in touch with for various reasons.
You can use any text editor but preferably one that supports wikilinks, YAML frontmatter, and queries.
- Obsidian by fellow Canadians Erica Xu and Shida Li
- Silver Bullet which is Open-Source by Dutch Zef Hemel
I haven't checked if GitJournal by Vishesh Handa supports YAML frontmatter but it does support wikilinks so you'll still be able to navigate your notes.
As shown in the above image, Obsidian has a graph view (aka Map of Content aka MoC) which is a really fun way to visualize and navigate your social network. The local graph view is much more useful at an individual person level.
Visual Studio is also handy for bulk changes.
Simply use Obsidian and start creating files for each Person and optionally Organization and Place using the Templates provided. Include wikilinks in the body of the file in the form of [[name]] to "connect" the people, places, and organizations together as you go.
- Create a file for a person in your network
- Use the Person.md template
- Name the file
FirstName Lastname.md - Fill in as little or as much of the metadata on the person
- List the people they're connected to under
## Peopleusing[[FirstName LastName]] - List their positions under
## PositionsusingTitle, [[Organization Name]] - Click on each person under
## People - Have a sip of your favorite drink
- Go to Step 2
For each Organization under ## Organization fill in as little or as much information on the organization.
For each person under ## People add tags like #friend or #strong to track the strength of the ties between them.
Organize your notes as you wish. I like to have folders.
Attachments- for any files, images, photosOrganizations- put all the company profiles in herePeople- put all the people in here. Subfolder with theirslugand then dated files for each interactionPersonal- my personal notesTemplates- the files from Templates
For most people, I create a folder for them and a sub-folder media for a photo of them and any images or files we shared with each other.
My Helper Tools put images and files I've shared into those media subfolders. For people that I haven't communicated with, I stuff those in People\others
These are a set of templates to track your social network. Each contain a set of metadata at the top of the files also known as YAML frontmatter. If you're not technical, don't worry as Obsidian makes it easy to edit that information.
| File | For what | Notes |
|---|---|---|
| Call.md | A phone call | Do people still make these? |
| Chat.md | Instant messaging chat | e.g. LinkedIn, Signal, SMS |
| Organization.md | Schools and companies | Where a Person studie, volunteers, or works |
| Person.md | A person | The actual person! |
| Place.md | A physical place | Places people including you have been (e.g vacations, recommendations) |
| Post.md | Social media or blog post | Material post by a Person |
| Product.md | Product | A product worked on by a Person and/or Organization |
| Video.md | Videos | e.g. YouTube video by People |
A big part of this working well will be maintaining the frontmatter, you can sip things in over time like a new skill for a person or a new interest. You don't have to do it all at once. Just start.
In this example, you can see if you click in the skills field, Obsidian shows a list of skills other people have which makes it easy to be consistent across all people with that skill.
Each key template has a slug field which is a one-word or hyphenated word that uniquely identifies the Person, Place, Organization from others. It needs to be unique within each of the categories.
For example, Organization.md has people: [] in the frontmatter which could contain a comma separated list of Person slugs from individual Person.md files.
The Chat.md template also has a people field to list the people that were part of the conversation.
The Place.md template has a people field which you could use to list people that recommended the place. You won't need to put people that live or work there in this field since that information is already in their Person.md template.
This is the most important template of the collectoin and there are two pages describing the file:
-
The Person's head describes each of the fields in the Person.md template.
-
The Person's body describes the sections of the body of the Person.md template.
With standard Obsidian (no additional plugins), create a file for each month of the year and include embedded queries to show the birthdays and anniveraries that month. The sample query files are here and here's what they look like:
Which results in this (a bit ugly as you see the regex):
I've written some Python tools to convert the exports from various messaging apps to Markdown.
So far, I've created:
- linkedin_md for Linkedin chats
- signal_md for Signal messages using
signald - signal_sqlite_md for Signal messages from it's SQLite DB
- sms_backup_md for SMS messages
- 2024-03-10: last_contact to see when I last contacted the person
- 2024-03-10: md_birthdays outputs a month-by-month calendar of birthdays
- 2024-03-10: sample a sample collection of famous computer science folk
- 2024-09-22: comms to show the most recent communications with a person
- 2024-09-29: embed_notes to embed dated interaction files into Person profiles
- sync_person_files merges selected frontmatter fields, bios, and positions from another Person-file collection into a personal vault. It modifies matched personal files in place; begin with
--dry-runand keep the external--state-dir(including its backups and review decisions) backed up. - dedup_media interactively removes byte-identical files from every
mediafolder in a vault and updates their Markdown references.
Why? So I can get my conversations with people in my network into my own files that I can control and use directly with my social network data. Each of those tools rely on message_md.
This tool is meant to be used on the command line to lookup the most recent communications with a person.
By default the contents of last 3 dated message files are shown e.g. 2024-09-12.md.
The Markdown is converted to plain text.
For this tool you need to install a few libraries:
pip install markdown
pip install rich
pip install html2text-for--folder- The folder where each Person has a subfolder named with their slug-sor--slug- The slug of the person e.g. 'sponge-bob'-dor--debug- Debug messages-nor--name- The first name of the person -- NOT IMPLEMENTED-xor--max- The maximum number of interaction files to dump-mor--markdown- To display the Markdown instead of plain text-tor--time- Show the time e.g. SpongeBob at 23:31"-cor--color- Use ANSI colors, otherwise just black/white text
The tools/embed_notes.py script embeds all dated interaction files (e.g., 2023-02-01.md, 2024-03-24.md) into each Person's profile under the ## Notes section. This creates a complete communication history timeline within each person's file.
The script also ensures that each Person file has a proper H1 title (e.g., # John Doe) that matches the filename, adding it if missing.
DISCLAIMER!: Please, always do test runs of this script on a copy of your files somewhere other than your main folder/vault. Only once you're confident that it is not mangling your precious files, run it on the main folder/vault.
For example, if you have these files in a person's folder:
spongebob-squarepants/
├── Spongebob Squarepants.md # Person profile
├── 2023-02-01.md # Interaction file
└── 2024-03-24.md # Interaction file
The script will update Spongebob Squarepants.md to include:
# Spongebob Squarepants
## Bio
...
## Notes
- Some manual notes here
![[spongebob-squarepants/2023-02-01.md]]
![[spongebob-squarepants/2024-03-24.md]]-for--folder- The folder where each Person has a subfolder named with their slug-dor--debug- Print extra info as files are processed-xor--max- Maximum number of people to process
The tools/most_contacted.py script goes through every file dated YYYY-MM-DD.md and then shows you who you communicated with the most number of days, over how long, and when was the last contact. Kind of a fun leaderboard that I shared with my siblings.
By default, the results are displayed on the command line or you can use the -o option to generate a CSV file and then play with it in Excel.
DISCLAIMER: this script was entirely crafted by ChatGPT based on about 40 prompts I gave it. Not sure who owns the code now but alas my duty to disclose be done.
-mor--my-slug- the short code you use for yourself e.g.sponge-bob-nor--top-n- Show thenpeople you communicate with the most number of days-oor--output-csv- Generate to a CSV file instead of displaying the results
The tools/scan_wikilinks.py script walks a vault, indexes every file, resolves Obsidian wikilinks, and writes two reports in the vault root:
.wikilink_index.mdwith frontmatter for the last scan time plus vault counts and a full link mapmissing_files.mdwith one clickable line per broken wikilink so you can jump from Obsidian into a Jottacloud search
root- The vault root to scan-ior--include-extensions- Only index files with these extensions, e.g..md,.jpg,.pdf(default: all)-oor--output- Folder where the generated index and broken-link report should be written, relative to the current working directory unless you pass an absolute path-xor--max- Stop after scanning this many files when testing--index-name- Hidden index file name to write in the vault root--missing-name- Broken-link report file name to write in the vault root--source-extensions- Comma-separated list of file extensions to scan for wikilinks-dor--debug- Show one-line progress updates for the folder or file currently being scanned
The tools/dedup_media.py script recursively indexes files inside every folder named media below a vault root. It identifies byte-identical files by their byte count and SHA-256 hash, detects common media types from their file contents, and then asks which copy to keep. It does not delete any file until you select the copy to keep. Before prompting, it writes the structured media_dedup_index.json report to C:\data\dev-output on Windows or /mnt/c/data/dev-output in WSL by default. On a later run, choose u to quickly compare current paths, sizes, and modification times against the saved manifest. It reuses hashes for unchanged files and hashes only new or changed same-size candidates. It also reuses parsed Markdown references for notes whose modification times have not changed, and reads only new or changed notes. Choose r to discard the manifest and hash all same-size candidates again.
For each set of identical files, the script makes every displayed media filename a hyperlink that opens it in VS Code. Select an original file number to retain it in place. Further numbered options combine the location from one file with the filename from another: for two files, option 3 is location 1 with filename 2 and option 4 is location 2 with filename 1. Only combinations with an available destination are listed, and none are listed when every filename is an opaque hash, number, or automatic camera/download name. Enter c to provide a custom vault-relative path and descriptive filename; the first identical copy is moved there. The selected copy is moved when necessary and all Markdown references are updated. Enter s to skip the set or q at any prompt to quit.
If the selected kept file has no filename extension but its contents identify a supported media type, the script offers to add the appropriate extension and update Markdown references to that kept file. This rename happens only after you answer y.
After it finds sets of identical files, the script first indexes media filenames so it can safely resolve shorthand Obsidian links such as ![[photo.jpg]]. It then indexes Obsidian wikilinks and ordinary Markdown links in all .md files below the vault root once. Each removal then updates only Markdown files known to reference that media file. It prints every changed Markdown filename as a clickable link with its source line number, showing the old target in red and the replacement in green.
py -3 tools\dedup_media.py -f path\to\vault -o C:\data\dev-output-for--folder- The vault root to scan. The tool indexes media files only from folders namedmedia, but updates references in every.mdfile below this root.-oor--output- Folder wheremedia_dedup_index.jsonis written. Defaults toC:\data\dev-outputon Windows and/mnt/c/data/dev-outputin WSL.
This project is licensed under the MIT License - see the LICENSE file for details.