Held-out LLM evaluation drafts (onboarding plan WP-10a)

bench/eval/llm-heldout/01-old-maps.md

@@ -0,0 +1,17 @@
+# An Old Map from the Attic
+
+I found it wedged behind a loose board in my aunt's attic, rolled tightly and held with a rubber band that crumbled to dust when I touched it. The map itself was hand-drawn on heavy paper that had yellowed to the color of old cream, and the ink had faded to pale blue and brown. It showed the roads around the town where my grandmother had grown up, with penciled notes in the margins and small crosses marking what I assumed were important places.
+
+The precision of it struck me first. Someone had spent hours on this, measuring distances and drawing the roads to actual scale. Not a printed map with its standardized symbols, but someone's deliberate effort to capture this particular landscape. I could see the hesitation in some of the lines where they'd gone over them multiple times, trying to get something right. The lettering was careful, practiced. This was work that mattered to whoever had made it.
+
+The roads don't match the current maps. Some are gone entirely. The state highway that runs through the center now doesn't appear, and I had to cross-reference with satellite imagery to understand what landscape this represented. According to the dates scribbled in the corner, it was from 1952, which would make it older than my grandmother's earliest child. Perhaps she had drawn it herself, or someone had drawn it for her. There was no name.
+
+What struck me most was what the mapmaker had chosen to mark. Roads, certainly, but also wells, an old schoolhouse, what looked like a railroad crossing that's long been removed, and several houses marked with small squares. One house had a circle around it and a note: "Gram's?" Another had a line drawn through it with "gone" written beside. These weren't administrative boundaries or population centers. These were the features that mattered to the person making the map. This was how they understood the place.
+
+I started to recognize some of the names. The corners and crossroads had the family surnames of people I'd heard about my whole life but never met. There was a place marked "Miller farm," another "Orchard." Someone had drawn a small tree symbol near what would now be the edge of town. The effort to represent a landscape so personally, so selectively, revealed something about what that place had meant.
+
+Studying the map, I realized how much information we lose when we discard these personal records. The printed maps of that era would show roads and town names, but they would never show which houses people considered significant, or which buildings had already vanished even then. They wouldn't show the mapmaker's uncertainty—those crossed-out roads, the question mark beside a place name. They wouldn't reveal that someone's home life was worth the careful work of documentation.
+
+I've copied it since then, photographed it carefully and stored the original in a climate-controlled folder. Sometimes I look at it and trace the roads with my finger, trying to imagine driving them as they existed in 1952. I wonder who the mapmaker was, and whether they knew that decades later, someone they'd never meet would hold their work and feel, suddenly and unexpectedly, connected to a place and a time they'd never experienced.
+
+The map still raises questions I can't answer. But that uncertainty is part of what makes it valuable. It's a direct connection to someone's effort and attention, preserved through decades. And in a time when maps are generated instantly and forgotten just as quickly, that deliberate, careful work feels like an act of love.

bench/eval/llm-heldout/02-stone-walls.md

@@ -0,0 +1,21 @@
+# Why Stone Walls Matter
+
+In old farm country, the stone walls are more than just history. They're a form of language—the way landowners, generation after generation, have described property, effort, and persistence. Every wall tells a story about what someone valued enough to spend their labor on, and what they had to clear away to do it.
+
+When European settlers first came to this region, they encountered a landscape that was not, as they might have expected, an open plain. The forests were dense and old, and when farmers wanted land to plant crops, they had to clear trees first. But underneath all that, once the trees were gone, was something unexpected: stones. Everywhere. More stones than earth. A field that looked like it might be farmable from a distance revealed itself as essentially a quarry with soil in the cracks. 
+
+Rather than haul the stones elsewhere—an impossible task given how many there were—settlers developed a practical solution. They built walls. These walls served practical purposes: they marked boundaries, they created pastures where you could keep animals without them wandering into the neighbors' land, they formalized the separation between farm and wild forest. But they were also by-products of farming itself. You can't plant a row of corn in soil laced with rocks the size of your head. So you pick them out. And once you've picked them out of one field, the logical place for them is on the boundary between your land and your neighbor's.
+
+What's remarkable is how they built them. The best walls, the ones that still stand straight and solid more than two hundred years later, weren't built with mortar. They were built by careful fitting, placing each stone so that its weight and angle held the ones next to it in place. This required skill and judgment. Some of the best wall builders became known in their communities for their work. A well-built wall would last generations without maintenance. A poorly built one would tumble down in a few decades, and you'd have the expensive work of rebuilding it.
+
+Walk along an old farm wall and you learn something about that particular place and time. The walls in areas with glacial rubble tend to be made of round stones, which are harder to fit than the more rectangular stones found elsewhere. Some walls are clearly very old, with less regular stonework and signs of centuries of settling. Others are more recent—from the eighteenth or nineteenth century—and show more careful, architectural attention. You can see where someone's technique improved as they moved along a wall, or where multiple different builders worked on the same structure.
+
+Fences served a different purpose and a different timeline. Most of the wooden fences that would have existed in these places are gone now—wood rots, especially in the wet climate of New England. But you still sometimes see old fence lines marked by stones placed to hold the wooden fence posts in place. Or you see a line of trees—lilacs, apple trees, cherries—that mark where a fence used to run. The farmer wouldn't have planted these trees as the fence, but they would have grown up around and through the fence line, and eventually, once the fence was gone, the trees would remain to mark where the boundary had been.
+
+What's important about stone walls is that they represent a labor intensive approach to land management. You couldn't build these walls without a serious commitment to staying on your land and maintaining your property. A nomadic farmer, or one who was going to leave in a few years, wouldn't invest in building a stone wall. So the presence of walls in a landscape is itself a marker of settlement and permanent occupation.
+
+Modern property is marked with surveyor's paint, metal stakes, and legal documents. These are more efficient—you don't need a surveyor to show up every few decades to rebuild your boundary marker. But something has been lost. The stone wall is a visible, physical assertion of human activity and intention on the landscape. When you walk through old farm country, the walls tell you where people decided their property began and ended, where they were willing to invest labor, where they felt secure enough to build something meant to last.
+
+The walls also document ecological change. In many areas, walls that once marked the boundary of cleared farmland now run through forests. That means the farms were abandoned, the fields grew back to trees, and the walls became embedded in woods rather than between fields. Some walls run through swamps now. The landscape has changed, but the walls remain, marking a version of the land that no longer exists.
+
+It's worth preserving and protecting these walls not just for their historical interest, but because they're part of how we understand what humans have done to shape this landscape. They're not monuments in the sense of being deliberately built to commemorate something grand. They're the practical, physical record of how people lived, what they valued, and how they transformed the land to make it useful.

bench/eval/llm-heldout/03-mud-season.md

@@ -0,0 +1,15 @@
+# Mud Season
+
+There's a season in Vermont that doesn't appear on any calendar, but everyone who lives here knows it by name. Mud season comes after the snow has melted but before the ground has firmed up again—usually April, sometimes extending into early May. It's a few weeks when the world between frozen and thawed.
+
+During mud season, the roads turn to clay and the fields become impassable. The ground releases all the water it's been holding frozen, and for a little while there's more water than earth. Driveways that were solid in March become treacherous. Gravel roads deteriorate into something closer to rivers. People learn which roads are passable and which ones you simply cannot use, and they plan their routes accordingly. The town might put up weight restrictions on certain roads—no heavy trucks, because a fully loaded log truck will sink two feet into the pavement.
+
+Your boots become a constant concern. You learn to stomp them off before entering a building, and you become accustomed to a certain amount of mud in every room. People keep a pair of mud boots by the door and wear them into town. It's not really possible to stay clean during mud season. You accept this and move on.
+
+There's something about it that feels primordial. The earth is moving, changing its state. You can see the process happening in real time. Fields that were hard enough to walk on are suddenly swampy. Streams that were quiet all winter are roaring with snowmelt. The dirt itself seems alive, releasing that particular smell of wet earth, of things decomposing, of the ground waking up.
+
+Animals struggle with it. Horses can't be ridden on muddy trails because their feet get sucked down into the muck. Chickens hate it. Even cars complain—the grime coats everything, getting into the undercarriage and requiring cleaning that most people just give up on until the roads dry.
+
+But there's also beauty in it. The mud season is when the world looks most honest. There's no pretense, no way to make things look tidy. The landscape is raw and messy and real. And it's temporary. The season lasts a few weeks, then the ground firms up, the sun gets stronger, and suddenly everything is green again and people emerge from their houses like bears from hibernation.
+
+Newcomers often find mud season to be the hardest part of living here—harder than the snow, harder than the cold. But once you've lived through it, you understand what it is: a transition. A reminder that the earth is constantly changing, constantly moving between states. And that if you live here, you have to move along with it.

bench/eval/llm-heldout/04-woodstove.md

@@ -0,0 +1,21 @@
+# Heating Season
+
+My woodstove arrived on a Wednesday in October, delivered by a man who studied my setup and said, "You sure about this?" and I said yes, not mentioning that I wasn't sure about anything. He installed the chimney, walked me through the operation once, and left me alone with a cast iron box that would, if I did this right, keep me warm for the next six months.
+
+The first real cold night, I built my first fire. The kindling caught, the logs caught, and I stood there watching the flames, aware that I was about to either sleep comfortably or learn a hard lesson about poor preparation. The stove worked. Heat poured out into the room.
+
+By January, I understood that heating with wood requires a kind of constant negotiation with the weather. On mild days, you don't need the fire at all, so you let it die down to coals and let the house cool slowly. On very cold nights, you load it completely before bed, pushing logs in until they're packed tightly enough to burn slowly through the dark hours. You learn how long different wood burns, how tight to pack it, how to adjust the air vents to control the speed of combustion. Too much air and everything burns in four hours. Too little and the fire smolders, producing smoke and creosote without enough heat.
+
+The house develops a temperature geography. The room with the stove is always warm. Rooms farther away stay cooler. I adapt by spending more time near the heat source—reading by the stove, cooking there when possible. The thermometer rarely goes above 65 degrees Fahrenheit in the rest of the house. But right next to the stove, it's sometimes too warm to sit.
+
+The wood itself becomes a project. I buy a cord in the fall, stacked in my yard. But this wood needs to be seasoned. Green wood—wood that was cut recently—has too much moisture and won't burn well. The wood needs to have dried for at least six months, preferably a year. So I buy this year's wood for next year's fire. There's a relationship between your past, present, and future that heating with wood makes unavoidable.
+
+Ashes accumulate. Every few days, I clean the stove, removing the accumulated ash and depositing it in a bucket. This ash is useful—it can go in the garden or on icy paths—but it also needs to be managed. Some ashes go back in the stove as a bed for the next fire; they insulate the logs and help the fire burn more completely.
+
+There's a satisfaction to this work that I didn't expect. Each day, I'm making a decision about heat. Each load of wood is a small investment in the next few hours of warmth. On cold nights, when the house is silent and warm, I feel the presence of the wood I cut or stacked or paid for months earlier. The heat is immediate, physical, and entirely dependent on my labor.
+
+This kind of heating is not efficient by modern standards. A forced-air oil furnace would be easier. But it would also disconnect me from the process. I wouldn't see the fuel arriving or be aware of how much I was burning. I wouldn't have the continuous negotiation with weather. The house would simply reach a set temperature and stay there.
+
+By spring, when the weather finally stays warm and I let the last fire die down to ash, I understand that something has shifted. I've spent six months in a slow dance with heat and cold. I've learned the properties of different woods, the sounds the stove makes when it's running right, the exact light of coals when you're about to load it for the night. I've stopped thinking about heating as an invisible system and started experiencing it as a daily practice.
+
+When the true warm weather comes and I close the damper for the season, it feels like the end of something. But not quite an ending. Because in the back of my mind, I'm already thinking about next year, about the wood that needs to dry, about the first fire I'll build come October.

bench/eval/llm-heldout/05-library.md

@@ -0,0 +1,23 @@
+# The Library on the Corner
+
+The library in my town occupies a brick building that used to be a bank, built sometime in the 1920s when the town was larger and more prosperous than it is now. The vault is still there, a massive iron door that the librarians use for storage. Sometimes you see them disappear behind it with a stack of books and emerge with their arms full of something else. The building still feels like it's taking care of something precious, which I suppose it is.
+
+I started going there without any particular plan, just on afternoons when the weather was good and I needed a reason to take a walk. The librarians knew nobody. They seemed glad when anyone came in. The quiet there is different from the quiet of being alone at home. It's a shared quiet, a space where other people are quietly doing their own things. Someone is reading. Someone is looking for information. Someone is just sitting, using the place as shelter and company.
+
+The collection is not comprehensive. If you're looking for a specific book, there's maybe a forty percent chance they have it. But what they do have is interesting—not because the selection is curated by brilliant minds, but because it reflects what people in this town have donated over the decades, and what the librarians have chosen to buy. The fiction section is heavy on mysteries and literary novels. There's a surprising amount of poetry. The non-fiction is idiosyncratic: a whole shelf of books about Vermont history, books about gardening, books about woodworking, a few about farming and animal husbandry.
+
+The librarians remember who you are. They ask how you're doing. If you return a book and mention that you didn't like it, they remember that next time you come in and suggest something else they think might suit you better. This kind of attention has become rare enough that it shocks people when they first experience it.
+
+The library has programs. On Tuesday afternoons, a woman teaches a knitting circle. On Thursday mornings, there's story time for children. Once a month, there's a community board where people post news and ask for help and advertise things they're selling. The bulletin board is covered with notices about town meetings and lost cats and yoga classes and someone looking for help with their garden.
+
+There's a computer section with three machines. An old printer that always seems to be jammed. A table where people spread out their documents. I've sat there working on my laptop while someone else was video calling their grandchildren on one of the library computers, and there was something sustaining about being in a space where people were connecting across distance.
+
+The library closes on Sundays and Mondays. This schedule makes no sense if you're thinking about maximum convenience. But it makes perfect sense if you're thinking about a town where the librarians have lives outside the library, where the building needs to close sometimes, where the goal is not to serve the maximum number of people but to serve the people in the town well.
+
+The collection is always slightly out of date. The books about computers are years old. The current events section contains magazines from last month. If you're trying to find the latest information on something, you're probably better off with Google. But if you're looking for a book to read, or you want to ask someone a question, or you want to be in a place where learning is quietly valued, it's perfect.
+
+I've watched people use it in different ways. A student used the computers to apply for colleges. Someone was reading about illness and treatment, sitting in a quiet corner, taking notes. A woman was teaching her elderly mother how to use the library, checking out large-print books, speaking patiently as if that patience was the whole point. A man was reading the newspaper, as he apparently did every day at the same time, and the librarians greeted him like an old friend.
+
+What I've come to understand about a library like this is that it's not really about the books, not entirely. It's about the existence of a space where learning and discovery are valued, where quiet is protected, where a community has invested in the idea that knowledge and reading matter enough to maintain a building and staff it and keep it open, even when that's not the most economical choice.
+
+The library is small and understaffed and underfunded. The building is old and needs repairs. The collection will never be comprehensive. But it's there, in a brick building that used to be a bank, holding space for the idea that a community should have a place where people can read and learn and ask questions and sit quietly together. In an age when most things are optimized for maximum efficiency and maximum reach, there's something almost radical about a library that's just trying to serve the people who live in its town.