feat(vscode): recipes view with Run and save, Learn view, walkthrough (0.2.0) (#292)

.github/workflows/vscode-e2e.yml

@@ -28,7 +28,7 @@ jobs:
       - uses: actions/cache@v4
         with:
           path: apps/vscode-extension/.vscode-test
-          key: vscode-test-${{ runner.os }}-1.90.0
+          key: vscode-test-${{ runner.os }}-1.127.0
       - name: The extension in a real editor, on a virtual display
         run: xvfb-run -a npm run vscode:e2e
         env:

apps/vscode-extension/README.md

@@ -15,6 +15,21 @@ Issue #228. The server side it relies on is `choose-folder` (#234, the Browse bu
     recipe from it, the same as dropping the file on Studio. A file inside the workspace folder is read in
     place; one from outside is copied into the workspace folder.
 
+## The OpenCraw sidebar
+
+- **Recipes** lists the recipes in your workspace (`*.ocw-in.json`, `*.ocw-out.json`, and the `*.input.json` / `*.output.json` the examples use)
+  **grouped by output id**: each output recipe is a row, the input recipes that fill it are under it.
+  - **Run** on an input recipe runs that one; on an output recipe it runs every input that feeds it. The crawl runs in a process of its own with
+    a notification you can cancel, and its messages go to the **OpenCraw** output.
+  - When the crawl is over you are asked where to save the result with your operating system's own save dialog. **The file's extension is the
+    format**: `.xlsx` (Excel), `.csv`, `.html` (a page that looks like Studio), `.pdf`, `.json`, `.jsonl` or `.yml`/`.yaml`. A name that is none
+    of these is explained and the dialog opens again.
+  - **Open in Studio** opens the recipe to change it.
+- **Learn** links the Studio manual, the tutorial, the recipe guides and where to ask.
+- **OpenCraw: Get started** opens a walkthrough (it also opens once, the first time the extension runs).
+
+Needs VS Code 1.127 (June 2026) or newer.
+
 The server starts on first use, in a process of its own, and stops with the window. In a remote window
 (SSH, WSL, Codespaces, dev containers) it runs where the workspace is and its port is forwarded to the
 browser on your machine.
@@ -33,6 +48,11 @@ editor (extension host)            Studio process                       browser
 - `src/studio-host`: forks the Studio process and asks it for links over IPC. The process is the editor's own
   executable run as plain Node (`ELECTRON_RUN_AS_NODE`), so the extension ships no Node.
 - `src/open-on-opencraw`: the command's outcome, the only place that touches the editor API.
+- `src/recipe-explorer`: finds the recipe files, reads each one's `kind`/`id`/`output` (the extension does not carry the engine), groups them by
+  output id, and is the recipes view.
+- `src/recipe-run`: the Run action: the plan (which files, `--only`), the process that runs the bundled `opencraw` command line, the save dialog
+  and the format from the extension, and `opencraw export` to write the records in it.
+- `src/learn-studio`: the Learn view and the Get started walkthrough (`media/walkthrough`).
 - `src/extension.ts`: the entry point the editor requires.
 
 **Recipe ▾ → Open folder…** in Studio shows the editor's own folder dialog (`vscode.window.showOpenDialog`): the
@@ -95,7 +115,7 @@ Open VSX is skipped when `OVSX_PAT` is not set.
 
 ## End-to-end test
 
-`npm run vscode:e2e` stages the extension as the editor loads it, starts a real VS Code (1.90.0, the oldest the
+`npm run vscode:e2e` stages the extension as the editor loads it, starts a real VS Code (1.127.0, the oldest the
 extension supports; downloaded into `.vscode-test` on the first run) and runs `e2e/suite/index.cjs` in its
 extension host: the commands exist, "Open on OpenCraw" resolves a link to a real Studio process, the page is
 served, `open-path` answers, and the `opencraw.hooksFile` setting reaches the server. The browser the link is

apps/vscode-extension/e2e/run-editor-test.mjs

@@ -9,12 +9,17 @@ import { runTests } from '@vscode/test-electron'
 
 const appDir = resolve(dirname(fileURLToPath(import.meta.url)), '..')
 /** The oldest editor the extension claims to support (`engines.vscode`). */
-const VERSION = '1.90.0'
+const VERSION = '1.127.0'
 
 const workspace = mkdtempSync(join(tmpdir(), 'opencraw-editor-workspace-'))
 const profile = mkdtempSync(join(tmpdir(), 'opencraw-editor-profile-'))
 mkdirSync(join(workspace, 'docs'))
 writeFileSync(join(workspace, 'docs', 'report.csv'), 'name,price\nbook,10\n')
+// Recipes for the recipes view: two inputs that fill one output, and an input whose output is not in the workspace.
+writeFileSync(join(workspace, 'cars.ocw-out.json'), JSON.stringify({ kind: 'output', id: 'cars', version: 1, fields: { model: { type: 'string' } } }))
+writeFileSync(join(workspace, 'kia.ocw-in.json'), JSON.stringify({ kind: 'input', id: 'kia', output: 'cars', mode: 'api', start: [], steps: [] }))
+writeFileSync(join(workspace, 'byd.ocw-in.json'), JSON.stringify({ kind: 'input', id: 'byd', output: 'cars', mode: 'api', start: [], steps: [] }))
+writeFileSync(join(workspace, 'lost.ocw-in.json'), JSON.stringify({ kind: 'input', id: 'lost', output: 'elsewhere', mode: 'api', start: [], steps: [] }))
 
 try {
   await runTests({

apps/vscode-extension/e2e/suite/index.cjs

@@ -41,6 +41,11 @@ async function run () {
   const registered = await vscode.commands.getCommands(true)
   assert.ok(registered.includes('opencraw.openStudio'), 'Open Studio is registered')
   assert.ok(registered.includes('opencraw.openOn'), 'Open on OpenCraw is registered')
+  for (const name of ['runRecipe', 'openRecipeInStudio', 'refreshRecipes', 'showWalkthrough', 'openLink']) assert.ok(registered.includes(`opencraw.${name}`), `${name} is registered`)
+
+  // The recipes view lists the workspace's recipes by the output they fill: an input whose output is missing is kept, under its name.
+  const groups = await api.recipeGroups()
+  assert.deepEqual(groups.map(group => [group.outputId, group.outputFile !== undefined, group.inputs.map(input => input.id)]), [['cars', true, ['byd', 'kia']], ['elsewhere', false, ['lost']]])
 
   const folder = vscode.workspace.workspaceFolders[0].uri
   const report = vscode.Uri.joinPath(folder, 'docs', 'report.csv')

apps/vscode-extension/media/walkthrough/collect.md

@@ -0,0 +1,13 @@
+# Click what you want
+
+Click the value you want to collect. Studio shows the ways to find it again on the next visit, with how many things each one matches:
+
+- the suggested one is the most reliable;
+- an id that changes on every visit is never used;
+- you can type your own.
+
+Click a **second, similar** value (the next product's name, say) and Studio collects the **whole list**.
+
+When a way to find it matches several things, you choose: only the first, or all of them as a list.
+
+Then press **Add to recipe**.

apps/vscode-extension/media/walkthrough/load.md

@@ -0,0 +1,10 @@
+# Load what you want to read
+
+In Studio press **Load** and choose:
+
+- **Web page**: paste an address. Cookie banners are closed for you.
+- **Document**: a PDF, a spreadsheet (xlsx, csv), a deck, a Word file, JSON, YAML, XML or Markdown.
+
+Whatever you load appears on the left, the way it looks: a page, a PDF page, a grid or a tree. The steps of the recipe are on the right.
+
+Big files are fine: a long spreadsheet or JSON shows only the part you are looking at.

apps/vscode-extension/media/walkthrough/open-studio.md

@@ -0,0 +1,7 @@
+# Open Studio
+
+Studio is where you build recipes by clicking. It starts on your computer, for this workspace, and opens in your browser: nothing is sent anywhere.
+
+Use **OpenCraw: Open Studio** from the command palette, or right-click any file or folder and choose **Open on OpenCraw**. A PDF, a spreadsheet, a Word file, a JSON or Markdown file starts a recipe straight away.
+
+The first time can take a few seconds while the server starts.

apps/vscode-extension/media/walkthrough/recipes.md

@@ -0,0 +1,8 @@
+# Run it from the editor
+
+The **OpenCraw** view in the sidebar lists your recipes, grouped by the output they fill.
+
+- **Run** on an input recipe runs just that one.
+- **Run** on an output recipe runs every input that feeds it.
+- When the crawl is over you choose where to save the result, and its format is the file's extension: Excel (`.xlsx`), CSV, a web page (`.html`), PDF, JSON, JSON Lines or YAML.
+- **Open in Studio** opens the recipe to change it.

apps/vscode-extension/media/walkthrough/run.md

@@ -0,0 +1,10 @@
+# Run a sample
+
+Press **Run sample**. The records appear in a table at the bottom, and the **Trace** tab tells you what the crawl did, step by step.
+
+If a value is missing or wrong, **Why?** explains which step produced it and what it saw.
+
+When it looks right, the recipe is already saved in your folder as two files:
+
+- an **input recipe** (`.ocw-in.json`): where to read, and what to take;
+- an **output recipe** (`.ocw-out.json`): the shape of the result. Different inputs can fill the same output, so different sources give you the same kind of data.

apps/vscode-extension/package.json

@@ -1,15 +1,15 @@
 {
   "name": "@opencraw/vscode",
-  "version": "0.1.6",
+  "version": "0.2.0",
   "private": true,
-  "description": "OpenCraw in the editor: starts the Studio server and opens a folder, recipe or document in the browser.",
+  "description": "OpenCraw in the editor: open folders, recipes and documents in Studio, run your recipes from a view and save the result in the format you choose, and learn how it all works.",
   "type": "module",
   "main": "dist/extension.cjs",
   "displayName": "OpenCraw",
   "publisher": "opencraw",
   "icon": "media/icon.png",
   "engines": {
-    "vscode": "^1.90.0"
+    "vscode": "^1.127.0"
   },
   "categories": [
     "Other"
@@ -28,15 +28,23 @@
     "views": {
       "opencraw": [
         {
-          "id": "opencraw.start",
-          "name": "OpenCraw"
+          "id": "opencraw.recipes",
+          "name": "Recipes",
+          "icon": "media/opencraw-activity-bar.svg",
+          "contextualTitle": "OpenCraw recipes"
+        },
+        {
+          "id": "opencraw.learn",
+          "name": "Learn",
+          "icon": "media/opencraw-activity-bar.svg",
+          "contextualTitle": "Learn OpenCraw"
         }
       ]
     },
     "viewsWelcome": [
       {
-        "view": "opencraw.start",
-        "contents": "Run recipes in your browser: Studio starts a local server and opens the page.\n[Open Studio](command:opencraw.openStudio)\nTo open a folder, recipe or document, right-click it in the Explorer and choose **Open on OpenCraw**."
+        "view": "opencraw.recipes",
+        "contents": "No recipes in this folder yet.\nStudio builds them for you: load a web page or a document, click what you want, and it writes the recipe.\n[Open Studio](command:opencraw.openStudio)\nRecipes are the files ending in `.ocw-in.json` (what to read) and `.ocw-out.json` (the shape of the result). They are grouped here by the output they fill."
       }
     ],
     "jsonValidation": [
@@ -74,6 +82,35 @@
         "command": "opencraw.openOn",
         "title": "Open on OpenCraw",
         "category": "OpenCraw"
+      },
+      {
+        "command": "opencraw.runRecipe",
+        "title": "Run",
+        "category": "OpenCraw",
+        "icon": "$(play)"
+      },
+      {
+        "command": "opencraw.openRecipeInStudio",
+        "title": "Open in Studio",
+        "category": "OpenCraw",
+        "icon": "$(browser)"
+      },
+      {
+        "command": "opencraw.refreshRecipes",
+        "title": "Refresh the recipes",
+        "category": "OpenCraw",
+        "icon": "$(refresh)"
+      },
+      {
+        "command": "opencraw.showWalkthrough",
+        "title": "Get started",
+        "category": "OpenCraw",
+        "icon": "$(mortar-board)"
+      },
+      {
+        "command": "opencraw.openLink",
+        "title": "Open a link",
+        "category": "OpenCraw"
       }
     ],
     "menus": {
@@ -102,9 +139,115 @@
         {
           "command": "opencraw.openOn",
           "when": "resourceScheme == file"
+        },
+        {
+          "command": "opencraw.runRecipe",
+          "when": "false"
+        },
+        {
+          "command": "opencraw.openRecipeInStudio",
+          "when": "false"
+        },
+        {
+          "command": "opencraw.openLink",
+          "when": "false"
+        }
+      ],
+      "view/title": [
+        {
+          "command": "opencraw.openStudio",
+          "when": "view == opencraw.recipes",
+          "group": "navigation@1"
+        },
+        {
+          "command": "opencraw.refreshRecipes",
+          "when": "view == opencraw.recipes",
+          "group": "navigation@2"
+        },
+        {
+          "command": "opencraw.showWalkthrough",
+          "when": "view == opencraw.learn",
+          "group": "navigation@1"
+        }
+      ],
+      "view/item/context": [
+        {
+          "command": "opencraw.runRecipe",
+          "when": "view == opencraw.recipes && viewItem =~ /^opencraw\\.(output\\.runnable|input)$/",
+          "group": "inline@1"
+        },
+        {
+          "command": "opencraw.openRecipeInStudio",
+          "when": "view == opencraw.recipes && viewItem =~ /^opencraw\\.(output|input)/ && viewItem != opencraw.output.missing",
+          "group": "inline@2"
+        },
+        {
+          "command": "opencraw.runRecipe",
+          "when": "view == opencraw.recipes && viewItem =~ /^opencraw\\.(output\\.runnable|input)$/",
+          "group": "navigation@1"
+        },
+        {
+          "command": "opencraw.openRecipeInStudio",
+          "when": "view == opencraw.recipes && viewItem =~ /^opencraw\\.(output|input)/ && viewItem != opencraw.output.missing",
+          "group": "navigation@2"
         }
       ]
-    }
+    },
+    "walkthroughs": [
+      {
+        "id": "opencraw.getStarted",
+        "title": "Get started with OpenCraw",
+        "description": "Collect data from a web page, a PDF, a spreadsheet or a file, by clicking: no code.",
+        "steps": [
+          {
+            "id": "open",
+            "title": "Open Studio",
+            "description": "Studio runs on your computer and opens in your browser, on this workspace.\n[Open Studio](command:opencraw.openStudio)",
+            "media": {
+              "markdown": "media/walkthrough/open-studio.md"
+            },
+            "completionEvents": [
+              "onCommand:opencraw.openStudio"
+            ]
+          },
+          {
+            "id": "load",
+            "title": "Load what you want to read",
+            "description": "A web address, a PDF, a spreadsheet, a Word file, JSON: Load puts it on the left of Studio.",
+            "media": {
+              "markdown": "media/walkthrough/load.md"
+            }
+          },
+          {
+            "id": "collect",
+            "title": "Click what you want",
+            "description": "Click a value to collect it. Click a second, similar one to collect the whole list. Pick the way to find it that will keep working.",
+            "media": {
+              "markdown": "media/walkthrough/collect.md"
+            }
+          },
+          {
+            "id": "run",
+            "title": "Run a sample",
+            "description": "Run sample shows the records in a table. Fix what looks wrong, then save.",
+            "media": {
+              "markdown": "media/walkthrough/run.md"
+            }
+          },
+          {
+            "id": "recipes",
+            "title": "Run it from the editor",
+            "description": "Your recipes are listed here, grouped by the output they fill. Run one, or run them all, and choose where to save the result and in what format.\n[Show the recipes](command:opencraw.recipes.focus)",
+            "media": {
+              "markdown": "media/walkthrough/recipes.md"
+            },
+            "completionEvents": [
+              "onCommand:opencraw.runRecipe"
+            ]
+          }
+        ]
+      }
+    ]
   },
   "nx": {
     "targets": {
@@ -192,7 +335,7 @@
     "@opencraw/studio": "^0.0.1"
   },
   "devDependencies": {
-    "@types/vscode": "1.90.0",
+    "@types/vscode": "1.125.0",
     "@vscode/test-electron": "^3.1.0"
   }
 }