feat: add scigantic-catalog agent

agents/scigantic-catalog/README.md

@@ -0,0 +1,163 @@
+# Scigantic Catalog
+
+A research data librarian for public scientific datasets. `SciganticCatalog` searches the
+[Scigantic](https://scigantic.com) catalog of several thousand curated open archives, returns each
+archive's schema card (format, columns, sample rows, data dictionaries, caveats) and a copy-paste
+loader for the user's own environment, and links arXiv papers to the datasets behind them. It reads
+the catalog through the public [Scigantic MCP server](https://github.com/Scigantic/scigantic-mcp),
+which needs no API key.
+
+## Overview
+
+Finding a public dataset normally means a web search, a registry page, a bucket listing and a
+download before you know whether the columns you need are there. This agent answers those
+questions from the catalog's schema cards first.
+
+Ask it for data on a topic and it searches the catalog, discards archives whose storage was probed
+and found unreadable, checks the schema cards of the best candidates, and recommends one with its
+storage location, format, key columns and a loader snippet. Ask it about a paper and it finds the
+datasets the paper used or suggests. Ask it what a dataset looks like and it answers from the
+schema card.
+
+The catalog spans genomics and transcriptomics, medical and microscopy imaging, cryo-EM and
+structural biology, chemistry and ADMET, neuroscience, ecology, earth observation, weather and
+climate, and materials. Every archive is read-only public data on S3, GCS, Hugging Face, Zenodo
+or plain HTTP. The same MCP server also exposes SureChEMBL patent chemistry tools, which the agent
+uses when asked about patents.
+
+Intended users: computational biologists, chemists, data scientists and anyone who needs to locate
+and understand a public dataset before committing compute to it. A successful outcome is a dataset
+brief the user can act on: where the data is, what is in it, how to load it, and what to watch out
+for.
+
+## Architecture
+
+`SciganticCatalog` is a prompt agent with no containerized tools. It calls the remote Scigantic MCP
+server over HTTP and uses Discovery's built-in resource tools to persist a finished dataset brief.
+
+```mermaid
+flowchart LR
+    U(["Data question or arXiv paper"]) --> A["SciganticCatalog<br/>(prompt agent)"]
+    A -->|search_archives| M[("Scigantic MCP server<br/>api.scigantic.com/mcp")]
+    A -->|get_schema_card<br/>get_data_access<br/>list_archive_files| M
+    A -->|lookup_paper| M
+    M --> C[("Scigantic catalog<br/>schema cards, file trees")]
+    A --> B(["Dataset brief:<br/>location, format, columns, loader, caveats"])
+```
+
+The MCP server is anonymous and read-only. It exposes these catalog tools:
+
+| Tool | What it returns |
+| --- | --- |
+| `search_archives(query, category?, limit?)` | Ranked archives with id, title, category, summary, `probeStatus` and `objectCountEstimate` |
+| `get_archive(id)` | Full metadata for one archive, including its schema card |
+| `get_schema_card(id)` | File format, columns, sample rows or headers, inlined READMEs and data dictionaries, caveats, starter cell |
+| `get_data_access(id, language?)` | Storage location plus loader snippets for the user's own environment |
+| `list_archive_files(id, limit?)` | A sample of the files or objects in the archive |
+| `get_sample_notebook(id)` | Cell-by-cell source of a worked starter notebook, when one exists |
+| `lookup_paper(query)` | An arXiv paper by id or title, with linked catalog archives |
+
+Six `surechembl_*` tools cover patent chemistry (compound lookup, patents for a compound, patent
+text and extracted structures, Solr patent search, structure search). The `search` and `fetch`
+tools are reduced aliases kept for ChatGPT connectors, and `list_compute_tiers` describes
+Scigantic's own hosted notebooks; the agent is instructed to ignore all three.
+
+The agent does not read the data. It returns the storage location and a loader snippet, and the
+data is read wherever the user runs that snippet.
+
+## Prerequisites
+
+- A Microsoft Discovery workspace and project, with a chat model deployment available to the
+  project. The agent is written for a reasoning model, so `agent.yaml` sets no model `options`.
+- Permission to add a tool in the Microsoft Foundry project behind your Discovery workspace.
+- Outbound HTTPS from the Foundry project to `api.scigantic.com`.
+
+No Scigantic account, API key or subscription is needed. The MCP endpoint is public.
+
+## Configuration
+
+`agent.yaml` uses the `{{CHAT-MODEL}}` placeholder for the model deployment. Supply your own
+deployment name at deploy time.
+
+| Parameter | Description | Example |
+| --- | --- | --- |
+| `{{CHAT-MODEL}}` | Microsoft Foundry model deployment name | `gpt-5-deployment` |
+
+The agent declares no `tools:` array. The Scigantic catalog is reached through a remote MCP
+connection registered once per Foundry project and attached to the agent:
+
+| Setting | Value |
+| --- | --- |
+| Name | `scigantic` |
+| Remote MCP Server endpoint | `https://api.scigantic.com/mcp` |
+| Authentication | None required. The endpoint is anonymous and read-only. |
+| Approval | **Always auto-approve all tools** |
+
+Approval must be set to auto-approve. The default asks for confirmation before every tool call,
+and in an autonomous task run there is nobody to confirm, so the agent stalls.
+
+[`mcp/README.md`](mcp/README.md) walks through the Foundry steps.
+
+## Usage
+
+Deploy the agent, connect the MCP tool as above, then address it in a Discovery Studio session:
+
+```text
+@SciganticCatalog I need single-cell RNA-seq of human cortex with cell-type annotations.
+What public datasets fit, and how are they structured?
+```
+
+It searches, checks the schema cards of the best candidates, and answers with a recommendation,
+a comparison table, the storage location, key columns and a loader snippet. Follow up
+conversationally:
+
+```text
+Is there a mouse equivalent?
+Show me a sample of the files.
+What are the caveats in the data dictionary?
+```
+
+Start from a paper instead of a topic:
+
+```text
+@SciganticCatalog Which datasets does arXiv 2410.12345 use, and which are in the catalog?
+```
+
+Or ask about patent chemistry:
+
+```text
+@SciganticCatalog Who has patented compounds similar to this SMILES since 2022: CC(=O)Oc1ccccc1C(=O)O
+```
+
+Pair this agent with Discovery's `core-python-agent` or `stat-agent` to run the loader snippet and
+begin analysis on the dataset it recommends.
+
+## Known Limitations
+
+- **No data retrieval.** The agent returns where the data is and how to load it. It does not
+  download or read the data itself. Run the loader snippet in a code agent or your own
+  environment.
+- **Some archives are credentialed.** The catalog lists archives whose storage turned out to need
+  credentials. These carry `probeStatus: empty` and the agent is instructed to skip them, but the
+  flag comes from a periodic probe and can lag a change in the upstream bucket.
+- **Search ranking is approximate.** Highly specific queries can rank a near-topic archive above
+  the best one. The agent rewords the query and checks schema cards before recommending an archive.
+- **Schema cards are sampled.** A card is built from a sample of files and rows, so a rare column
+  or a format that appears only deep in the tree can be missing from it.
+- **Paper lookup is arXiv only.** DOIs, PubMed ids and non-arXiv papers do not resolve.
+- **No per-user state.** The endpoint is anonymous, so there are no saved searches or
+  preferences across sessions.
+
+## Support
+
+For questions about the catalog, the MCP server or the data behind an archive, open an issue at
+<https://github.com/Scigantic/scigantic-mcp/issues> or write to <[email protected]>.
+
+For issues with this catalog entry, open a discussion in the
+[Microsoft Discovery repository](https://techcommunity.microsoft.com/category/azure/discussions/microsoft-discovery-discussions).
+
+## Contributing
+
+Improvements to this agent definition are welcome. Please read
+[CONTRIBUTING.md](../../CONTRIBUTING.md) in the repository root for the catalog's submission flow,
+schema requirements and PR checks.

agents/scigantic-catalog/agent.yaml

@@ -0,0 +1,93 @@
+kind: prompt
+name: SciganticCatalog
+displayName: Scigantic Catalog
+description: >-
+  Finds public scientific datasets in the Scigantic catalog and explains how each one is
+  structured and how to load it, before any data is downloaded. Searches several thousand curated
+  open archives across the life, chemical, earth and physical sciences, returns schema cards
+  (format, columns, sample rows, data dictionaries, caveats) and copy-paste loader snippets, lists
+  sample files, and links arXiv papers to the datasets behind them. Read-only; never fabricates a
+  dataset or a path.
+model:
+  id: '{{CHAT-MODEL}}'
+instructions: |-
+  You are a research data librarian. You help scientists find public datasets for a question and
+  understand each dataset's structure well enough to start analysis, using the Scigantic MCP tools.
+  Everything you say about a dataset must come from a tool result in this conversation. You never
+  describe a dataset, a column, a file path or a storage location from memory, and when the catalog
+  has nothing suitable you say so instead of inventing a match.
+
+  ## Tools
+
+  Invoke one tool at a time. Read the result, then decide the next call.
+
+  - `search_archives(query, category?, limit?)`: natural-language search over the whole catalog.
+    Matching uses both the archive description and a sample of real file names, so a distinctive
+    filename or path pattern is a good query. Each hit carries `id`, `title`, `category`, a short
+    summary, and two trust signals: `probeStatus` and `objectCountEstimate`. A `probeStatus` of
+    `empty` means the storage was probed and holds no readable objects (usually credentialed);
+    do not recommend such an archive, or say plainly that it needs credentials.
+  - `get_archive(id)`: full metadata for one archive, including the schema card when one exists.
+  - `get_schema_card(id)`: the compact schema card only: file format, columns, sample rows or
+    headers, inlined READMEs and data dictionaries, caveats, and a starter code cell. This is the
+    fastest way to understand a dataset's structure.
+  - `get_data_access(id, language?)`: where the data lives (S3, GCS, HTTP, Hugging Face) plus
+    copy-paste loader snippets for the user's own environment.
+  - `list_archive_files(id, limit?)`: a sample of the files or objects in the archive.
+  - `get_sample_notebook(id)`: the cell-by-cell source of a worked starter notebook, when one
+    exists.
+  - `lookup_paper(query)`: an arXiv paper by bare id or title, with any catalog archives linked to
+    it. arXiv only: DOIs and non-arXiv papers do not resolve.
+  - `surechembl_*` tools: patent chemistry from SureChEMBL (compound lookup, patents mentioning
+    a compound, a patent's text and extracted structures, Solr patent search, structure search).
+    Use these only when the user asks about patents or patent chemistry.
+  - Ignore `search`, `fetch` and `list_compute_tiers`. The first two are reduced aliases of
+    `search_archives` and `get_archive`; the third describes Scigantic's own hosted notebooks and
+    does not apply inside Discovery.
+  - Discovery's data-handling tools: use them to save a finished dataset brief as a workspace
+    asset when the user is going to act on it, and to read a file the user attached. Never save
+    a raw tool payload as an asset, and never write a placeholder for a search that failed.
+
+  ## How to work
+
+  1. Pin down what data the user needs: organism or system, modality or measurement, scale, and
+     any format or storage constraint they stated. Do not add constraints they did not give. Ask
+     one focused question only when two readings would lead to different searches.
+  2. Search with `search_archives`. Start with one natural-language query; if the top hits are off
+     topic, vary the wording or add a `category` filter rather than paging. Try at most four
+     queries. Stop as soon as you have a handful of relevant archives.
+  3. Qualify the candidates. Drop hits whose `probeStatus` is `empty`. Prefer archives with a
+     larger `objectCountEstimate` when two are otherwise alike. If the user's question is
+     specific, pull `get_schema_card` for the top two or three to check that the columns or files
+     actually hold what they need before recommending them.
+  4. For the archive the user picks (or the clear best match), gather `get_schema_card`,
+     `get_data_access`, and a short `list_archive_files` sample. Call `get_sample_notebook` only
+     if the user wants a worked example. Call `lookup_paper` when the user starts from a paper.
+  5. Answer per the output rules below.
+
+  ## Output
+
+  - Lead with the recommendation: which archive, why it fits, and the single most important
+    caveat from its schema card. Then the detail. Natural GitHub Flavored Markdown; use a table
+    to compare several candidate archives (title, category, size signal, format, fit).
+  - For the recommended archive give: the storage location, the file format and key columns or
+    files, and the loader snippet from `get_data_access`, verbatim, in a fenced code block.
+    Quote caveats from the schema card (missing values, units, licensing, access notes).
+  - Always give the archive page link in the form `https://scigantic.com/archive/<id>` so the
+    user can inspect it.
+  - Follow-ups ("what about mouse?", "is there a parquet version?", "show me the files") get a
+    short conversational answer grounded in results already in hand, with at most one new tool
+    call to fill a specific gap.
+  - If the catalog has no good match, say so, name the closest misses and why they fall short,
+    and suggest a reworded query. Do not pad with datasets you remember from elsewhere.
+
+  ## Integrity
+
+  - Never fabricate an archive id, title, bucket, path, column name, row count or license.
+  - Distinguish what the schema card states from what you infer. Label inference as such.
+  - Loader snippets run in the user's environment, not yours. Do not claim to have executed them.
+discoveryExtensions:
+  humanInTheLoop: Disabled
+  tools: []
+  disableDataHandlingTools: false
+  disableDiscoveryInjectedTools: false

agents/scigantic-catalog/mcp/README.md

@@ -0,0 +1,92 @@
+# Connecting the Scigantic MCP tool
+
+`SciganticCatalog` reaches the Scigantic catalog through an **MCP tool**: a connection to the remote
+Scigantic Model Context Protocol server, registered in the Microsoft Foundry project that backs your
+Discovery workspace.
+
+Discovery Studio does not yet expose MCP tool registration, so this is done in Foundry. You register
+the tool **once per project**, then attach it to each agent that needs it. The steps mirror the
+[Wiley Nexus walkthrough](../../wiley-nexus/mcp/README.md), which has a screenshot of each dialog;
+only the connection values differ.
+
+**Before you start:** create the agent in Discovery Studio from `agent.yaml` and **save** it. The
+*Open in Foundry* button appears only on a saved agent. There is no API key to obtain.
+
+## 1. Open the agent in Foundry
+
+In Discovery Studio, open the agent under **Resources, then Agents**, and select **Open in Foundry**.
+
+## 2. Go to Tools and start a new connection
+
+In Foundry's left nav, select **Tools**, open the **Tools** tab, and select **Connect a tool**. If
+`scigantic` is already registered in this project, skip to step 5.
+
+## 3. Choose a custom MCP tool
+
+In the **Select a tool** dialog, switch to the **Custom** tab, choose **Model Context Protocol
+(MCP)**, and select **Create**.
+
+## 4. Enter the Scigantic connection details
+
+| Field | Value |
+| --- | --- |
+| **Name** | `scigantic` |
+| **Remote MCP Server endpoint** | `https://api.scigantic.com/mcp` |
+| **Authentication** | None. The server is anonymous and read-only. |
+
+If the dialog requires a credential before it will connect, choose **Key-based** and enter a
+header such as `X-Scigantic-Client` with any value. The server ignores request headers it does not
+use, including `Authorization`.
+
+Select **Connect**.
+
+## 5. Attach the tool to your agent
+
+Open the `scigantic` tool, select **Use in an agent**, and pick `SciganticCatalog` (use **View all
+agents** if it is not under **Recent agents**).
+
+## 6. Configure the tool and set approval to auto-approve
+
+In the agent, find `scigantic` in the **Tools** list and open its configuration:
+
+| Setting | What to enter |
+| --- | --- |
+| **Allowed scigantic tools** | Leave empty, or allow only the catalog tools listed below. |
+| **Approval setting for tools in this MCP server for this agent** | **`Always auto-approve all tools`** |
+
+Approval must be set to auto-approve. The default asks for confirmation before every tool call,
+and in an autonomous task run there is nobody to confirm, so the agent stalls.
+
+Select **Apply**.
+
+## 7. Save the agent
+
+Select **Save** (or **Publish**). Back in Discovery Studio the agent lists `scigantic` among its
+tools, alongside the resource-handling tools Discovery injects into every agent.
+
+## The Scigantic tools
+
+| Tool | What it does |
+| --- | --- |
+| `search_archives` | Natural-language search over the catalog; returns ranked archives with trust signals |
+| `get_archive` | Full metadata for one archive |
+| `get_schema_card` | The archive's schema card: format, columns, sample rows, sidecar docs, caveats, starter cell |
+| `get_data_access` | Storage location and loader snippets for the user's own environment |
+| `list_archive_files` | A sample of the archive's files or objects |
+| `get_sample_notebook` | Source of a worked starter notebook, when one exists |
+| `lookup_paper` | An arXiv paper and the catalog archives linked to it |
+| `surechembl_compound`, `surechembl_patents_for_compound`, `surechembl_patent`, `surechembl_patent_chemistry`, `surechembl_search_patents`, `surechembl_structure_search` | SureChEMBL patent chemistry |
+| `search`, `fetch` | Reduced aliases kept for ChatGPT connectors; the agent ignores them |
+| `list_compute_tiers` | Scigantic's own hosted notebook tiers; not applicable in Discovery, the agent ignores it |
+
+Every tool is read-only and declares `readOnlyHint`. For the server's source, a stdio variant for
+clients that cannot use HTTP, and the response shapes, see
+<https://github.com/Scigantic/scigantic-mcp>.
+
+## Notes
+
+- **Register once, attach many.** The tool is a project-level resource. Adding it to a second
+  agent is just steps 5 and 6.
+- **Verify the connection** by asking the agent for a dataset on any topic. A working connection
+  answers with archive ids and `https://scigantic.com/archive/<id>` links; a broken one says the
+  catalog returned nothing.

agents/scigantic-catalog/metadata.yaml

@@ -0,0 +1,24 @@
+name: scigantic-catalog
+type: agent
+version: 1.0.0
+associated_tools: []
+publisher:
+  name: Scigantic
+  contact: [email protected]
+  support_url: https://github.com/Scigantic/scigantic-mcp/issues
+  party: 3p
+description: >
+  Finds public scientific datasets and explains their structure before any data is downloaded.
+  Searches the Scigantic catalog of several thousand curated open archives (genomics, imaging,
+  structural biology, chemistry, neuroscience, earth and climate science) through the Scigantic MCP
+  server, returns each archive's schema card (format, columns, sample rows, data dictionaries) and a
+  copy-paste loader for the user's environment, and links arXiv papers to their datasets. Read-only,
+  no API key.
+tags:
+  - open-data
+  - dataset-discovery
+  - data-access
+  - schema-card
+  - life-sciences
+  - earth-science
+  - patent-chemistry