> ## Documentation Index
> Fetch the complete documentation index at: https://subtext.fullstory.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LangSmith and Subtext: From a Signal to the Session

> Attach the Subtext session URL to LangSmith trace metadata, then jump from any flagged run — an error, a low eval score, a thumbs-down — straight into the user's real session.

LangSmith tells you *that* a run misbehaved: it errored, scored low on an eval, or a human thumbs-downed it. It can't tell you whether the person on the other end was actually blocked, confused, or fine. Subtext can. Attach the Subtext session URL to your LangSmith trace metadata and every flagged run becomes a jumping-off point — your agent reads the `subtext_url`, opens that person's real session, and walks the decisive moments with screenshots, the component tree, network, and console. LangSmith detects; Subtext diagnoses.

<Note>
  This guide assumes the [Subtext capture snippet](/docs/install/overview) is installed and the `langsmith` SDK is installed with tracing enabled (`LANGSMITH_TRACING=true`, `LANGSMITH_API_KEY` set), or that you run on LangSmith Deployment (formerly LangGraph Platform).
</Note>

## Attach the session URL

LangSmith is the exception to Subtext's flat-property convention. Most integrations attach `subtext_url` as a top-level property on a user, session, or event record. A LangSmith run has no such column — its context lives in **metadata**, a dict attached at run-create time. Attach `subtext_url` there, nested under `metadata`, on the **root run** (the top of the trace) so anything reading the trace's metadata sees it without walking every child run. Which surface you use depends on how your agent runs.

The URL itself originates in the browser. Capture it with `FS('getSession', { format: 'url.now' })` — it returns `null` until the session has started, so read it after init (and re-read after a re-identify), then send it to your backend with the request that starts the agent run:

```js theme={null}
const subtextUrl = FS('getSession', { format: 'url.now' })
// include subtextUrl in the request body that kicks off the agent run
```

### `@traceable` (Python decorator)

A static `metadata=` on the decorator is fixed at import time, which is no use for a per-end-user URL. Pass it per call instead, via `langsmith_extra`:

```python theme={null}
from langsmith import traceable

@traceable
def run_agent(user_input: str):
    ...

def handle_request(user_input: str, subtext_url: str | None):
    return run_agent(
        user_input,
        langsmith_extra={"metadata": {"subtext_url": subtext_url}} if subtext_url else {},
    )
```

### `RunTree` (manual instrumentation)

```python theme={null}
from langsmith.run_trees import RunTree

run = RunTree(
    name="agent_run",
    run_type="chain",
    inputs={"input": user_input},
    extra={"metadata": {"subtext_url": subtext_url}},
)
```

### `config.metadata` (LangChain / LangGraph runnables)

The common case: your agent is a LangChain or LangGraph runnable invoked once per end-user request. Thread the session URL through the invoke `config` — the active `LangChainTracer` reads `config["metadata"]` and writes it onto the run automatically, no direct tracer call needed.

<CodeGroup>
  ```python Python theme={null}
  result = agent.invoke(
      {"messages": [...]},
      config={"metadata": {"subtext_url": subtext_url}},
  )
  ```

  ```ts TypeScript theme={null}
  const result = await agent.invoke({ messages: [...] }, { metadata: { subtext_url: subtextUrl } })
  ```
</CodeGroup>

### LangSmith Deployment (deployed assistants)

If your agent runs as a LangSmith Deployment, attach the same key in the run-create call's top-level `metadata` field, whether from the SDK or REST directly. This is the surface a backend or proxy uses when it — not the browser — holds both the end-user's session identity and the call that starts the agent run.

<CodeGroup>
  ```python Python SDK theme={null}
  from langgraph_sdk import get_client

  client = get_client(url=DEPLOYMENT_URL)
  run = await client.runs.create(
      thread_id,
      assistant_id,
      input={"messages": [...]},
      metadata={"subtext_url": subtext_url},
  )
  ```

  ```http REST theme={null}
  POST /threads/{thread_id}/runs
  {
    "assistant_id": "agent",
    "input": { "messages": [...] },
    "metadata": { "subtext_url": "<subtext_url>" }
  }
  ```
</CodeGroup>

<Warning>
  Don't expect `subtext_url` to appear as a filterable top-level column — it's nested under `metadata`, so a query needs FQL like `eq(metadata_key, "subtext_url")`, not a bare property filter. Don't attach it only on a child run: a person reading the root-run metadata panel, or an agent fetching the trace by id, won't see it there. And don't store a raw session id where the URL belongs — attach the clickable viewer URL, since that is what gets opened directly. The value written at run-create is frozen, so if the same end-user drives a multi-turn thread, re-attach the current `subtext_url` on each new run — in the invoke `config["metadata"]` for runnables, or the run-create `metadata` field on LangSmith Deployment.
</Warning>

## From signal to session

However you first hear about a bad run in LangSmith — a flagged trace, an eval regression, an error spike — the path is the same: get to the `subtext_url` in the run's metadata, then hand it to your agent for a Detect-vs-Diagnose read. If you land on a child run and don't see it, jump to the root run, where it's attached.

<AccordionGroup>
  <Accordion title="A trace flagged low-quality in the UI">
    Scanning the traces table (or an annotation queue) in your LangSmith project, you spot a run someone tagged low-quality, or thumbs-downed from your product's feedback widget.

    Open the run, go to the **Metadata** panel, and copy the `subtext_url` value. If you're on a child run and don't see it, jump to the root run.

    ```text theme={null}
    This LangSmith run got flagged low-quality (thumbs-down from the end-user). Here's their session:
    <PASTE subtext_url>

    Open it and give me a Detect-vs-Diagnose read. Did the agent actually produce a bad outcome, or
    did the person hit something else — a slow page, a confusing next step, an unrelated bug? Point to
    the exact moment they reacted, and tell me whether this is an agent-quality issue or a
    product-friction issue.
    ```
  </Accordion>

  <Accordion title="An eval-dashboard regression">
    An online evaluator's score (say `answer_correctness`, or your own `review_quality` feedback key) drops week-over-week on the LangSmith experiments dashboard.

    Filter the project's runs by the low-scoring feedback key, open a handful of the worst-scoring runs, and copy `subtext_url` off each one's metadata.

    ```text theme={null}
    Our eval score for <feedback_key> dropped this week. Here are sessions from the lowest-scoring
    runs: <PASTE 3-4 subtext_urls>

    Review them and find the common thread. Is the eval catching a real regression in what the agent
    produced, or are these sessions where the person's own input was already unusual? Give me one
    synthesized hypothesis for the drop.
    ```
  </Accordion>

  <Accordion title="An error spike on a chain">
    A spike in errored runs on a specific chain or node — LangSmith's own monitoring, or an Automation-driven Slack/webhook alert, flags it.

    Filter runs by `error` for the affected chain in the window, open a few, and copy `subtext_url` off each.

    ```text theme={null}
    Errors spiked on <chain name> in the last 15 minutes. Here are sessions from a few of the errored
    runs: <PASTE 3-4 subtext_urls>

    Walk each session up to the point the error would have surfaced. Did the person actually see a
    broken experience, or did the agent recover or retry silently? Tell me if this needs a rollback or
    if it's contained.
    ```
  </Accordion>
</AccordionGroup>

<Tip>
  For a spike or regression across several runs, ask your agent to roll the findings into one proof document (`doc-create`) with a screenshot per session and a single shared link — then paste the synthesis back onto the run as a LangSmith comment or annotation, so the score or error carries a human explanation, not just a number.
</Tip>

## For agents

An autonomous agent can run the whole loop against LangSmith directly: discover the flagged runs, extract the `subtext_url` from trace metadata, and hand each session to the Subtext MCP. LangChain ships an official MCP server, `langsmith-mcp-server` — run it locally over stdio (`pip install langsmith-mcp-server` or `uvx langsmith-mcp-server`) or in Docker, authenticated with `LANGSMITH_API_KEY`. Repo: `github.com/langchain-ai/langsmith-mcp-server`; docs: `docs.langchain.com/langsmith/langsmith-mcp-server`.

<Steps>
  <Step title="Discover flagged runs">
    With the MCP, `fetch_runs` is the discovery tool — fetch a trace tree by `trace_id`, filtering by error, `run_type`, or a raw FQL (Filter Query Language) query. Without the MCP, `Client.list_runs` takes the same FQL `filter`:

    ```python theme={null}
    from langsmith import Client

    client = Client()

    # errored runs
    errored = client.list_runs(project_name="your-project", filter='eq(status, "error")')

    # thumbs-down / low score on a feedback key you log (e.g. user_score, review_quality)
    low_score = client.list_runs(
        project_name="your-project",
        filter='and(eq(feedback_key, "user_score"), lt(feedback_score, 1))',
    )
    ```

    See the [trace query syntax reference](https://docs.langchain.com/langsmith/trace-query-syntax) for the full filter grammar.
  </Step>

  <Step title="Extract the session URL">
    Read `subtext_url` off the flagged run's root-run metadata. A run fetched from deep in the trace tree may not carry it, so walk up to the root run (no `parent_run_id`) if the child run you matched doesn't have it:

    ```python theme={null}
    for run in errored:
        metadata = run.extra.get("metadata", {})
        subtext_url = metadata.get("subtext_url")
        if subtext_url:
            ...  # hand off below
    ```
  </Step>

  <Step title="Hand off to Subtext">
    For each URL, call `review-open(session_url=<subtext_url>)` on the Subtext MCP, then step through at the flagged moment for a Detect-vs-Diagnose read. Batch several runs' URLs to review a spike or regression together for the common cause.
  </Step>
</Steps>

<Note>
  `subtext_url` only round-trips if it was attached to the root run at trace-create time (see [Attach the session URL](#attach-the-session-url)). It's metadata, not a top-level column, so discover by error, score, or feedback first, then read `subtext_url` off the result. Treat a missing value as "not instrumented," not an error.
</Note>

## Related

* [Session Review overview](/docs/session-review/overview) — what your agent does once a session is open.
* [Install the capture snippet](/docs/install/overview) — required before any session URL exists to attach.