> For the complete documentation index, see [llms.txt](https://docs.usepylon.com/pylon-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.usepylon.com/pylon-docs/agents/support-agent/test.md).

# Test

Testing makes sure you can be confident your AI Agent will behave as you want it to.

## Custom Chat

Simulate an interaction with the AI and see how it handles different questions. Either choose from one of the sample questions or ask a question directly.

<figure><img src="https://982123570-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F9hXBoCTTNw5flARlUVJf%2Fuploads%2FdIzYezXMyqjxCdEiJTcc%2Fplayground_single.gif?alt=media&amp;token=d23be15e-6a6b-4d1d-b013-99c0c5bf985c" alt="" width="563"><figcaption></figcaption></figure>

To test agent behavior for a specific customer or channel, you can select your desired values for Channel, Account, and Contact.

## Backtest

Backtesting shows how your agent would have handled real past issues. Use it to check a new agent before you deploy it, or to confirm that a change to guidance, skills, or resources actually helped.

To start, go to **Settings > AI controls > Agents** and click **Backtest**, or click **Backtest** on an agent's **Variations** page.

#### Run a backtest

1. Click **New Backtest** and give it a name.
2. **Select test issues.** Choose one:
   * **Custom selection**: Filter your past issues and pick the ones to test.
   * **Issues with bad AI responses**: Start with issues where an agent's reply was rated poorly. These are the issues most worth fixing.
   * **Existing dataset**: Reuse a saved set of issues (see Datasets).
3. **Select which agent to use.** Choose up to 3 agents or variations to run against the same issues, so you can compare them side by side.
4. Optionally, turn on **Grade with Agent QA** to score every reply with the same rubric Agent QA uses on live issues.
5. Click **Start Backtest**.

Backtests never message customers or change the original issues.

#### Review results

For each issue, you can see what the agent replied and what it did. That includes the actions it took, whether it escalated, and its reasoning. Each result shows how the agent handled the issue:

* **Replied only**: The agent answered without escalating.
* **Replied + escalated**: The agent answered, then handed off to your team.
* **Escalated**: The agent handed off to your team.
* **Silent escalation**: The agent handed off without replying to the customer.

If you turned on **Grade with Agent QA**, each reply also gets a score, with pass rates and critical fails for each agent. Click **Review Issues** to go through results one by one. Confirm or change each grade with **Pass**, **Fail**, or **N/A**, and add a note explaining why.

{% content-ref url="/pages/nE60ROBDO9ZBYKfi8avZ" %}
[Quality Assurance](/pylon-docs/agents/support-agent/monitor-and-iterate/quality-assurance.md)
{% endcontent-ref %}

#### Turn feedback into fixes

When a reply isn't right, tell Pylon what went wrong, such as **incorrect information**, **too wordy**, or **wrong action**. Describe what the agent should have said or done instead, and Pylon suggests updates to the agent's guidance, skills, or knowledge base articles.

After you make a change, click **Re-run Backtest** to check that it fixed the problem without breaking anything else.

#### Datasets

A dataset is a saved set of past issues you can reuse across backtests. Reusing the same set makes it easy to compare results over time. Each dataset holds up to 1 To create one, open the **Datasets** tab and n select issues to add. To build a dataset from issues an agent has already worked on, filter by **Support agent history**. That includes issues the agent handled in QA mode.&#x20;

#### Variations&#x20;

A variation is an alternate version of an agerent guidance, skills, or tools. Create avariation from the agent's **Variations** page, backtest it against the current version, and switch to whichever one performs better.&#x20;

{% hint style="success" %}
**Tip:** Keep a dataset of your 50 to 100 most common or most important issues. Re-run it every time you change your agent, to make sure the change helps and nothing else gets worse.
{% endhint %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.usepylon.com/pylon-docs/agents/support-agent/test.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.