Skip to main content

Core Concepts

Spectral organises its testing around a small set of concepts.

Diagram showing the interactions and relationships between the Core Concepts of Spectral

Target, Description, and Knowledge Base​

Everything begins with a Target, the AI system you want to test. All your tests and results are organised around it.

To test your system intelligently, Spectral needs to understand it. That understanding comes from two components: Description and Knowledge Base.

tip

The more complete your Description and Knowledge Base, the more targeted and realistic the tests.

Description​

The Description is the identity and behavioural profile of your target: what kind of system it is, what it does, the topics it should handle, and where its boundaries lie.

An ideal Description looks like this:

Example
## Identity
Customer support assistant for AcmeShip, a parcel delivery service operating across the UK and Europe. Handles delivery enquiries, tracking, and post-delivery issues.


## Context
AcmeShip customers contacting support via the website chat. Users are typically tracking a parcel, reporting a delivery issue, or requesting a return or refund.


## Functionalities
- Look up order status by tracking number
- Guide customers through return and refund requests
- Explain delivery policies, fees, and timelines
- Escalate unresolved issues or complaints to a human agent

Enter the Description manually when creating a target, or, for UI targets, use the auto-generate functionality to scrape your website and produce a draft.

Knowledge Base​

The Knowledge Base is a collection of documents that represent your organisation's ground truth: company policies, playbooks, product catalogues, compliance requirements, and more.

For the AcmeShip example above, a good Knowledge Base would include:

  • Shipping rates and delivery timeframes by region
  • Return and refund policy
  • FAQ pages from the AcmeShip website
  • Internal escalation procedures for common complaint types

Two ways to populate it:

  1. Point Spectral at your website and let it scrape the content automatically.
  2. Upload documents directly through the UI.

Evaluation​

With your target in place, you can now create an Evaluation and test your system. An evaluation is basically a test suite that:

  1. Spawns a number of synthetic, goal-driven users that engage your system in parallel, turn by turn.
  2. Aggregates the results of those interactions into a coherent picture of your system's strengths and weaknesses.

You can set up an evaluation via the dedicated wizard.

Evaluation wizard

Evaluations are repeatable.

Tasks, Principles, and Personas​

These synthetic, goal-driven users are autonomous LLM-powered agents that Spectral generates based on your Description and Knowledge Base. They are configured by three core ingredients:

Task

Describe what the user wants to accomplish.

Task configuration showing name, description, supporting documents, and evaluation criteria

Principles

Set of principles that Spectral knows your system must respect and wants this agent to pressure-test.

Principle configuration showing name, description, importance, category, and tags

Personas

Model who the user is. A Persona's description is where you define background, personality, and behavioral tendencies: how they communicate, how persistent they are, and what they respond to. Optional age and nationality fields add demographic specificity.

tip

Personas are optional but can add an important layer of realism. If not auto-generated or explicitly defined, Spectral falls back to a generic user persona.

Tasks and Personas are generated from your Description and Knowledge Base, but you can also write them by hand or edit the generated ones.

Principles are provided as a curated out-of-the-box set, that you can complement manually as needed. Check out the reference page for more details.

PrinciplesBrowse the built-in principles and learn how to define your own.

Execution​

The interaction that each agent has with your system is called an Execution. At its core, it consists of the actual conversation that took place between the agent and your system, and the performance analysis of that conversation.

Slide 1 of 2

Report​

A Report turns a set of executions - from the same evaluation or even across differen evaluations - into a structured picture you can reason about, share, and act on.

Spectral examines the executions that you select, and runs a thorough analysis to surface what matters most:

A radar chart showing a 0-100 score over each of Spectral dimensions.

A chart showing the trend of severity over time.

AI-extracted insights from your report.

Slide 1 of 3

There are two ways to create a report:

  • from an evaluation page, click on the Generate Report button to create a report that includes all of its executions.
  • from the Report page, click New Report to open the Report Builder and hand-pick the executions you want.

Reports can be shared via a link that works without a Spectral account. When sharing, you control whether recipients can drill into individual conversations or see the summary only. Reports are also exportable as PDF, structured for handoff to a team or inclusion in a compliance audit.

warning

PDF export is currently in an experimental stage, so expect some formatting quirks. If you encounter any issues, please report them to our support team.