# Sourcebot

Sourcebot Docs

## Getting Started

- [What is agenteval?](https://help.newtools.space/what-is-agenteval.md) - agenteval is a framework-agnostic Python toolkit that scores LLM agents with a statistical pass rate instead of a binary unit-test result. You assert on tool use and behavior across repeated runs, then track that pass rate across releases.
- [Quickstart](https://help.newtools.space/quickstart-e8b4c85e.md) - Install agenteval-py, write a test with tracer.wrap and tracer.assert_that, and get a 10-run pass rate from the CLI.
- [Agent Guide](https://help.newtools.space/agent-guide-3895cb0a.md) - Signatures, tables, and copy-paste commands for wiring @agenteval.test, Tracer, AssertionSet, adapters, and the CLI from a single page.

## Core Concepts

- [How agenteval Works](https://help.newtools.space/how-it-works.md) - See the four moving parts of agenteval, Tracer, Runner, AssertionSet, and Reporter, and how they turn repeated agent runs into a pass rate.
- [The Inbound/Outbound Email Model](https://help.newtools.space/email-model.md) - Explains the normalized email object every Primitive SDK returns and the receive → inspect → send/reply/forward workflow, including wait mode and the four terminal delivery statuses.
- [Webhook Contract and Event Types](https://help.newtools.space/webhook-contract.md) - The webhook contract is the canonical email.received JSON Schema plus a shared signature scheme and event-type catalog that every Primitive SDK validates and generates code from identically.
- [x402 Payments Model](https://help.newtools.space/x402-payments-model.md) - The non-custodial payment flow shared identically by every Primitive SDK and the CLI: register a payout address, issue a challenge, sign it locally, and settle on chain, all governed by an org-level spend policy.
- [The @agenteval.test decorator](https://help.newtools.space/test-decorator.md) - Decorate a test function so the Runner executes it N times and scores a pass rate against your threshold. Configure n, threshold, and tags for async or sync functions.

## Integrations

- [Framework adapters](https://help.newtools.space/adapters-overview.md) - Decide whether to wrap tools with tracer.wrap or use an OpenAI, Anthropic, or LangChain adapter. This page covers the shared wrap_tools and extract_token_usage contract and when to skip adapters.
- [Instrument a LangChain agent](https://help.newtools.space/langchain-adapter.md) - Record LangChain tool start, end, and error events automatically by passing AgentEvalCallbackHandler into invoke or ainvoke. Skip per-tool tracer.wrap() when the agent already runs through LangChain.

## Reference

- [CLI Reference](https://help.newtools.space/cli-reference-902a307e.md) - Lookup for every agenteval run flag, exit code, and the JSON report written by --output. Use it when you wire the CLI into a script or a CI step.
- [Python API Reference](https://help.newtools.space/python-api-reference.md) - Signatures, parameters, and defaults for every name the agenteval package exports, from @agenteval.test and run_suite to Tracer, AssertionSet, and RichReporter.
- [Installation and extras](https://help.newtools.space/installation-and-extras.md) - Choose the pip command for agenteval-py and the extra that matches your stack so you get the CLI, the import, and only the adapter or dev dependencies you need.

## Troubleshooting

- [Errors and Exceptions](https://help.newtools.space/errors-and-exceptions.md) - Diagnose DiscoveryError, collected assertion failures, and TracerError by matching the message you see to the code that raised it. Each section states the cause and the concrete fix.
- [Common issues and FAQ](https://help.newtools.space/faq.md) - Fix tests that never show up, pass rates that swing under concurrency, and wide confidence intervals that make a green threshold unreliable.

## Guides

- [Reporting Results](https://help.newtools.space/reporting-results.md) - Print color-coded pass rates with RichReporter after every suite, then export the same SuiteResult as JSON for dashboards.
- [Using agenteval in CI](https://help.newtools.space/ci-integration.md) - Fail the GitHub Actions build when an agent drops below its pass-rate threshold, and keep eval-report.json as an artifact you can inspect after the job ends.

## other

- [Crawl an entire website](https://help.newtools.space/crawl-an-entire-website.md) - Use the Crawl endpoint to fetch data for every page on a given website.
- [Scrape a web page](https://help.newtools.space/scrape-a-web-page.md) - Use the Scrape endpoint playground to fetch LLM-ready content from a single URL.
- [Run an AI agent to gather structured data](https://help.newtools.space/run-an-ai-agent-to-gather-structured-data.md) - Use the Agent feature to run a natural-language data-gathering task across multiple URLs or a prompt.

## Optional

- [Full context](https://help.newtools.space/llms-full.txt) - Complete documentation for all articles
- [JSON index](https://help.newtools.space/llms.json) - Machine-readable article index