# Future AGI Documentation > Future AGI is an AI lifecycle platform for building, evaluating, observing, and optimizing AI applications. This documentation covers the Python SDK, platform features, integrations, and API reference. ## Docs ### Get Started - [Overview](https://docs.futureagi.com/docs) - [Send your first trace](https://docs.futureagi.com/docs/get-started/send-your-first-trace) - [Route your first LLM request](https://docs.futureagi.com/docs/get-started/route-your-first-llm-request) - [Connect no code agents](https://docs.futureagi.com/docs/get-started/connect-no-code-agents) - [Create your first prompt](https://docs.futureagi.com/docs/get-started/create-your-first-prompt) - [Migrate from Langfuse](https://docs.futureagi.com/docs/get-started/migrate/langfuse) - [Overview](https://docs.futureagi.com/docs/self-hosting) - [Requirements](https://docs.futureagi.com/docs/self-hosting/requirements) - [Installation](https://docs.futureagi.com/docs/self-hosting/installation) - [System configuration](https://docs.futureagi.com/docs/self-hosting/configuration/system) - [Environment variables](https://docs.futureagi.com/docs/self-hosting/configuration/environment) - [Profiles](https://docs.futureagi.com/docs/self-hosting/configuration/profiles) - [Launch mode](https://docs.futureagi.com/docs/self-hosting/configuration/launch-mode) - [Overview](https://docs.futureagi.com/docs/self-hosting/production) - [Checklist](https://docs.futureagi.com/docs/self-hosting/production/checklist) - [Security & TLS](https://docs.futureagi.com/docs/self-hosting/production/security-tls) - [Backups & restore](https://docs.futureagi.com/docs/self-hosting/production/backups-restore) - [Monitoring](https://docs.futureagi.com/docs/self-hosting/production/monitoring) - [Upgrades & rollback](https://docs.futureagi.com/docs/self-hosting/production/upgrades-rollback) - [Troubleshooting & FAQs](https://docs.futureagi.com/docs/self-hosting/troubleshooting) - [Support](https://docs.futureagi.com/docs/self-hosting/support) - [What's new](https://docs.futureagi.com/docs/release-notes) ### Agent Playground - [Overview](https://docs.futureagi.com/docs/agent-playground) - [Understanding Agent Playground](https://docs.futureagi.com/docs/agent-playground/concepts/understanding-agent-playground) - [Versions & Execution](https://docs.futureagi.com/docs/agent-playground/concepts/versions-and-execution) - [Create an agent](https://docs.futureagi.com/docs/agent-playground/guides/create-agent) - [Overview](https://docs.futureagi.com/docs/agent-playground/guides/build-workflow) - [Configure an LLM Prompt node](https://docs.futureagi.com/docs/agent-playground/guides/build-workflow/configure-an-llm-prompt-node) - [Configure an Agent node](https://docs.futureagi.com/docs/agent-playground/guides/build-workflow/configure-an-agent-node) - [Set input variables](https://docs.futureagi.com/docs/agent-playground/guides/build-workflow/set-input-variables) - [Run an agent](https://docs.futureagi.com/docs/agent-playground/guides/run-an-agent) - [Manage versions](https://docs.futureagi.com/docs/agent-playground/guides/manage-versions) - [Limits & rules](https://docs.futureagi.com/docs/agent-playground/reference/limits-and-rules) - [Agent Playground FAQ & fixes](https://docs.futureagi.com/docs/agent-playground/troubleshooting) ### Annotations - [Overview](https://docs.futureagi.com/docs/annotations) - [Understanding Annotation](https://docs.futureagi.com/docs/annotations/concepts/understanding-annotation) - [Labels](https://docs.futureagi.com/docs/annotations/concepts/labels) - [Queues & Items](https://docs.futureagi.com/docs/annotations/concepts/queues-and-items) - [Scores](https://docs.futureagi.com/docs/annotations/concepts/scores) - [Create a label](https://docs.futureagi.com/docs/annotations/guides/create-label) - [Create a queue](https://docs.futureagi.com/docs/annotations/guides/create-queue) - [Overview](https://docs.futureagi.com/docs/annotations/guides/explore-queue) - [Add items](https://docs.futureagi.com/docs/annotations/guides/explore-queue/add-items) - [Track progress & agreement](https://docs.futureagi.com/docs/annotations/guides/explore-queue/progress-and-agreement) - [Automate item intake](https://docs.futureagi.com/docs/annotations/guides/explore-queue/automate-item-intake) - [Annotate items](https://docs.futureagi.com/docs/annotations/guides/annotate-items) - [Review submissions](https://docs.futureagi.com/docs/annotations/guides/review-submissions) - [Annotate without a queue](https://docs.futureagi.com/docs/annotations/guides/annotate-without-a-queue) - [Export annotations](https://docs.futureagi.com/docs/annotations/guides/export-annotations) - [Label types & values](https://docs.futureagi.com/docs/annotations/reference/label-types-and-values) - [Queue settings & limits](https://docs.futureagi.com/docs/annotations/reference/queue-settings-and-limits) - [SDK & API](https://docs.futureagi.com/docs/annotations/reference/sdk-api) - [Annotation FAQ & fixes](https://docs.futureagi.com/docs/annotations/troubleshooting) ### Agent Command Center - [Overview](https://docs.futureagi.com/docs/command-center) - [How it works](https://docs.futureagi.com/docs/command-center/concepts/core) - [Virtual keys & access control](https://docs.futureagi.com/docs/command-center/concepts/virtual-keys) - [Configuration](https://docs.futureagi.com/docs/command-center/concepts/configuration) - [Platform integration](https://docs.futureagi.com/docs/command-center/concepts/platform-integration) - [Supported providers](https://docs.futureagi.com/docs/command-center/features/providers) - [Self-hosted models](https://docs.futureagi.com/docs/command-center/features/self-hosted-models) - [Endpoints overview](https://docs.futureagi.com/docs/command-center/api/endpoints) - [Chat completions](https://docs.futureagi.com/docs/command-center/api/chat) - [Embeddings & reranking](https://docs.futureagi.com/docs/command-center/api/embeddings) - [Media endpoints](https://docs.futureagi.com/docs/command-center/api/media) - [Assistants API](https://docs.futureagi.com/docs/command-center/api/assistants) - [Files & vector stores](https://docs.futureagi.com/docs/command-center/api/files) - [Async & batch](https://docs.futureagi.com/docs/command-center/api/async-batch) - [Request & response headers](https://docs.futureagi.com/docs/command-center/api/headers) - [Routing & reliability](https://docs.futureagi.com/docs/command-center/features/routing) - [Guardrails](https://docs.futureagi.com/docs/command-center/features/guardrails) - [Caching](https://docs.futureagi.com/docs/command-center/features/caching) - [Rate limiting](https://docs.futureagi.com/docs/command-center/features/rate-limiting) - [Cost tracking](https://docs.futureagi.com/docs/command-center/features/cost-tracking) - [Observability](https://docs.futureagi.com/docs/command-center/features/observability) - [Shadow experiments](https://docs.futureagi.com/docs/command-center/features/shadow-experiments) - [Webhooks](https://docs.futureagi.com/docs/command-center/features/webhooks) - [Custom Properties](https://docs.futureagi.com/docs/command-center/features/custom-properties) - [MCP & A2A](https://docs.futureagi.com/docs/command-center/features/mcp-a2a) - [Organization management](https://docs.futureagi.com/docs/command-center/admin/organizations) - [Self-hosted](https://docs.futureagi.com/docs/command-center/deployment/self-hosted) - [Error handling](https://docs.futureagi.com/docs/command-center/guides/errors) - [Troubleshooting](https://docs.futureagi.com/docs/command-center/guides/troubleshooting) ### Dataset - [Overview](https://docs.futureagi.com/docs/dataset) - [Understanding Datasets](https://docs.futureagi.com/docs/dataset/concepts/understanding-datasets) - [Static & Dynamic Columns](https://docs.futureagi.com/docs/dataset/concepts/static-and-dynamic-columns) - [Synthetic Data](https://docs.futureagi.com/docs/dataset/concepts/synthetic-data) - [Create a dataset](https://docs.futureagi.com/docs/dataset/guides/create-a-dataset) - [Add rows](https://docs.futureagi.com/docs/dataset/guides/add-rows) - [Add columns](https://docs.futureagi.com/docs/dataset/guides/add-columns) - [Run a prompt on every row](https://docs.futureagi.com/docs/dataset/guides/run-a-prompt-on-every-row) - [Run an experiment](https://docs.futureagi.com/docs/dataset/guides/run-an-experiment) - [Manage datasets](https://docs.futureagi.com/docs/dataset/guides/manage-datasets) - [Limits & Data Types](https://docs.futureagi.com/docs/dataset/reference/limits-and-data-types) - [Dynamic column methods](https://docs.futureagi.com/docs/dataset/reference/dynamic-column-methods) - [Dataset FAQ & fixes](https://docs.futureagi.com/docs/dataset/troubleshooting) ### Error Feed - [Overview](https://docs.futureagi.com/docs/error-feed) - [Understanding Error Feed](https://docs.futureagi.com/docs/error-feed/concepts/understanding-error-feed) - [Severity & Status](https://docs.futureagi.com/docs/error-feed/concepts/severity-and-status) - [Trace error analysis](https://docs.futureagi.com/docs/error-feed/concepts/trace-error-analysis) - [Turn on Error Feed](https://docs.futureagi.com/docs/error-feed/guides/turn-on-error-feed) - [Triage issues](https://docs.futureagi.com/docs/error-feed/guides/triage-issues) - [Investigate an issue](https://docs.futureagi.com/docs/error-feed/guides/investigate-an-issue) - [Run a root cause analysis](https://docs.futureagi.com/docs/error-feed/guides/run-root-cause-analysis) - [Create a Linear issue](https://docs.futureagi.com/docs/error-feed/guides/create-linear-issue) - [Issue fields & filters](https://docs.futureagi.com/docs/error-feed/reference/issue-fields) - [Error taxonomy](https://docs.futureagi.com/docs/error-feed/reference/error-taxonomy) - [No issues in the feed](https://docs.futureagi.com/docs/error-feed/troubleshooting/no-issues-in-the-feed) - [Issue counts look wrong](https://docs.futureagi.com/docs/error-feed/troubleshooting/issue-counts-look-wrong) - [Analysis doesn't finish](https://docs.futureagi.com/docs/error-feed/troubleshooting/analysis-does-not-finish) ### Evaluation - [Overview](https://docs.futureagi.com/docs/evaluation) - [Understanding Evaluation](https://docs.futureagi.com/docs/evaluation/concepts/understanding-evaluation) - [Eval types](https://docs.futureagi.com/docs/evaluation/concepts/eval-types) - [Eval templates & versions](https://docs.futureagi.com/docs/evaluation/concepts/eval-templates) - [Evaluator models](https://docs.futureagi.com/docs/evaluation/concepts/evaluator-models) - [Error localization](https://docs.futureagi.com/docs/evaluation/concepts/error-localization) - [Guardrails](https://docs.futureagi.com/docs/evaluation/concepts/guardrails) - [Composite evals](https://docs.futureagi.com/docs/evaluation/concepts/composite-evals) - [Feedback](https://docs.futureagi.com/docs/evaluation/concepts/feedback) - [Ground truth](https://docs.futureagi.com/docs/evaluation/concepts/ground-truth) - [Running Evaluations](https://docs.futureagi.com/docs/evaluation/guides/running-evaluations) - [The Evaluations page](https://docs.futureagi.com/docs/evaluation/guides/explore-playground) - [Test an eval](https://docs.futureagi.com/docs/evaluation/guides/explore-playground/test-an-eval) - [Usage & analytics](https://docs.futureagi.com/docs/evaluation/guides/explore-playground/usage-analytics) - [Create a custom eval](https://docs.futureagi.com/docs/evaluation/guides/custom-evals) - [Build a composite evals](https://docs.futureagi.com/docs/evaluation/guides/composite-evals) - [Set up guardrails](https://docs.futureagi.com/docs/evaluation/guides/guardrails) - [Add ground truth](https://docs.futureagi.com/docs/evaluation/guides/ground-truth) - [Collect feedback](https://docs.futureagi.com/docs/evaluation/guides/collect-feedback) - [Use custom models](https://docs.futureagi.com/docs/evaluation/guides/custom-models) - [Evaluate in CI/CD](https://docs.futureagi.com/docs/evaluation/guides/cicd) - [Advanced usage](https://docs.futureagi.com/docs/evaluation/guides/advanced-usage) - [Overview](https://docs.futureagi.com/docs/evaluation/builtin) - [Context Adherence](https://docs.futureagi.com/docs/evaluation/builtin/context-adherence) - [Context Relevance](https://docs.futureagi.com/docs/evaluation/builtin/context-relevance) - [Completeness](https://docs.futureagi.com/docs/evaluation/builtin/completeness) - [Chunk Attribution](https://docs.futureagi.com/docs/evaluation/builtin/chunk-attribution) - [Chunk Utilization](https://docs.futureagi.com/docs/evaluation/builtin/chunk-utilization) - [Groundedness](https://docs.futureagi.com/docs/evaluation/builtin/groundedness) - [Detect Hallucination](https://docs.futureagi.com/docs/evaluation/builtin/detect-hallucination) - [Eval Ranking](https://docs.futureagi.com/docs/evaluation/builtin/eval-ranking) - [Recall@K](https://docs.futureagi.com/docs/evaluation/builtin/recall-at-k) - [Precision@K](https://docs.futureagi.com/docs/evaluation/builtin/precision-at-k) - [NDCG@K](https://docs.futureagi.com/docs/evaluation/builtin/ndcg-at-k) - [MRR](https://docs.futureagi.com/docs/evaluation/builtin/mrr) - [Hit Rate](https://docs.futureagi.com/docs/evaluation/builtin/hit-rate) - [Retrieval Metrics](https://docs.futureagi.com/docs/evaluation/builtin/retrieval-metrics) - [PII Detection](https://docs.futureagi.com/docs/evaluation/builtin/pii) - [Toxicity](https://docs.futureagi.com/docs/evaluation/builtin/toxicity) - [Sexist](https://docs.futureagi.com/docs/evaluation/builtin/sexist) - [Prompt Injection](https://docs.futureagi.com/docs/evaluation/builtin/prompt-injection) - [Data Privacy Compliance](https://docs.futureagi.com/docs/evaluation/builtin/data-privacy) - [Cultural Sensitivity](https://docs.futureagi.com/docs/evaluation/builtin/cultural-sensitivity) - [Bias Detection](https://docs.futureagi.com/docs/evaluation/builtin/bias-detection) - [No Racial Bias](https://docs.futureagi.com/docs/evaluation/builtin/no-racial-bias) - [No Gender Bias](https://docs.futureagi.com/docs/evaluation/builtin/no-gender-bias) - [No Age Bias](https://docs.futureagi.com/docs/evaluation/builtin/no-age-bias) - [Answer Refusal](https://docs.futureagi.com/docs/evaluation/builtin/answer-refusal) - [No Harmful Therapeutic Guidance](https://docs.futureagi.com/docs/evaluation/builtin/no-harmful-therapeutic-guidance) - [Clinically Inappropriate Tone](https://docs.futureagi.com/docs/evaluation/builtin/clinically-inappropriate-tone) - [Is Harmful Advice](https://docs.futureagi.com/docs/evaluation/builtin/is-harmful-advice) - [Conversation Coherence](https://docs.futureagi.com/docs/evaluation/builtin/conversation-coherence) - [Conversation Resolution](https://docs.futureagi.com/docs/evaluation/builtin/conversation-resolution) - [Evaluate Function Calling](https://docs.futureagi.com/docs/evaluation/builtin/llm-function-calling) - [Task Completion](https://docs.futureagi.com/docs/evaluation/builtin/task-completion) - [Customer Agent: Loop Detection](https://docs.futureagi.com/docs/evaluation/builtin/customer-agent-loop-detection) - [Customer Agent: Context Retention](https://docs.futureagi.com/docs/evaluation/builtin/customer-agent-context-retention) - [Customer Agent: Query Handling](https://docs.futureagi.com/docs/evaluation/builtin/customer-agent-query-handling) - [Customer Agent: Termination Handling](https://docs.futureagi.com/docs/evaluation/builtin/customer-agent-termination-handling) - [Customer Agent: Interruption Handling](https://docs.futureagi.com/docs/evaluation/builtin/customer-agent-interruption-handling) - [Customer Agent: Conversation Quality](https://docs.futureagi.com/docs/evaluation/builtin/customer-agent-conversation-quality) - [Customer Agent: Objection Handling](https://docs.futureagi.com/docs/evaluation/builtin/customer-agent-objection-handling) - [Customer Agent: Language Handling](https://docs.futureagi.com/docs/evaluation/builtin/customer-agent-language-handling) - [Customer Agent: Human Escalation](https://docs.futureagi.com/docs/evaluation/builtin/customer-agent-human-escalation) - [Customer Agent: Clarification Seeking](https://docs.futureagi.com/docs/evaluation/builtin/customer-agent-clarification-seeking) - [Customer Agent: Prompt Conformance](https://docs.futureagi.com/docs/evaluation/builtin/customer-agent-prompt-conformance) - [Customer Agent: Task Completion](https://docs.futureagi.com/docs/evaluation/builtin/customer-agent-task-completion) - [Conversation Hallucination](https://docs.futureagi.com/docs/evaluation/builtin/conversation-hallucination) - [Tool Call Accuracy](https://docs.futureagi.com/docs/evaluation/builtin/tool-call-accuracy) - [Trajectory Match](https://docs.futureagi.com/docs/evaluation/builtin/trajectory-match) - [Step Count](https://docs.futureagi.com/docs/evaluation/builtin/step-count) - [Tone](https://docs.futureagi.com/docs/evaluation/builtin/tone) - [Instruction Adherence](https://docs.futureagi.com/docs/evaluation/builtin/instruction-adherence) - [Summary Quality](https://docs.futureagi.com/docs/evaluation/builtin/summary-quality) - [Translation Accuracy](https://docs.futureagi.com/docs/evaluation/builtin/translation-accuracy) - [No LLM Reference](https://docs.futureagi.com/docs/evaluation/builtin/no-llm-reference) - [No Apologies](https://docs.futureagi.com/docs/evaluation/builtin/no-apologies) - [Is Polite](https://docs.futureagi.com/docs/evaluation/builtin/is-polite) - [Is Concise](https://docs.futureagi.com/docs/evaluation/builtin/is-concise) - [Is Helpful](https://docs.futureagi.com/docs/evaluation/builtin/is-helpful) - [Is Good Summary](https://docs.futureagi.com/docs/evaluation/builtin/is-good-summary) - [Is Informal Tone](https://docs.futureagi.com/docs/evaluation/builtin/is-informal-tone) - [Contains Code](https://docs.futureagi.com/docs/evaluation/builtin/is-code) - [Text to SQL](https://docs.futureagi.com/docs/evaluation/builtin/text-to-sql) - [Is JSON](https://docs.futureagi.com/docs/evaluation/builtin/is-json) - [One Line](https://docs.futureagi.com/docs/evaluation/builtin/contain-evals) - [Contains Valid Link](https://docs.futureagi.com/docs/evaluation/builtin/contains-valid-link) - [Is Email](https://docs.futureagi.com/docs/evaluation/builtin/is-email) - [No Invalid Links](https://docs.futureagi.com/docs/evaluation/builtin/no-invalid-links) - [Is Refusal](https://docs.futureagi.com/docs/evaluation/builtin/is-refusal) - [Code & Output Validation Checks](https://docs.futureagi.com/docs/evaluation/builtin/code-output-validation-checks) - [Fuzzy Match](https://docs.futureagi.com/docs/evaluation/builtin/fuzzy-match) - [Ground Truth Match](https://docs.futureagi.com/docs/evaluation/builtin/ground-truth-match) - [BLEU Score](https://docs.futureagi.com/docs/evaluation/builtin/bleu) - [ROUGE Score](https://docs.futureagi.com/docs/evaluation/builtin/rouge) - [Levenshtein Similarity](https://docs.futureagi.com/docs/evaluation/builtin/lavenshtein-similarity) - [Numeric Similarity](https://docs.futureagi.com/docs/evaluation/builtin/numeric-similarity) - [Embedding Similarity](https://docs.futureagi.com/docs/evaluation/builtin/embedding-similarity) - [Semantic List Contains](https://docs.futureagi.com/docs/evaluation/builtin/semantic-list-contains) - [Similarity & Image-Quality Metrics](https://docs.futureagi.com/docs/evaluation/builtin/similarity-image-quality-metrics) - [Audio Transcription (ASR/STT)](https://docs.futureagi.com/docs/evaluation/builtin/audio-transcription) - [Audio Quality](https://docs.futureagi.com/docs/evaluation/builtin/audio-quality) - [TTS Accuracy](https://docs.futureagi.com/docs/evaluation/builtin/tts-accuracy) - [Audio & ASR Metrics](https://docs.futureagi.com/docs/evaluation/builtin/audio-asr-metrics) - [Dead Air Detection](https://docs.futureagi.com/docs/evaluation/builtin/dead-air-detection) - [Caption Hallucination](https://docs.futureagi.com/docs/evaluation/builtin/caption-hallucination) - [Synthetic Image Evaluator](https://docs.futureagi.com/docs/evaluation/builtin/synthetic-image-evaluator) - [OCR Evaluation](https://docs.futureagi.com/docs/evaluation/builtin/ocr-evaluation) - [FID Score](https://docs.futureagi.com/docs/evaluation/builtin/fid-score) - [CLIP Score](https://docs.futureagi.com/docs/evaluation/builtin/clip-score) - [Image Instruction Adherence](https://docs.futureagi.com/docs/evaluation/builtin/image-instruction-adherence) - [Statistical & Classification Metrics](https://docs.futureagi.com/docs/evaluation/builtin/statistical-classification-metrics) - [NLP & Text Metrics](https://docs.futureagi.com/docs/evaluation/builtin/nlp-text-metrics) - [Output types & scoring](https://docs.futureagi.com/docs/evaluation/reference/output-types) - [SDK & API](https://docs.futureagi.com/docs/evaluation/reference/sdk-api) - [Evaluation FAQ & fixes](https://docs.futureagi.com/docs/evaluation/troubleshooting) ### Falcon AI - [Overview](https://docs.futureagi.com/docs/falcon-ai) - [Understanding Falcon AI](https://docs.futureagi.com/docs/falcon-ai/concepts/understanding-falcon-ai) - [Skills](https://docs.futureagi.com/docs/falcon-ai/concepts/skills) - [MCP Connectors](https://docs.futureagi.com/docs/falcon-ai/concepts/mcp-connectors) - [Chat with Falcon AI](https://docs.futureagi.com/docs/falcon-ai/guides/chat-with-falcon-ai) - [Manage conversations](https://docs.futureagi.com/docs/falcon-ai/guides/manage-conversations) - [Create a skill](https://docs.futureagi.com/docs/falcon-ai/guides/create-skill) - [Connect an MCP server](https://docs.futureagi.com/docs/falcon-ai/guides/connect-mcp-server) - [Choose connector tools](https://docs.futureagi.com/docs/falcon-ai/guides/choose-connector-tools) - [Use the MCP Server in your IDE](https://docs.futureagi.com/docs/falcon-ai/guides/use-the-mcp-server) ### Knowledge Base - [Overview](https://docs.futureagi.com/docs/knowledge-base) - [Understanding Knowledge Base](https://docs.futureagi.com/docs/knowledge-base/concepts/understanding-knowledge-base) - [Create a knowledge base](https://docs.futureagi.com/docs/knowledge-base/guides/create-knowledge-base) - [Update a knowledge base](https://docs.futureagi.com/docs/knowledge-base/guides/update-knowledge-base) - [Manage with the SDK](https://docs.futureagi.com/docs/knowledge-base/guides/manage-with-the-sdk) ### Observability - [Overview](https://docs.futureagi.com/docs/observe) - [Quickstart](https://docs.futureagi.com/docs/observe/quickstart) - [Spans](https://docs.futureagi.com/docs/observe/concepts/spans) - [Traces](https://docs.futureagi.com/docs/observe/concepts/traces) - [Sessions](https://docs.futureagi.com/docs/observe/concepts/sessions) - [Users](https://docs.futureagi.com/docs/observe/concepts/users) - [Voice observability](https://docs.futureagi.com/docs/observe/concepts/voice-observability) - [Observability model](https://docs.futureagi.com/docs/observe/concepts/observability-model) - [Overview](https://docs.futureagi.com/docs/observe/guides/explore-dashboard) - [Filters](https://docs.futureagi.com/docs/observe/guides/explore-dashboard/filters) - [Views](https://docs.futureagi.com/docs/observe/guides/explore-dashboard/views) - [Display options](https://docs.futureagi.com/docs/observe/guides/explore-dashboard/display-options) - [Setup alerts](https://docs.futureagi.com/docs/observe/guides/setup-alerts) - [Setup evals](https://docs.futureagi.com/docs/observe/guides/setup-evals) - [Voice Observability](https://docs.futureagi.com/docs/observe/features/voice) - [Filters](https://docs.futureagi.com/docs/observe/reference/filters) - [Export and endpoints](https://docs.futureagi.com/docs/observe/reference/export-formats) - [traceAI](https://docs.futureagi.com/docs/observe/concepts/traceai) - [No traces appear](https://docs.futureagi.com/docs/observe/troubleshooting/no-traces-appearing) - [Missing spans or fields](https://docs.futureagi.com/docs/observe/troubleshooting/missing-attributes) - [Traces are noisy or incomplete](https://docs.futureagi.com/docs/observe/troubleshooting/noisy-or-incomplete-traces) - [Dashboard numbers look wrong](https://docs.futureagi.com/docs/observe/troubleshooting/dashboard-numbers-look-wrong) - [Alerts not firing](https://docs.futureagi.com/docs/observe/troubleshooting/alerts-did-not-fire) ### Optimization - [Overview](https://docs.futureagi.com/docs/optimization) - [Understanding optimization](https://docs.futureagi.com/docs/optimization/concepts/understanding-optimization) - [Choosing an optimizer](https://docs.futureagi.com/docs/optimization/concepts/choosing-an-optimizer) - [Run an optimization](https://docs.futureagi.com/docs/optimization/guides/run-an-optimization) - [Read optimization results](https://docs.futureagi.com/docs/optimization/guides/read-optimization-results) - [Optimize from the SDK](https://docs.futureagi.com/docs/optimization/guides/optimize-from-the-sdk) - [Overview](https://docs.futureagi.com/docs/optimization/reference/optimizers) - [Random Search](https://docs.futureagi.com/docs/optimization/reference/optimizers/random-search) - [Bayesian Search](https://docs.futureagi.com/docs/optimization/reference/optimizers/bayesian-search) - [ProTeGi](https://docs.futureagi.com/docs/optimization/reference/optimizers/protegi) - [Meta-Prompt](https://docs.futureagi.com/docs/optimization/reference/optimizers/meta-prompt) - [PromptWizard](https://docs.futureagi.com/docs/optimization/reference/optimizers/promptwizard) - [GEPA](https://docs.futureagi.com/docs/optimization/reference/optimizers/gepa) - [SDK & API](https://docs.futureagi.com/docs/optimization/reference/sdk-api) - [Optimization FAQ & fixes](https://docs.futureagi.com/docs/optimization/troubleshooting) ### Prompt - [Overview](https://docs.futureagi.com/docs/prompt) - [Understanding Prompts](https://docs.futureagi.com/docs/prompt/concepts/understanding-prompts) - [Versions & Labels](https://docs.futureagi.com/docs/prompt/concepts/versions-and-labels) - [Prompt Engineering](https://docs.futureagi.com/docs/prompt/concepts/prompt-engineering) - [Create a prompt](https://docs.futureagi.com/docs/prompt/guides/create-a-prompt) - [Run a prompt](https://docs.futureagi.com/docs/prompt/guides/run-a-prompt) - [Commit & compare versions](https://docs.futureagi.com/docs/prompt/guides/commit-and-compare-versions) - [Evaluate prompt outputs](https://docs.futureagi.com/docs/prompt/guides/evaluate-prompt-outputs) - [Track prompt performance](https://docs.futureagi.com/docs/prompt/guides/track-prompt-performance) - [Organize prompts in folders](https://docs.futureagi.com/docs/prompt/guides/organize-prompts-in-folders) - [Model configuration](https://docs.futureagi.com/docs/prompt/reference/model-configuration) - [SDK & API](https://docs.futureagi.com/docs/prompt/reference/sdk-api) - [Prompt FAQ & fixes](https://docs.futureagi.com/docs/prompt/troubleshooting) ### Protect - [Overview](https://docs.futureagi.com/docs/protect) - [Understanding Protect](https://docs.futureagi.com/docs/protect/concepts/understanding-protect) - [Turn on a guardrail](https://docs.futureagi.com/docs/protect/guides/turn-on-a-guardrail) - [Test a guardrail](https://docs.futureagi.com/docs/protect/guides/test-a-guardrail) - [Review guardrail activity](https://docs.futureagi.com/docs/protect/guides/review-guardrail-activity) - [Run Protect from the SDK](https://docs.futureagi.com/docs/protect/guides/run-protect-from-the-sdk) - [Guardrail checks](https://docs.futureagi.com/docs/protect/reference/guardrail-checks) - [Guardrail changes not taking effect](https://docs.futureagi.com/docs/protect/troubleshooting/guardrail-changes-not-taking-effect) - [Guardrail fires on the wrong requests](https://docs.futureagi.com/docs/protect/troubleshooting/guardrail-fires-on-the-wrong-requests) - [Protect SDK rejects an input](https://docs.futureagi.com/docs/protect/troubleshooting/protect-sdk-rejects-an-input) ### RBAC - [Admin & Settings](https://docs.futureagi.com/docs/admin-settings) - [API Keys](https://docs.futureagi.com/docs/admin-settings/api-keys) - [Profile & Security](https://docs.futureagi.com/docs/admin-settings/profile-security) - [Organization Settings](https://docs.futureagi.com/docs/admin-settings/organization-settings) - [User Management](https://docs.futureagi.com/docs/admin-settings/user-management) - [Workspace Management](https://docs.futureagi.com/docs/admin-settings/workspace-management) - [AI Providers](https://docs.futureagi.com/docs/admin-settings/ai-providers) - [Integrations](https://docs.futureagi.com/docs/admin-settings/integrations) - [Usage Summary](https://docs.futureagi.com/docs/admin-settings/usage-summary) - [Roles & Permissions](https://docs.futureagi.com/docs/roles-and-permissions) - [Overview](https://docs.futureagi.com/docs/billing) - [What you are billed for](https://docs.futureagi.com/docs/billing/concepts/what-you-are-billed-for) - [Pricing](https://docs.futureagi.com/docs/billing/reference/pricing) - [Manage your plan](https://docs.futureagi.com/docs/billing/guides/manage-your-plan) - [Control spend](https://docs.futureagi.com/docs/billing/guides/control-spend) - [FAQ](https://docs.futureagi.com/docs/faq) ### Simulation - [Overview](https://docs.futureagi.com/docs/simulation) - [Understanding Simulation](https://docs.futureagi.com/docs/simulation/concepts/understanding-simulation) - [Agent definitions & versions](https://docs.futureagi.com/docs/simulation/concepts/agent-definitions) - [Scenarios](https://docs.futureagi.com/docs/simulation/concepts/scenarios) - [Personas](https://docs.futureagi.com/docs/simulation/concepts/personas) - [Runs & results](https://docs.futureagi.com/docs/simulation/concepts/runs-and-results) - [Replay](https://docs.futureagi.com/docs/simulation/concepts/replay) - [Optimization](https://docs.futureagi.com/docs/simulation/concepts/optimization) - [Connect your agent](https://docs.futureagi.com/docs/simulation/guides/connect-your-agent) - [Create scenarios](https://docs.futureagi.com/docs/simulation/guides/create-scenarios) - [Create personas](https://docs.futureagi.com/docs/simulation/guides/create-personas) - [Overview](https://docs.futureagi.com/docs/simulation/guides/explore-scenarios) - [Explore scenario graph](https://docs.futureagi.com/docs/simulation/guides/explore-scenarios/scenario-graph) - [Add rows](https://docs.futureagi.com/docs/simulation/guides/explore-scenarios/add-rows) - [Add columns](https://docs.futureagi.com/docs/simulation/guides/explore-scenarios/add-columns) - [Create a simulation](https://docs.futureagi.com/docs/simulation/guides/create-simulation) - [Run a voice simulation](https://docs.futureagi.com/docs/simulation/guides/run-voice-simulation) - [Run a chat simulation](https://docs.futureagi.com/docs/simulation/guides/run-chat-simulation) - [Simulate a prompt](https://docs.futureagi.com/docs/simulation/guides/prompt-simulation) - [Edit evals in a simulation](https://docs.futureagi.com/docs/simulation/guides/edit-evals) - [Evaluate tool calls](https://docs.futureagi.com/docs/simulation/guides/evaluate-tool-calls) - [Replay chat sessions](https://docs.futureagi.com/docs/simulation/guides/replay-chat) - [Replay voice calls](https://docs.futureagi.com/docs/simulation/guides/replay-voice) - [Overview](https://docs.futureagi.com/docs/simulation/guides/explore-results) - [Calls & transcripts](https://docs.futureagi.com/docs/simulation/guides/explore-results/calls-and-transcripts) - [Analytics & metrics](https://docs.futureagi.com/docs/simulation/guides/explore-results/analytics) - [Fix My Agent](https://docs.futureagi.com/docs/simulation/guides/fix-my-agent) - [Running optimizations](https://docs.futureagi.com/docs/simulation/guides/running-optimizations) - [Optimization runs](https://docs.futureagi.com/docs/simulation/guides/optimization-runs) - [Built-in personas](https://docs.futureagi.com/docs/simulation/reference/built-in-personas) - [Voice providers](https://docs.futureagi.com/docs/simulation/reference/voice-providers) - [Call metrics](https://docs.futureagi.com/docs/simulation/reference/call-metrics) - [SDK & API](https://docs.futureagi.com/docs/simulation/reference/sdk-api) - [Simulation FAQ & fixes](https://docs.futureagi.com/docs/simulation/troubleshooting) ### Integrations - [Overview](https://docs.futureagi.com/docs/integrations) - [OpenAI](https://docs.futureagi.com/docs/integrations/traceai/openai) - [Anthropic](https://docs.futureagi.com/docs/integrations/traceai/anthropic) - [AWS Bedrock](https://docs.futureagi.com/docs/integrations/traceai/bedrock) - [Vertex AI](https://docs.futureagi.com/docs/integrations/traceai/vertexai) - [Google GenAI](https://docs.futureagi.com/docs/integrations/traceai/google_genai) - [Google ADK](https://docs.futureagi.com/docs/integrations/traceai/google_adk) - [Groq](https://docs.futureagi.com/docs/integrations/traceai/groq) - [MistralAI](https://docs.futureagi.com/docs/integrations/traceai/mistralai) - [Together AI](https://docs.futureagi.com/docs/integrations/traceai/togetherai) - [Ollama](https://docs.futureagi.com/docs/integrations/traceai/ollama) - [Portkey](https://docs.futureagi.com/docs/integrations/traceai/portkey) - [LangChain](https://docs.futureagi.com/docs/integrations/traceai/langchain) - [LangGraph](https://docs.futureagi.com/docs/integrations/traceai/langgraph) - [LlamaIndex](https://docs.futureagi.com/docs/integrations/traceai/llamaindex) - [LlamaIndex Workflows](https://docs.futureagi.com/docs/integrations/traceai/llamaindex-workflows) - [LiteLLM](https://docs.futureagi.com/docs/integrations/traceai/litellm) - [CrewAI](https://docs.futureagi.com/docs/integrations/traceai/crewai) - [AutoGen](https://docs.futureagi.com/docs/integrations/traceai/autogen) - [Haystack](https://docs.futureagi.com/docs/integrations/traceai/haystack) - [DSPy](https://docs.futureagi.com/docs/integrations/traceai/dspy) - [OpenAI Agents](https://docs.futureagi.com/docs/integrations/traceai/openai_agents) - [Smol Agents](https://docs.futureagi.com/docs/integrations/traceai/smol_agents) - [Instructor](https://docs.futureagi.com/docs/integrations/traceai/instructor) - [PromptFlow](https://docs.futureagi.com/docs/integrations/traceai/promptflow) - [Guardrails](https://docs.futureagi.com/docs/integrations/traceai/guardrails) - [MCP](https://docs.futureagi.com/docs/integrations/traceai/mcp) - [Mastra](https://docs.futureagi.com/docs/integrations/traceai/mastra) - [Vercel AI SDK](https://docs.futureagi.com/docs/integrations/traceai/vercel) - [LiveKit](https://docs.futureagi.com/docs/integrations/traceai/livekit) - [Pipecat](https://docs.futureagi.com/docs/integrations/traceai/pipecat) - [Overview](https://docs.futureagi.com/docs/integrations/traceai/java) - [Spring Boot](https://docs.futureagi.com/docs/integrations/traceai/spring-boot) - [OpenAI](https://docs.futureagi.com/docs/integrations/traceai/java/openai) - [Anthropic](https://docs.futureagi.com/docs/integrations/traceai/java/anthropic) - [AWS Bedrock](https://docs.futureagi.com/docs/integrations/traceai/java/bedrock) - [Cohere](https://docs.futureagi.com/docs/integrations/traceai/java/cohere) - [Pinecone](https://docs.futureagi.com/docs/integrations/traceai/java/pinecone) - [LLM Providers](https://docs.futureagi.com/docs/integrations/traceai/java/llm-providers) - [Vector Databases](https://docs.futureagi.com/docs/integrations/traceai/java/vector-databases) - [Frameworks](https://docs.futureagi.com/docs/integrations/traceai/java/frameworks) - [n8n](https://docs.futureagi.com/docs/integrations/traceai/n8n) - [Langfuse](https://docs.futureagi.com/docs/integrations/import/langfuse) - [Datadog](https://docs.futureagi.com/docs/integrations/export/datadog) - [PostHog](https://docs.futureagi.com/docs/integrations/export/posthog) - [Mixpanel](https://docs.futureagi.com/docs/integrations/export/mixpanel) - [PagerDuty](https://docs.futureagi.com/docs/integrations/export/pagerduty) - [Cloud Storage](https://docs.futureagi.com/docs/integrations/export/cloud-storage) - [Message Queues](https://docs.futureagi.com/docs/integrations/export/message-queues) ### Get Started - [Overview](https://docs.futureagi.com/docs/cookbook) - [Your First Evaluation](https://docs.futureagi.com/docs/cookbook/quickstart/first-eval) - [Dataset Management](https://docs.futureagi.com/docs/cookbook/quickstart/dataset-management) - [Dataset SDK Batch Eval](https://docs.futureagi.com/docs/cookbook/quickstart/batch-eval) - [Evaluator SDK Basics](https://docs.futureagi.com/docs/cookbook/using-futureagi-evals) - [Datasets with the SDK](https://docs.futureagi.com/docs/cookbook/using-futureagi-dataset) - [Knowledge Base SDK](https://docs.futureagi.com/docs/cookbook/using-futureagi-kb) - [Protect Rules SDK](https://docs.futureagi.com/docs/cookbook/using-futureagi-protect) - [Self-Hosted Docker Compose](https://docs.futureagi.com/docs/cookbook/self-hosting/docker-compose-quickstart) - [Monitoring and Alerts](https://docs.futureagi.com/docs/cookbook/quickstart/monitoring-alerts) - [Production Quality Monitoring](https://docs.futureagi.com/docs/cookbook/use-cases/production-quality-monitoring) - [Protect Safety Guardrails](https://docs.futureagi.com/docs/cookbook/quickstart/protect-guardrails) - [CI/CD Eval Pipeline](https://docs.futureagi.com/docs/cookbook/quickstart/cicd-eval-pipeline) - [Evaluation-Driven Development](https://docs.futureagi.com/docs/cookbook/quickstart/eval-driven-dev) ### Use Cases - [Overview](https://docs.futureagi.com/docs/cookbook/use-cases) - [Multi-Turn Conversation Eval](https://docs.futureagi.com/docs/cookbook/quickstart/conversation-eval) - [Session-Based Observability](https://docs.futureagi.com/docs/cookbook/quickstart/session-observability) - [LangGraph Agent Observability](https://docs.futureagi.com/docs/cookbook/observe-langgraph-agent-and-obtain-insights) - [Chat Simulation with Personas](https://docs.futureagi.com/docs/cookbook/quickstart/chat-simulation-personas) - [Chat Simulation & Fix My Agent](https://docs.futureagi.com/docs/cookbook/chat-simulation-fix-agent) - [Prompt Workbench Simulation](https://docs.futureagi.com/docs/cookbook/quickstart/prompt-workbench-simulation) - [End-to-End Agent Testing](https://docs.futureagi.com/docs/cookbook/use-cases/end-to-end-agent-testing) - [RAG Evaluation](https://docs.futureagi.com/docs/cookbook/quickstart/rag-evaluation) - [Hallucination Detection](https://docs.futureagi.com/docs/cookbook/quickstart/hallucination-detection) - [Evaluating RAG Applications](https://docs.futureagi.com/docs/cookbook/evaluate-rag) - [RAG Chatbot Trustworthiness](https://docs.futureagi.com/docs/cookbook/trustworthy-rag) - [Decrease RAG Hallucinations](https://docs.futureagi.com/docs/cookbook/decrease-hallucination) - [LangChain RAG Evaluation](https://docs.futureagi.com/docs/cookbook/rag-langchain) - [LlamaIndex PDF RAG Chatbot](https://docs.futureagi.com/docs/cookbook/llamaindex-pdf-rag) - [MongoDB Atlas RAG Chatbot](https://docs.futureagi.com/docs/cookbook/mongodb) - [Knowledge Base](https://docs.futureagi.com/docs/cookbook/quickstart/knowledge-base) - [Voice Simulation](https://docs.futureagi.com/docs/cookbook/quickstart/voice-simulation) - [Voice Agent Simulate SDK](https://docs.futureagi.com/docs/cookbook/simulate-sdk) - [CrewAI Research Team](https://docs.futureagi.com/docs/cookbook/crewai-research-team) - [Google ADK Error Feed](https://docs.futureagi.com/docs/cookbook/error-feed/google-adk-multi-agent) - [Tool-Calling Agent Simulation](https://docs.futureagi.com/docs/cookbook/quickstart/tool-calling-simulation) - [Agent Function Calling Eval](https://docs.futureagi.com/docs/cookbook/ai-agents) - [LangChain & LangGraph Observability](https://docs.futureagi.com/docs/cookbook/langchain-langgraph) - [Meeting Summarization Eval](https://docs.futureagi.com/docs/cookbook/meeting-summarization) - [AI SDR Outreach Eval](https://docs.futureagi.com/docs/cookbook/ai-sdr) - [Image Evaluation](https://docs.futureagi.com/docs/cookbook/image-evaluation) - [Multimodal Evaluation](https://docs.futureagi.com/docs/cookbook/quickstart/multimodal-eval) - [Tone, Toxicity & Bias Evals](https://docs.futureagi.com/docs/cookbook/quickstart/tone-toxicity-bias-eval) - [Text-to-SQL Evaluation](https://docs.futureagi.com/docs/cookbook/quickstart/text-to-sql-eval) - [Text-to-SQL Agent](https://docs.futureagi.com/docs/cookbook/text-to-sql) ### Platform - [Overview](https://docs.futureagi.com/docs/cookbook/platform) - [Instrument and Verify](https://docs.futureagi.com/docs/cookbook/quickstart/instrument-and-verify) - [Instrument and Verify a Voice Agent](https://docs.futureagi.com/docs/cookbook/quickstart/instrument-and-verify-voice) - [Manual Tracing](https://docs.futureagi.com/docs/cookbook/quickstart/manual-tracing) - [Distributed Tracing](https://docs.futureagi.com/docs/cookbook/quickstart/distributed-tracing) - [Inline Evals in Tracing](https://docs.futureagi.com/docs/cookbook/quickstart/inline-evals-tracing) - [Portkey Integration](https://docs.futureagi.com/docs/cookbook/portkey-integration) - [Debug Traces from IDE](https://docs.futureagi.com/docs/cookbook/mcp/debug-traces-from-ide) - [Falcon AI Trace Debugging](https://docs.futureagi.com/docs/cookbook/falcon-ai/context-aware-debugging) - [Falcon AI End-to-End Workflow](https://docs.futureagi.com/docs/cookbook/falcon-ai/end-to-end) - [Custom Eval Metrics](https://docs.futureagi.com/docs/cookbook/quickstart/custom-eval-metrics) - [Async & Batch Evaluations](https://docs.futureagi.com/docs/cookbook/quickstart/async-batch-eval) - [Eval Correction Loop](https://docs.futureagi.com/docs/cookbook/evaluation/eval-correction-loop) - [Eval Metrics for Optimization](https://docs.futureagi.com/docs/cookbook/eval-metrics-optimization) - [Comparing Prompts and Models](https://docs.futureagi.com/docs/cookbook/quickstart/experimentation-compare-prompts) - [Synthetic Data Generation](https://docs.futureagi.com/docs/cookbook/quickstart/synthetic-data-generation) - [Dynamic Dataset Columns](https://docs.futureagi.com/docs/cookbook/quickstart/dynamic-dataset-columns) - [Hugging Face Dataset Import](https://docs.futureagi.com/docs/cookbook/quickstart/huggingface-dataset-import) - [Dataset Annotation](https://docs.futureagi.com/docs/cookbook/quickstart/dataset-annotation) - [Golden Datasets from Traces](https://docs.futureagi.com/docs/cookbook/falcon-ai/eval-datasets-from-traces) - [Prompt Versioning](https://docs.futureagi.com/docs/cookbook/quickstart/prompt-versioning) - [Prompt Optimization](https://docs.futureagi.com/docs/cookbook/quickstart/prompt-optimization) - [Basic Prompt Optimization](https://docs.futureagi.com/docs/cookbook/basic-optimization) - [End-to-End Prompt Optimization](https://docs.futureagi.com/docs/cookbook/end-to-end-optimization) - [GEPA Optimization](https://docs.futureagi.com/docs/cookbook/gepa-optimization) - [Dataset Optimization](https://docs.futureagi.com/docs/cookbook/quickstart/dataset-optimization) - [Custom Datasets for Optimization](https://docs.futureagi.com/docs/cookbook/import-datasets) - [Comparing Prompt Optimizers](https://docs.futureagi.com/docs/cookbook/quickstart/compare-optimizers) - [Comparing Optimizers](https://docs.futureagi.com/docs/cookbook/compare-optimization) - [Semantic Caching](https://docs.futureagi.com/docs/cookbook/command-center/semantic-caching) ### SDK Overview - [SDK Overview](https://docs.futureagi.com/docs/sdk) - [Evaluation](https://docs.futureagi.com/docs/sdk/list/evaluation) - [TraceAI](https://docs.futureagi.com/docs/sdk/list/traceai) - [Core SDKs](https://docs.futureagi.com/docs/sdk/list/core) ### Evaluation - [Overview](https://docs.futureagi.com/docs/sdk/evals) - [Running Evaluations](https://docs.futureagi.com/docs/sdk/evals/evaluate) - [AutoEval](https://docs.futureagi.com/docs/sdk/evals/autoeval) - [LLM-as-Judge](https://docs.futureagi.com/docs/sdk/evals/llm-judge) - [Guardrails](https://docs.futureagi.com/docs/sdk/evals/guardrails-module) - [Local & Hybrid](https://docs.futureagi.com/docs/sdk/evals/local) - [Distributed Evaluator](https://docs.futureagi.com/docs/sdk/evals/distributed) - [Streaming](https://docs.futureagi.com/docs/sdk/evals/streaming) - [Cloud Evals](https://docs.futureagi.com/docs/sdk/evals/cloud-evals) - [Feedback Loops](https://docs.futureagi.com/docs/sdk/evals/feedback) - [Code Security](https://docs.futureagi.com/docs/sdk/evals/code-security) - [OpenTelemetry](https://docs.futureagi.com/docs/sdk/evals/otel) - [Overview](https://docs.futureagi.com/docs/sdk/evals/metrics) - [String & Similarity](https://docs.futureagi.com/docs/sdk/evals/metrics/string) - [JSON & Structured](https://docs.futureagi.com/docs/sdk/evals/metrics/json) - [Hallucination](https://docs.futureagi.com/docs/sdk/evals/metrics/hallucination) - [RAG](https://docs.futureagi.com/docs/sdk/evals/metrics/rag) - [Agents & Functions](https://docs.futureagi.com/docs/sdk/evals/metrics/agents) - [Guardrails](https://docs.futureagi.com/docs/sdk/evals/metrics/guardrails) ### traceAI - [Overview](https://docs.futureagi.com/docs/sdk/tracing) - [Set up tracing](https://docs.futureagi.com/docs/sdk/tracing/set-up-tracing) - [Instrument with helpers](https://docs.futureagi.com/docs/sdk/tracing/instrument-with-traceai-helpers) - [Set session & user IDs](https://docs.futureagi.com/docs/sdk/tracing/set-session-user-id) - [Attributes, metadata & tags](https://docs.futureagi.com/docs/sdk/tracing/add-attributes-metadata-tags) - [Log prompt templates](https://docs.futureagi.com/docs/sdk/tracing/log-prompt-templates) - [Events, exceptions & status](https://docs.futureagi.com/docs/sdk/tracing/add-events-exceptions-status) - [Mask attributes](https://docs.futureagi.com/docs/sdk/tracing/mask-span-attributes) - [Create tool spans](https://docs.futureagi.com/docs/sdk/tracing/create-tool-spans) - [Get span context](https://docs.futureagi.com/docs/sdk/tracing/get-current-span-context) - [In-line evals](https://docs.futureagi.com/docs/sdk/tracing/in-line-evals) - [Annotate via API](https://docs.futureagi.com/docs/sdk/tracing/annotating-using-api) - [Advanced examples](https://docs.futureagi.com/docs/sdk/tracing/advanced-tracing-examples) - [Langfuse integration](https://docs.futureagi.com/docs/sdk/tracing/langfuse-integration) - [register()](https://docs.futureagi.com/docs/sdk/tracing/register) - [FITracer & custom spans](https://docs.futureagi.com/docs/sdk/tracing/fitracer) - [Context helpers](https://docs.futureagi.com/docs/sdk/tracing/context-helpers) - [TraceConfig](https://docs.futureagi.com/docs/sdk/tracing/trace-config) - [EvalTags](https://docs.futureagi.com/docs/sdk/tracing/eval-tags) - [Instrumentors](https://docs.futureagi.com/docs/sdk/tracing/instrumentors) - [Environment variables](https://docs.futureagi.com/docs/sdk/tracing/environment-variables) - [Semantic conventions](https://docs.futureagi.com/docs/sdk/tracing/semantic-conventions) ### Simulation - [Overview](https://docs.futureagi.com/docs/sdk/simulate) ### Datasets - [Overview](https://docs.futureagi.com/docs/sdk/datasets) ### Prompt Optimization - [Overview](https://docs.futureagi.com/docs/sdk/optimization) ### Annotation Queues - [Overview](https://docs.futureagi.com/docs/sdk/annotation-queues) - [Labels](https://docs.futureagi.com/docs/sdk/annotation-queues/labels) - [Queue management](https://docs.futureagi.com/docs/sdk/annotation-queues/queues) - [Queue lifecycle](https://docs.futureagi.com/docs/sdk/annotation-queues/lifecycle) - [Queue items](https://docs.futureagi.com/docs/sdk/annotation-queues/items) - [Annotations](https://docs.futureagi.com/docs/sdk/annotation-queues/annotations) - [Scores](https://docs.futureagi.com/docs/sdk/annotation-queues/scores) - [Progress & analytics](https://docs.futureagi.com/docs/sdk/annotation-queues/analytics) - [Export](https://docs.futureagi.com/docs/sdk/annotation-queues/export) - [Data models](https://docs.futureagi.com/docs/sdk/annotation-queues/data-models) ### Knowledge Base - [Overview](https://docs.futureagi.com/docs/sdk/knowledgebase) ### Protect - [Overview](https://docs.futureagi.com/docs/sdk/protect) ### API Reference - [Introduction](https://docs.futureagi.com/docs/api) - [Health Check](https://docs.futureagi.com/docs/api/health/healthcheck) - [List Eval Tasks](https://docs.futureagi.com/docs/api/eval-tasks/list-eval-tasks-filtered) - [Create Eval Task](https://docs.futureagi.com/docs/api/eval-tasks/create-eval-task) - [Get Eval Task](https://docs.futureagi.com/docs/api/eval-tasks/get-eval-task) - [Update Eval Task](https://docs.futureagi.com/docs/api/eval-tasks/update-eval-task) - [Delete Eval Task](https://docs.futureagi.com/docs/api/eval-tasks/delete-eval-task) - [Bulk Delete Eval Tasks](https://docs.futureagi.com/docs/api/eval-tasks/bulk-delete-eval-tasks) - [Pause Eval Task](https://docs.futureagi.com/docs/api/eval-tasks/pause-eval-task) - [Unpause Eval Task](https://docs.futureagi.com/docs/api/eval-tasks/unpause-eval-task) - [Eval Task Aggregations](https://docs.futureagi.com/docs/api/eval-tasks/eval-task-aggregations) - [List Custom Eval Configs](https://docs.futureagi.com/docs/api/custom-eval-configs/list-configs-filtered) - [Create Custom Eval Config](https://docs.futureagi.com/docs/api/custom-eval-configs/create-custom-eval-config) - [Get Custom Eval Config](https://docs.futureagi.com/docs/api/custom-eval-configs/get-custom-eval-config) - [Update Custom Eval Config](https://docs.futureagi.com/docs/api/custom-eval-configs/update-custom-eval-config) - [Delete Custom Eval Config](https://docs.futureagi.com/docs/api/custom-eval-configs/delete-custom-eval-config) - [Check Config Exists](https://docs.futureagi.com/docs/api/custom-eval-configs/check-config-exists) - [Get Eval Template Names](https://docs.futureagi.com/docs/api/dataset-evals/get-eval-template-names) - [Create Custom Eval Template](https://docs.futureagi.com/docs/api/dataset-evals/create-custom-eval-template) - [List Dataset Evals](https://docs.futureagi.com/docs/api/dataset-evals/list-dataset-evals) - [Get Eval Structure](https://docs.futureagi.com/docs/api/dataset-evals/get-eval-structure) - [Add Dataset Eval](https://docs.futureagi.com/docs/api/dataset-evals/add-dataset-eval) - [Start Evals Process](https://docs.futureagi.com/docs/api/dataset-evals/start-evals-process) - [Delete Dataset Eval](https://docs.futureagi.com/docs/api/dataset-evals/delete-dataset-eval) - [Edit and Run Eval](https://docs.futureagi.com/docs/api/dataset-evals/edit-and-run-eval) - [List Scenarios](https://docs.futureagi.com/docs/api/scenarios/listscenarios) - [Get Scenario Details](https://docs.futureagi.com/docs/api/scenarios/getscenario) - [Create Scenario](https://docs.futureagi.com/docs/api/scenarios/createscenario) - [Edit Scenario](https://docs.futureagi.com/docs/api/scenarios/editscenario) - [Delete Scenario](https://docs.futureagi.com/docs/api/scenarios/deletescenario) - [Add Rows with AI](https://docs.futureagi.com/docs/api/scenarios/addscenariorowswithai) - [Add Columns](https://docs.futureagi.com/docs/api/scenarios/addcolumns) - [List Personas](https://docs.futureagi.com/docs/api/personas/listpersonas) - [Create Persona](https://docs.futureagi.com/docs/api/personas/createpersona) - [Update Persona](https://docs.futureagi.com/docs/api/personas/updatepersona) - [Delete Persona](https://docs.futureagi.com/docs/api/personas/deletepersona) - [Duplicate Persona](https://docs.futureagi.com/docs/api/personas/duplicatepersona) - [List Agent Definitions](https://docs.futureagi.com/docs/api/agent-definitions/listagentdefinitions) - [Create Agent Definition](https://docs.futureagi.com/docs/api/agent-definitions/createagentdefinition) - [Get Agent Definition](https://docs.futureagi.com/docs/api/agent-definitions/getagentdefinition) - [Delete Agent Definitions](https://docs.futureagi.com/docs/api/agent-definitions/deleteagentdefinitions) - [Fetch from Provider](https://docs.futureagi.com/docs/api/agent-definitions/fetchassistantfromprovider) - [List Agent Versions](https://docs.futureagi.com/docs/api/agent-versions/listagentversions) - [Create Agent Version](https://docs.futureagi.com/docs/api/agent-versions/createagentversion) - [Get Agent Version](https://docs.futureagi.com/docs/api/agent-versions/getagentversion) - [Get Version Call Executions](https://docs.futureagi.com/docs/api/agent-versions/getversioncallexecutions) - [Get Version Eval Summary](https://docs.futureagi.com/docs/api/agent-versions/getversionevalsummary) - [List Test Runs](https://docs.futureagi.com/docs/api/run-tests/listruntests) - [Create Run Test](https://docs.futureagi.com/docs/api/run-tests/createruntest) - [Get Test Run Details](https://docs.futureagi.com/docs/api/run-tests/getruntestdetails) - [Delete Test Run](https://docs.futureagi.com/docs/api/run-tests/deleteruntest) - [Execute Run Test](https://docs.futureagi.com/docs/api/run-tests/executeruntest) - [Get Test Executions](https://docs.futureagi.com/docs/api/run-tests/gettestexecutions) - [Get Test Scenarios](https://docs.futureagi.com/docs/api/run-tests/gettestscenarios) - [Get Eval Summary](https://docs.futureagi.com/docs/api/run-tests/getevalsummary) - [Compare Eval Summaries](https://docs.futureagi.com/docs/api/run-tests/compareevalsummaries) - [Add Eval Configs](https://docs.futureagi.com/docs/api/run-tests/addevalconfigs) - [Update Eval Config](https://docs.futureagi.com/docs/api/run-tests/updateevalconfig) - [Delete Eval Config](https://docs.futureagi.com/docs/api/run-tests/deleteevalconfig) - [Run New Evals](https://docs.futureagi.com/docs/api/run-tests/runnewevalsontestexecution) - [Rerun Test Executions](https://docs.futureagi.com/docs/api/run-tests/reruntestexecutions) - [Delete Test Executions](https://docs.futureagi.com/docs/api/run-tests/deletetestexecutions) - [Get Execution Details](https://docs.futureagi.com/docs/api/test-executions/gettestexecutiondetails) - [Get Execution KPIs](https://docs.futureagi.com/docs/api/test-executions/getkpis) - [Get Performance Summary](https://docs.futureagi.com/docs/api/test-executions/getperformancesummary) - [Cancel Execution](https://docs.futureagi.com/docs/api/test-executions/cancelexecution) - [Rerun Calls](https://docs.futureagi.com/docs/api/test-executions/reruncalls) - [Get Call Details](https://docs.futureagi.com/docs/api/test-executions/getcallexecutiondetails) - [Get Simulation Metrics](https://docs.futureagi.com/docs/api/simulation-analytics/metrics) - [Get Simulation Runs](https://docs.futureagi.com/docs/api/simulation-analytics/runs) - [Get Simulation Analytics](https://docs.futureagi.com/docs/api/simulation-analytics/analytics) - [List Datasets](https://docs.futureagi.com/docs/api/datasets/list-datasets) - [Create Dataset](https://docs.futureagi.com/docs/api/datasets/create-dataset) - [Create Empty Dataset](https://docs.futureagi.com/docs/api/datasets/create-empty-dataset) - [Upload Dataset from File](https://docs.futureagi.com/docs/api/datasets/upload-dataset) - [Create from HuggingFace](https://docs.futureagi.com/docs/api/datasets/create-dataset-from-huggingface) - [Clone Dataset](https://docs.futureagi.com/docs/api/datasets/clone-dataset) - [Duplicate Dataset](https://docs.futureagi.com/docs/api/datasets/duplicate-dataset) - [Add as New Dataset](https://docs.futureagi.com/docs/api/datasets/add-as-new) - [Update Dataset](https://docs.futureagi.com/docs/api/datasets/update-dataset) - [Merge Dataset](https://docs.futureagi.com/docs/api/datasets/merge-dataset) - [Delete Dataset](https://docs.futureagi.com/docs/api/datasets/delete-dataset) - [Add Rows from File](https://docs.futureagi.com/docs/api/datasets/add-rows-from-file) - [Add Empty Rows](https://docs.futureagi.com/docs/api/datasets/add-empty-rows) - [Add Rows from Existing](https://docs.futureagi.com/docs/api/datasets/add-rows-from-existing) - [Add Rows from HuggingFace](https://docs.futureagi.com/docs/api/datasets/add-rows-from-huggingface) - [Duplicate Rows](https://docs.futureagi.com/docs/api/datasets/duplicate-rows) - [Delete Rows](https://docs.futureagi.com/docs/api/datasets/delete-rows) - [Update Cell Value](https://docs.futureagi.com/docs/api/datasets/update-cell-value) - [Get Column Details](https://docs.futureagi.com/docs/api/datasets/columns/get-column-details) - [Get Column Config](https://docs.futureagi.com/docs/api/datasets/columns/get-column-config) - [Add Static Column](https://docs.futureagi.com/docs/api/datasets/columns/add-static-column) - [Add Multiple Static Columns](https://docs.futureagi.com/docs/api/datasets/columns/add-multiple-static-columns) - [Add Columns](https://docs.futureagi.com/docs/api/datasets/columns/add-columns) - [Update Column Name](https://docs.futureagi.com/docs/api/datasets/columns/update-column-name) - [Update Column Type](https://docs.futureagi.com/docs/api/datasets/columns/update-column-type) - [Delete Column](https://docs.futureagi.com/docs/api/datasets/columns/delete-column) - [Add Run Prompt Column](https://docs.futureagi.com/docs/api/datasets/run-prompt/add-run-prompt-column) - [Edit Run Prompt Column](https://docs.futureagi.com/docs/api/datasets/run-prompt/edit-run-prompt-column) - [Get Run Prompt Config](https://docs.futureagi.com/docs/api/datasets/run-prompt/retrieve-run-prompt-column-config) - [Get Run Prompt Options](https://docs.futureagi.com/docs/api/datasets/run-prompt/retrieve-run-prompt-options) - [Get Model Voices](https://docs.futureagi.com/docs/api/datasets/run-prompt/get-model-voices) - [TTS Voices](https://docs.futureagi.com/docs/api/datasets/run-prompt/tts-voices) - [Get Column Values](https://docs.futureagi.com/docs/api/datasets/run-prompt/get-column-values) - [Run Prompt Stats](https://docs.futureagi.com/docs/api/datasets/analytics/run-prompt-stats) - [Eval Stats](https://docs.futureagi.com/docs/api/datasets/analytics/eval-stats) - [Annotation Summary](https://docs.futureagi.com/docs/api/datasets/analytics/annotation-summary) - [Explanation Summary](https://docs.futureagi.com/docs/api/datasets/analytics/explanation-summary) - [Create Score](https://docs.futureagi.com/docs/api/annotations/scores/create-score) - [Bulk Create Scores](https://docs.futureagi.com/docs/api/annotations/scores/bulk-create-scores) - [Get Scores for Source](https://docs.futureagi.com/docs/api/annotations/scores/get-scores-for-source) - [List Scores](https://docs.futureagi.com/docs/api/annotations/scores/list-scores) - [Delete Score](https://docs.futureagi.com/docs/api/annotations/scores/delete-score) - [Create Label](https://docs.futureagi.com/docs/api/annotations/labels/create-label) - [List Labels](https://docs.futureagi.com/docs/api/annotations/labels/list-labels) - [Get Label](https://docs.futureagi.com/docs/api/annotations/labels/get-label) - [Update Label](https://docs.futureagi.com/docs/api/annotations/labels/update-label) - [Delete Label](https://docs.futureagi.com/docs/api/annotations/labels/delete-label) - [Restore Label](https://docs.futureagi.com/docs/api/annotations/labels/restore-label) - [Create Queue](https://docs.futureagi.com/docs/api/annotations/queues/create-queue) - [List Queues](https://docs.futureagi.com/docs/api/annotations/queues/list-queues) - [Get Queue](https://docs.futureagi.com/docs/api/annotations/queues/get-queue) - [Update Queue](https://docs.futureagi.com/docs/api/annotations/queues/update-queue) - [Delete Queue](https://docs.futureagi.com/docs/api/annotations/queues/delete-queue) - [Update Status](https://docs.futureagi.com/docs/api/annotations/queues/update-status) - [Get Progress](https://docs.futureagi.com/docs/api/annotations/queues/get-progress) - [Get Analytics](https://docs.futureagi.com/docs/api/annotations/queues/get-analytics) - [Get Agreement](https://docs.futureagi.com/docs/api/annotations/queues/get-agreement) - [Export](https://docs.futureagi.com/docs/api/annotations/queues/export) - [Export to Dataset](https://docs.futureagi.com/docs/api/annotations/queues/export-to-dataset) - [Add Label to Queue](https://docs.futureagi.com/docs/api/annotations/queues/add-label) - [Remove Label](https://docs.futureagi.com/docs/api/annotations/queues/remove-label) - [Get or Create Default](https://docs.futureagi.com/docs/api/annotations/queues/get-or-create-default) - [Find Queues for Source](https://docs.futureagi.com/docs/api/annotations/queues/find-queues-for-source) - [List Items](https://docs.futureagi.com/docs/api/annotations/items/list-items) - [Add Items](https://docs.futureagi.com/docs/api/annotations/items/add-items) - [Bulk Remove Items](https://docs.futureagi.com/docs/api/annotations/items/bulk-remove-items) - [Get Annotate Detail](https://docs.futureagi.com/docs/api/annotations/items/get-annotate-detail) - [Get Next Item](https://docs.futureagi.com/docs/api/annotations/items/get-next-item) - [Submit Annotations](https://docs.futureagi.com/docs/api/annotations/items/submit-annotations) - [Complete Item](https://docs.futureagi.com/docs/api/annotations/items/complete-item) - [Skip Item](https://docs.futureagi.com/docs/api/annotations/items/skip-item) - [Get Item Annotations](https://docs.futureagi.com/docs/api/annotations/items/get-item-annotations) - [Assign Items](https://docs.futureagi.com/docs/api/annotations/items/assign-items) - [Release Item](https://docs.futureagi.com/docs/api/annotations/items/release-item) - [Bulk Annotate Spans](https://docs.futureagi.com/docs/api/annotations/bulk/bulk-annotate-spans) ### Additional Pages - [Getcallexecutions](https://docs.futureagi.com/docs/api/run-tests/getcallexecutions) - [Updatetestcomponents](https://docs.futureagi.com/docs/api/run-tests/updatetestcomponents) - [Addemptyrowstodataset](https://docs.futureagi.com/docs/api/scenarios/addemptyrowstodataset) - [Command Center API Reference](https://docs.futureagi.com/docs/command-center/concepts/api-reference) - [Server-Sent Events Streaming in Agent Command Center](https://docs.futureagi.com/docs/command-center/features/streaming) - [Agent Command Center Quickstart: Connect in 5 Minutes](https://docs.futureagi.com/docs/command-center/quickstart) - [Google ADK Integration with Future AGI for AI Tracing](https://docs.futureagi.com/docs/integrations/google-adk) - [Google GenAI Integration with Future AGI for Gemini Tracing](https://docs.futureagi.com/docs/integrations/google-genai) - [OpenAI Agents SDK Integration with Future AGI Tracing](https://docs.futureagi.com/docs/integrations/openai-agents) - [Smol Agents Integration with Future AGI for HF Tracing](https://docs.futureagi.com/docs/integrations/smol-agents) - [Auto-Instrumentation Integrations: Future AGI Tracing](https://docs.futureagi.com/docs/integrations/traceai) - [Alerts and Monitors: Observe Metric Threshold Notifications](https://docs.futureagi.com/docs/observe/features/alerts) - [Dashboards: Custom Metric Visualization in Observe](https://docs.futureagi.com/docs/observe/features/dashboard) - [Run Evals on Traces in Future AGI Observe](https://docs.futureagi.com/docs/observe/features/evals) - [Set Up Observability with Future AGI Observe](https://docs.futureagi.com/docs/observe/features/quickstart) - [Explore sessions & users](https://docs.futureagi.com/docs/observe/features/session) - [User Dashboard: Per-User Trace and Session Analytics](https://docs.futureagi.com/docs/observe/features/users) - [Annotations Quickstart: Labels, Queues, and Annotation Setup](https://docs.futureagi.com/docs/quickstart/annotations) - [Command Center Gateway Quickstart: First LLM Request](https://docs.futureagi.com/docs/quickstart/command-center-gateway) - [Generate Synthetic Data Quickstart with Future AGI Datasets](https://docs.futureagi.com/docs/quickstart/generate-synthetic-data) - [Create Prompts Quickstart: Future AGI Prompt Workbench](https://docs.futureagi.com/docs/quickstart/prompts) - [Running Evals in Simulation: Score Agent Interactions](https://docs.futureagi.com/docs/quickstart/running-evals-in-simulation) - [Setup Observability with Future AGI for LLM Monitoring](https://docs.futureagi.com/docs/quickstart/setup-observability) - [Test Case: Future AGI Python SDK Test Reference](https://docs.futureagi.com/docs/sdk/testcase) - [Self-Hosting System Configuration](https://docs.futureagi.com/docs/self-hosting/configuration) - [Self-Hosting with Docker Compose](https://docs.futureagi.com/docs/self-hosting/docker-compose) - [Self-Hosting User Management](https://docs.futureagi.com/docs/self-hosting/user-management) - [Understanding Observability: LLM Tracing Core Concepts](https://docs.futureagi.com/docs/tracing/concepts) - [What is OpenTelemetry? Future AGI Tracing Explained](https://docs.futureagi.com/docs/tracing/concepts/otel) - [What are Spans? LLM, Tool, and Chain Span Types](https://docs.futureagi.com/docs/tracing/concepts/spans) - [What is traceAI? Future AGI Open-Source Tracing SDK](https://docs.futureagi.com/docs/tracing/concepts/traceai) - [What are Traces? Understanding Trace Structure](https://docs.futureagi.com/docs/tracing/concepts/traces) ## Full Documentation For the complete documentation with all page content, see [llms-full.txt](https://docs.futureagi.com/llms-full.txt).