Observability

A minimalist TypeScript middleware framework for building AI agents with composable guards, model routing, memory, testing, and observability.

An open-source control plane for building and operating AI agents as callable services, with routing, coordination, asynchronous execution, memory, and observability.

A local-first terminal interface and report generator for reviewing AI coding-agent sessions, usage costs, latency, failures, and run health.

A TypeScript library for monitoring agent heartbeats, correlating cross-agent traces, detecting cascade failures, and replaying failure chains.

Arize Phoenix is a source-available AI observability and evaluation platform, built on OpenTelemetry and OpenInference, for tracing, evaluating and debugging AI applications and agents in your own environment.

A local agent-experience graph for coding agents that captures sessions, recalls project context, and turns recurring workflow friction into reviewable improvements.

ClawMetry is an open-source, local-first dashboard that reads existing session files from OpenClaw, NVIDIA NemoClaw, Goose, and Qwen Code. It provides agent session replay, tool-call timelines, and token and cost reporting where runtimes expose that data.

halo-record creates tamper-evident, hash-chained records of AI agent tool calls, model calls, and data access, with verifiable reports and policy checks.

Helicone is an open-source observability platform for Language Learning Models (LLMs), providing features like request logging, caching, rate limiting, cost and latency tracking, UI-based prompt iteration, and collaboration tools

An open-source replay-based evaluation toolkit for AI agents that records or imports production traces as sessions and reruns them against model, prompt, or code changes.

LangSmith provides tools for debugging, testing, evaluating, and monitoring LLM applications, integrating seamlessly with LangChain for comprehensive AI agent observability

Observatory provides TypeScript and Python OpenTelemetry packages for instrumenting LLM and agent frameworks, plus a local widget for viewing traces. Integrations include Vercel AI SDK, Claude Agent SDK, LangChain, Mastra, OpenClaw, CrewAI, Agno, and LiteLLM.

OrcaReplay records a coding agent run from outside the process, replays it offline against the recorded bytes with no model called, and forks a checkpoint onto other models graded by a verify command.

Project Telos is a local-first MCP and CLI workbench for inspectable agent workflows, with provenance, routing ledgers, action receipts, context packs, diagnostics, and replayable verification packets.