Drop into the stack you already have
NOMOO is designed to integrate into the way you already build. One-line integration, automatic instrumentation, and a framework-agnostic SDK mean the tooling joins your infrastructure in an afternoon, whatever your agents are built on and wherever they run.
Eight surfaces, one SDK call
Integration is deliberately shallow. There is one place where NOMOO enters your code, and everything else is either discovered automatically or configured once.
Framework-agnostic, not framework-specific
There is a meaningful difference between a tool that ships an adapter per framework and one that does not need adapters. NOMOO instruments at the call level, which is why the same entry point works across orchestration libraries and across the in-house code most teams end up writing anyway.
Why that choice matters later
Framework-specific instrumentation ties your history to a dependency. Swap orchestration libraries and the record restarts from zero, which is exactly when you most need the comparison. Run two frameworks side by side during a migration and you get two incompatible views of one system.
Because NOMOO captures calls rather than library internals, a migration shows up in your traces as what it is: the same agent, doing the same work, with a different orchestrator in the middle.
- One entry point.
instrument()is the whole integration, on every stack. - One trace shape. Spans, kinds, and hierarchy are identical regardless of framework.
- Comparable across a migration. History survives a change of orchestration library.
- In-house code included. Hand-rolled agent loops are instrumented the same way.
| Stack | Captured without extra work |
|---|---|
| LangChain | Chain and agent executors, tool calls, retrievers |
| LlamaIndex | Query engines, retrievers, node post-processing |
| CrewAI | Crew and agent tasks, delegated steps between members |
| AutoGen | Conversable agents, group chat turns, tool execution |
| Semantic Kernel | Planners, kernel functions, plugin invocations |
| Haystack | Pipelines, components, retriever and reader nodes |
| In-house loops | Agent entry point plus any function you mark as a span |
model: "planner-v2",
temperature: 0.2,
top_p: 0.9,
max_tokens: 384
}
prompt: 318,
completion: 142
}
total_ms: 312,
first_token_ms: 186
}
Any endpoint, one set of fields
Model calls are captured as spans with their rendered prompt, their parameters, their token counts, their timing, and their completion. Provider APIs differ in shape; the span does not, which is what makes a before and after comparison across a model change meaningful rather than approximate.
That covers hosted provider APIs, gateways and proxies in front of them, and models you serve yourself. A self-hosted endpoint behind your own gateway produces the same span as a commercial API, because the capture happens on your side of the call.
- Parameters recorded per call. Temperature and sampling settings are part of the span, not a global assumption.
- Token accounting. Prompt and completion counts per span, so cost resolves to a step rather than a bill.
- Latency split. Time to first token separated from total duration, which is where streaming problems hide.
- Provider-neutral. Swap models without rewriting instrumentation or losing comparability.
The step most agents fail at, captured properly
In practice, a large share of wrong answers are retrieval problems wearing a model's voice. If a lookup returns nothing useful and the model answers anyway, the fault is upstream of the sentence everybody is reading.
Queries and filters
The query text, the embedding model, the top-k value, and any metadata filters applied. Reproducing a bad retrieval starts with the exact request that produced it.
Hits and scores
How many passages came back, what they scored, and which source they came from. An empty result and a low-confidence result are different failures with different fixes.
Grounding check
Automated review compares the composed answer against what retrieval actually returned, and flags claims that nothing in the run supports.
Store-agnostic, like everything else
Retrieval spans are captured at the call your agent makes, so the vector database, search index, or hybrid setup behind it is your choice. Swapping stores changes the implementation behind a span, not the record of what the agent asked for.
Agent runs in the same pane as the rest of your services
Your platform team already operates a telemetry pipeline, and an agent that is invisible to it is a gap in the system of record. NOMOO spans follow OpenTelemetry semantics, so they can be delivered to your collector alongside everything else.
One capture, two destinations
Set a collector endpoint at initialization and captured spans are mirrored to it. There is no second instrumentation pass, no duplicated spans, and no divergence between what NOMOO shows and what your dashboards show.
The practical gain is correlation. When an agent's tool call hits an internal service, both halves of that hop are in the same trace, so a latency problem does not stop being traceable at the boundary of the agent.
- Standard span semantics. No translation layer and no proprietary wire format.
- Service correlation. Agent spans join the traces your services already emit.
- Your retention rules apply. Once a span reaches your collector, it lives under your policies.
import nomoo nomoo.init( project="support-triage", environment="production", otel_endpoint="http://collector.internal:4317", otel_service_name="support-agent", )
import { init } from "@nomoo/sdk"; init({ project: "support-triage", environment: "production", otelEndpoint: "http://collector.internal:4317", otelServiceName: "support-agent", });
Quality checks where decisions are already made
Findings only change behavior if they reach the place where the work happens: the pipeline that approves a release and the channel the on-call engineer is already reading.
- name: Score agent against the suite run: nomoo eval --suite support-triage --min-score 8.0 - name: Compare with the deployed release run: nomoo compare --baseline production --fail-on-drop 0.3
Gate the release, then watch the traffic
Run a scored evaluation as a step in your pipeline. Set a floor on the absolute score, a ceiling on the drop against the currently deployed release, or both. A change that makes the agent measurably worse does not merge.
After the deploy, regression detection takes over on live traffic, because an evaluation suite only covers the behavior you thought to write down. Real traffic finds the rest.
- Score thresholds in CI. Fail a build on an absolute floor or a relative drop.
- Webhooks on findings. Push high-severity reviews and regressions to your own channels.
- Release tagging. Pass the release through
init()so every later chart can attribute the change.
Our cloud, or inside your network
Where the platform runs is a procurement question as much as a technical one, and it should not change how your engineers write code. It does not: the SDK call is identical either way.
NOMOO cloud
The fastest path from install to first trace. Nothing to provision, nothing to operate, and upgrades arrive without a maintenance window on your side. Projects are isolated, redaction runs in your process before anything is transmitted, and retention is set per project.
- Zero operational surface. No collector, queue, or database for you to run.
- Immediate start. Install the SDK and traces arrive on the first run.
Your own infrastructure
For teams whose agent arguments cannot leave their network, the platform runs inside your environment. Capture, storage, review, and optimization all stay within your boundary, and the SDK points at your endpoint instead of ours.
- Data stays inside. Spans never cross your network boundary.
- Same SDK, same views. One configuration value differs between the two modes.
What we do with what you capture
Agent traces contain the most sensitive data in the system, because arguments are whatever a user said plus whatever your tools looked up. The controls that matter are the ones that run before anything leaves your process.
| Control | Where it applies | What you decide |
|---|---|---|
| Redaction | In the SDK, before a span is buffered | Which fields, paths, or patterns are replaced at the source |
| Payload retention | Per project and environment | How long full arguments and results are stored |
| Aggregate retention | Per project | How long scores and trends outlive the payloads behind them |
| Sampling | At capture time | Whether every run is recorded in full, or a defined share of them |
| Residency | At deployment | Whether spans leave your network at all |
| Export | Continuously | Which systems receive a copy of the record you are building |
Redaction is the first thing to configure
Every other setting here has a defensible default. This one does not, because only your team knows which arguments carry regulated data. Decide the redaction rules on day one, before the first production trace, and the rest of the setup can wait.
Tell us what you are running
Describe your stack and we will walk through exactly what instrumenting it looks like, including the parts that would need work.