One record of what your agents did, and one loop for making them better
NOMOO covers the whole lifecycle of an AI agent in production: capturing what happened, making it legible, and feeding what it learns back into the prompts and parameters that drive the next run. Eight capabilities, three stages, one continuous loop.
Observability is the input, not the product
Most tooling stops at the dashboard. A trace that nobody acts on is a cost, not an asset. NOMOO treats the execution record as the raw material for a closed loop: capture the run, score it, find the pattern across runs, propose the change, and measure whether it held.
Capture
Automatic instrumentation records every nested call, with its arguments, its result, its timing, and its place in the hierarchy.
Understand
Hierarchical views and live session streams make the run readable, and automated critique says what went wrong and where.
Improve
Patterns across many runs drive prompt and parameter proposals, reviewed by you and applied where they earn their place.
Verify
Scores keep accumulating after the change, so a fix that did not hold shows up as a regression rather than a belief.
Why the loop has to close
An agent is not a static artifact. Its behavior shifts when a model is upgraded, when a tool changes its response shape, when traffic brings questions nobody anticipated. A one-time evaluation describes the system you had last quarter. A standing loop describes the one you are running now.
Capture: instrument and trace
Nothing downstream works without a complete record, and a record nobody can be bothered to produce is worth nothing. So capture is deliberately the cheapest part of using NOMOO.
Instrument
Add the SDK, call init() with a project name, and pass your agent through instrument(). That is the integration. Your orchestration, control flow, and prompts stay exactly where they are.
Automatic instrumentation wraps the agent and follows it down: the planner it calls, the tools that planner selects, the HTTP request inside the tool, the model call that composes the answer. You did not declare any of that, and you do not maintain it as the agent changes.
- No manual spans. Standard agent, tool, and model calls are discovered at runtime.
- No infrastructure. There is no collector to deploy and no schema to migrate.
- Escape hatches where you want them. Mark your own functions as spans when a step matters to you and nothing in the stack would reveal it.
import nomoo from my_agents import support_agent nomoo.init( project="support-triage", environment="production", release="v1.7.2", ) agent = nomoo.instrument(support_agent) @nomoo.span(kind="tool") def lookup_order(order_id: str) -> dict: # captured with its arguments and return value return orders.get(order_id)
Trace
A NOMOO trace is a tree, not a log. Each span knows its parent, its children, its duration, its status, the arguments it received, and the value it returned. That structure is the difference between evidence and noise.
It is what lets you answer the question that actually matters during an incident: not "was there an error", but "which decision led here, and what did the agent know when it made it".
- Arguments and results, not summaries. The query that was sent and the hits that came back, side by side.
- Timing at every depth. A slow run resolves to the one subtree responsible for it.
- Status per step. Successes, warnings, and failures marked where they happened, not rolled up into one verdict.
Understand: visualize, monitor, evaluate
A complete record is necessary and not sufficient. Twelve thousand spans a day only help if a person can find the three that matter, and if something is reading the rest on their behalf.
Visualize
Interactive hierarchical views lay out parent and child relationships with execution flow, so the path from request to outcome is something you follow rather than reconstruct. Expand a branch to see what it cost, collapse the rest to keep the shape of the run in view.
Monitor
Real-time session monitoring streams live updates while agents are still working. Steps appear as they execute, which turns a four-minute run from a black box with a verdict at the end into something you can watch and interrupt.
Evaluate
An automated reviewer reads the run the way a careful engineer would: did retrieval support the answer, was a tool called twice for nothing, did the plan match the tools available. Findings carry a severity and attach to the exact span that caused them.
Critique that points at a line, not a score
A single number tells you something is wrong and nothing about what to do. NOMOO's automated analysis produces findings: a severity, the span responsible, a description of the behavior, and a suggested change.
Severity is what makes the volume survivable. Informational findings accumulate quietly for pattern analysis. Warnings sit in a queue. Errors are the ones worth paging someone about, and they stay rare because the other two tiers absorb the noise.
- Pinned to a span. Every finding resolves to a step you can open and read.
- Graded by severity. Information, warning, and error, so triage is possible at volume.
- Carries a suggestion. The finding says what to change, not only that something is off.
Unsupported claim in response. The answer states a 45 day refund window. No retrieved passage contains that figure and the highest scoring hit states 30 days. Suggestion: require a citation for any policy figure, and fail the response when retrieval returns no passage above the score threshold.
Redundant tool call. lookup_order was invoked twice with identical arguments inside one run, adding 221ms for no new information. Suggestion: memoize tool results per run, or add a cache guard on this tool.
Plan depth above baseline. This plan produced six sub-tasks against a trailing median of four. Not a fault on its own, and worth watching if it holds across the release.
Improve: detect, optimize, refine
Scored history is what turns observability into leverage. Once every run carries a score and a release tag, drift becomes measurable, improvement becomes provable, and the work of tuning stops depending on one engineer's intuition.
Detect
Performance is tracked across runs and releases. When behavior degrades, NOMOO flags the regression and attributes it to the change it followed, so the investigation starts with a suspect instead of a hunch.
Optimize
Continuous analysis of performance patterns drives automatic tuning of prompts and sampling parameters toward higher response quality, with every proposal traceable to the runs that motivated it.
Refine
The same loop applies to a network of agents. An orchestrator delegating to specialists is traced as one run, so you can tell a bad handoff from a bad worker.
A proposal you can read
Prompt changes arrive as a diff, next to the pattern that produced them and the score they are expected to move.
A regression with an address
Quality by release, so a drop is tied to the deploy that preceded it rather than argued about in a retrospective.
Mean score fell from 8.7 to 7.9 within an hour of the v1.6 rollout. Pattern analysis attributes most of the drop to plans that grew by two steps.
The data model
Everything in the platform rests on four objects. They are worth knowing, because they are the vocabulary of every view, filter, and export in the product.
| Object | What it represents | Carries |
|---|---|---|
| Project | One agent or agent network, scoped by environment and release. | Name, environment, release tag, redaction and retention settings |
| Session | A continuous interaction, which may contain several runs in sequence. | Session ID, start and end time, status, the runs it holds |
| Trace | One complete agent run, from the entry call to the final result. | Root span, total duration, outcome status, quality score, release |
| Span | A single captured call inside a run, nested under its parent. | Name, kind, arguments, return value, duration, status, parent and children |
| Kind | Captured for | Typical contents |
|---|---|---|
llm | Model calls, including planning and composition steps | Rendered prompt, parameters, token counts, completion |
tool | Any function the agent chooses to invoke | Arguments, return value, nested calls the tool itself made |
retrieval | Lookups against a vector store or search index | Query, filters, hit count, scores of returned passages |
fn | Plain application functions on the path of a run | Arguments, return value, duration |
Portable by construction
Spans map onto OpenTelemetry semantics, so a NOMOO trace can be exported to the systems your platform team already runs. Keeping the record portable is the point: instrumentation you cannot take with you is a liability dressed as a feature.
A week with the loop running
What the platform actually changes is the shape of the work. Less reconstruction, more reading. Fewer arguments about what happened, because the record settles them.
From ticket to span in two minutes
A user says the agent quoted the wrong refund window. You filter sessions by the account, open the run, and read the retrieval span: two passages, both about shipping, neither about refunds. The composition span cited neither. The cause is retrieval, not the model, and you knew that before standing up.
The deploy that announces itself
v1.6 ships at 14:00. By 14:40 mean quality has slipped eight tenths of a point and a regression is flagged against the release. Pattern analysis points at plan depth. The rollback decision takes one conversation instead of three days of anecdote.
The same warning, four hundred times
Redundant tool calls have been informational all week, and now they are the top finding by volume. The proposal is a cache guard on one tool. It is not a clever fix, and it would never have been prioritized without the count behind it.
A prompt edit with evidence attached
NOMOO proposes adding a citation requirement to the composition prompt, with the ungrounded-answer findings that motivated it. You read the diff, adopt it, and tag the release. Next week's scores say whether you were right.
What we optimize the product for
Every tool makes trades. These are ours, stated plainly, so you can tell quickly whether NOMOO fits how your team works.
Adoption cost beats feature count
Instrumentation that takes a sprint does not get installed, and tooling that is not installed observes nothing. One line is not a marketing number, it is the reason the rest of the platform ever gets used.
Evidence over verdicts
A score with no trace behind it cannot be argued with or acted on. Every number in NOMOO opens into the runs that produced it.
The engineer decides
Automation proposes prompt and parameter changes and shows its reasoning. Adopting them is a human decision, because the consequences of an agent's behavior land on humans.
No lock-in by design
Framework-agnostic capture, OpenTelemetry-shaped spans, and plain data export. The history you build with us stays yours and stays portable.
Out of the critical path
Capture is buffered and asynchronous. Observability that slows the thing it observes gets switched off during the incident it was bought for.
Defaults that are already right
Minimal configuration to start, with depth available when you want it. A tool that demands a configuration project before the first trace has mispriced your attention.
See the loop on your own agent
Bring the run you cannot explain. We will instrument it, read the trace together, and show you what the automated review finds in it.