Complete visibility into every execution
The NOMOO SDK captures detailed information about every run: nested function calls, the arguments passed, the results returned, and the hierarchical relationships between each step. What was an opaque autonomous process becomes a record you can read, search, and hand to a colleague who was not there.
What a trace actually contains
Instrumentation is only as useful as the fields it keeps. A span that records a name and a duration tells you an agent was slow. A span that records the arguments and the return value tells you why.
| Field | Meaning | Why it matters |
|---|---|---|
| Name | The qualified call that produced the span. | Lets you group identical steps across thousands of runs. |
| Kind | Model call, tool, retrieval, or plain function. | Different kinds get different review rules and different charts. |
| Arguments | The values the call received, structured. | Reproducing a failure starts with the exact inputs. |
| Return value | What the call handed back to its caller. | Shows whether a step succeeded in substance, not only in status. |
| Duration | Wall-clock time for the span and its subtree. | Attributes latency to a branch instead of the whole run. |
| Status | Success, warning, or error at that step. | A run can succeed overall while a step inside it quietly failed. |
| Parent and children | Position in the hierarchy of the run. | Turns a list of events into a decision path. |
| Timestamps | Start and end, to the millisecond. | Reveals serial work that could have run concurrently. |
Captured automatically
Agent entry points, planning and composition calls to a model, tool invocations the agent selects, retrieval against a vector store, and the HTTP or database work nested inside those tools.
Captured when you ask
Your own functions, marked with a decorator where a step matters to your domain and nothing in the stack would otherwise reveal it. One line per function, and it inherits the hierarchy around it.
Never captured
Fields matched by your redaction rules, which are applied in the SDK before anything leaves your process. A redacted value is replaced, not stored and hidden.
One run, read from the top
This is a support agent answering a refund question. Twenty-three spans, 1.8 seconds, one warning. Everything a reviewer needs is on the screen at once: the shape of the run on the left, the contents of the selected step on the right.
tools.search_kb
query: "refund window",
top_k: 5,
filters: { locale: "en" }
}
hits: 5,
top_score: 0.92,
source: "policy/refunds#30d"
}
children 3 · subtree 482ms
Relevant passages retrieved on the first attempt. The composed answer cites the top hit, so grounding holds.
Read it the way you would read a stack trace
Start at the root and descend into whatever looks wrong. The planner took 312 milliseconds and produced four sub-tasks, which is normal for this agent. Retrieval took 486 and returned a high-scoring policy passage, which is the outcome you wanted.
Then there is the warning. lookup_order queried the same order twice, 31 milliseconds of waste and a sign that nothing is guarding the call. Small on one run, and this agent handles forty thousand a day.
- Depth is meaning. Indentation shows which step caused which, so causality is visible without reading code.
- Timing rolls up. Each span carries its own duration and the duration of everything beneath it.
- Status does not hide. A warning deep in a subtree surfaces on the branch that contains it.
- Tags separate concerns. Model calls, tools, and retrieval are distinguishable at a glance.
- Release travels with the run. Every trace is attributable to the version of the agent that produced it.
The arguments and the result, side by side
Selecting a span opens everything that was captured for it. This is where most debugging sessions end, because the inputs and the outputs of a step are usually enough to explain it.
Why structure beats a log line
A log statement is a sentence someone wrote in advance about what they expected to need. It flattens nested values, drops the fields nobody anticipated, and separates the call from its result by however many lines of unrelated output happened in between.
The inspector keeps the call intact. Arguments stay structured, including nested objects and filters. The return value sits directly beneath them. The position in the run is stated, so you can move up to the caller or down into the work the step delegated.
- Structured values. Nested arguments and results stay navigable instead of being stringified.
- Review attached. Automated findings for the span appear with it, not in a separate queue.
- Navigable hierarchy. Jump to the parent call or into any child from the inspector itself.
respond.compose
temperature: 0.3,
top_p: 0.9,
max_tokens: 512
}
prompt: 1,284,
completion: 96
}
text: "Your refund window is 30 days from delivery...",
citations: ["policy/refunds#30d"]
}
Every policy figure in the response appears in a cited passage. No unsupported claims found.
Runs in context, live as they happen
A single trace explains one run. A session explains a conversation: several runs in sequence, where the interesting failure is often the handoff between them rather than anything inside one.
Watch a run instead of waiting for it
Real-time session monitoring streams updates while agents are working. Spans appear as they open and close, so a long investigation is something you observe in progress rather than a verdict delivered four minutes later.
That matters most when an agent takes actions. If a run is about to apply the wrong shipping method, the useful moment to know is before the write, not in tomorrow's report.
- Live span stream. Steps surface the moment they execute, with partial results visible.
- Session grouping. Consecutive runs for the same interaction stay together with their shared context.
- Status at a glance. Running, complete, warned, and failed sessions are distinguishable in one list.
Finding the three runs that matter
Volume is the thing that breaks most observability setups. Forty thousand runs a day is not a browsing problem, it is a query problem, so traces are filterable along the dimensions that actually separate a bad run from a normal one.
| Dimension | Typical use |
|---|---|
| Project and environment | Separate production traffic from staging while comparing the two. |
| Release tag | Isolate the runs produced by one deploy to confirm or clear a regression. |
| Status | Pull only runs that warned or failed, at any depth in the hierarchy. |
| Quality score | Review the lowest-scoring one percent, which is where new failure modes appear first. |
| Duration | Find the slow tail, then read which subtree the time went into. |
| Span name | Collect every run that touched one tool, to judge whether that tool is the problem. |
| Finding type | Group by the automated critique, for example every ungrounded answer this week. |
| Metadata | Filter on your own attached fields, such as tenant, locale, or plan tier. |
Attach your own dimensions
Metadata you set at init() or per run travels with every span in that run. Tag by tenant, locale, customer tier, or experiment arm, and every filter and chart in the product can then be cut along the lines your business actually cares about. See attaching metadata for the call.
Your traces leave the way they came in
Spans follow OpenTelemetry semantics, so a NOMOO trace can be exported to the collector your platform team already runs. Agent runs end up in the same pane as the rest of your services, and the correlation between an agent's tool call and the service it called is not lost at the boundary.
Beyond telemetry, runs can be streamed downstream as structured data: into a warehouse for analysis, into a review queue, or into an evaluation set you maintain yourself. The instrumentation you install is not a one-way door.
- OpenTelemetry-shaped spans. Export to an existing collector without a translation layer.
- Structured run export. Stream complete traces with their scores and findings attached.
- Webhooks on findings. Push high-severity reviews to the channel your on-call already watches.
import nomoo nomoo.init( project="support-triage", otel_endpoint="http://collector.internal:4317", otel_service_name="support-agent", ) # spans are delivered to NOMOO and to your collector # no second instrumentation pass, no duplicate spans
Capture everything useful, store only what you should
An agent's arguments are the most sensitive data your system handles, because they are whatever the user said plus whatever your tools looked up. Deciding what is kept is a configuration question, and it belongs to you.
Redaction in the SDK
Rules are applied in your process, before a span is buffered. Match by field name, by path, or by pattern, and the value is replaced at the source rather than stored and masked later.
Retention you set
Choose how long full span payloads are kept, per project and per environment. Aggregates and scores can outlive the payloads, so trend history survives a short retention window.
Where it runs
Run against our cloud or inside your own network. The deployment choice does not change the SDK call or the shape of the data you get back.
Set redaction before you set anything else
The one configuration worth doing on day one is deciding which fields never leave your process. Everything else in NOMOO has a sensible default. This does not, because only your team knows which arguments carry regulated data.
Read the run you cannot explain
Bring one agent and one failure you have not been able to pin down. We will instrument it and open the trace with you.