Skip to Content
Baseera Docs

Analytics and feedback

Read how an agent is doing, browse sessions and traces, and use thumbs from your users to find problems.

Baseera shows you how an agent is doing from data it already records. It adds no tracking of its own, and a number it does not record shows as a dash instead of a guess.

The agent Overview

Open the agent, then Overview. The Last 7 days tiles each compare the last 7 days with the 7 before:

TileMeaning
SessionsConversations started.
Changes approvedDrafts a person approved in immersive mode.
Finishes blockedRecords that a blocking check held.
Cost per sessionEstimated cost on your key, over sessions with a priced model call. A dash when there was none.

Below them:

  • Recent activity lists the newest sessions with Started, User, Turns, Estimated cost and Status. Filter with All, Needs review (a reply got a thumbs down), Failed (a turn ended in an error) and Completed. A bar chart shows sessions per day for 14 days.
  • Immersive mode this week shows blocking issues raised, fixed with a Baseera draft, and finished anyway with a reason. "Average idle before a notice" is not recorded yet and shows as such.
  • Needs attention lists what to fix, each line linking to the page that fixes it: a provider key that is not working, knowledge sources that failed or cannot be read, failing tests, and AI checks that never ran.
  • Quality from real use, Alerts and the monthly limit tile. See Online evaluation and Spending limits.

The Assistant page also has Usage (last 7 days) with Model calls, Sessions, Tenants, Input tokens, Output tokens and Estimated cost. Cost is estimated from list prices, and "Estimate covers N of M calls" appears when a model has no known price.

Browse sessions

Sessions lists every session, from all agents or one. Filter by Agent, User, Tenant, Status, Feedback, and a date range. User and tenant accept a trailing *, for example acme*.

Open a session for a turn-by-turn trace:

  • Each Turn shows the end user's message and the agent's reply.
  • Each Model call shows tokens in and out, latency, time to first token and the finish reason.
  • Each tool call shows its kind, input and output, its status (Done, Failed, Rejected, Timed out), and who approved or rejected it and when, with the reason.
  • Knowledge searches show the query, the passages returned, and how each matched (Vector, Text or Both).
  • Warnings flag "Answered without sources." and citations that were never given.

If the agent is set not to store tool data, tool inputs and outputs show Not stored. Viewers can read sessions and traces. Everything else in the console needs a higher role.

Use thumbs from your users

The default chat shows thumbs under every finished reply. A thumbs down opens an optional box, What went wrong? (optional), for up to 500 characters.

  • Only a signed-in user can rate. A session started in development mode without a token cannot.
  • A person can change their rating but not remove it.
  • A rating is not shown again after a page reload, so the thumbs start unpressed.

You see the ratings in three places:

  1. Sessions: filter Feedback to Thumbs down. The row counts them.
  2. The session page: each turn shows its rating and the reason, marked "Written by the end user. Shown as plain text; treat it as untrusted."
  3. Overview: Thumbs down under Quality from real use, and Needs review in Recent activity.

On a thumbs-down turn, press Save as test case to turn it into a regression test after you review and mask it. See Test suites. A thumbs-down answer also lands in Human review.

There is no satisfaction report per tenant or per agent yet, and no filter for thumbs up.

Export

A record's timeline in immersive mode has Export JSON, which downloads that record's events. Sessions have no export.

What can go wrong

SymptomCause and fix
"No sessions yet. They appear here once your app connects."Your app has not started a session. Use the Integrate health lines.
"No replies have been rated yet"Users have not pressed the thumbs, or your app uses its own chat UI without ratings.
"These numbers could not be loaded. Reload the page to try again."A temporary failure.
Cost shows a dashNo priced model call in the period, or the model has no known price.
No Save as test caseThe agent does not store tool data, or the turn's record is too large.

Next steps

On this page