# Monitor
### **Overview**
The Monitor feature in ZBrain delivers continuous, end-to-end oversight of every deployed agent and application. It automatically captures each input and output, evaluates the output against a comprehensive set of metrics, and surfaces real-time performance trends, so you can detect issues early, correct them quickly, and maintain consistently high-quality AI interactions.
Monitor schedule evaluations at defined intervals, log both successes and failures, and present the results in an intuitive console. Built-in notifications keep you updated on whether the flow is successful or fails.
#### Metric categories
* **Non-LLM Metrics -** Relies on deterministic checks (health, exact match, similarity) without invoking an LLM.
* **Health check** - Confirms the app/agent can return a valid response; halts further checks if an invalid response is received.
* **Exact match**- Compares the app/agent agent response character-by-character with the expected output.
* **F1 score:** Balances precision and recall to evaluate content overlap.
* **Levenshtein similarity:** Measures similarity based on edit distance between two strings.
* **ROUGE-L score:** Detects the longest common sequence between the response and reference text.
* **LLM-based -**Utilizes a language model to evaluate answers for relevance and factual accuracy.
* **Response relevancy** - Measures how accurately the response answers the user’s query.
* **Faithfulness** - Evaluates factual alignment with context to minimize hallucinations.
* **Performance** - Measures the total time (in milliseconds) taken by the LLM to return a response after receiving a query. It provides the **Response Latency** metric. Using this metric, users can monitor and enforce execution time thresholds for AI Agents, Flows, or Apps.
* Set thresholds in either seconds or minutes.
* When a threshold is breached or satisfaction is achieved, the system triggers *Success* or *Failure* evaluations and sends relevant notifications.
* **LLM-as-a-judge -** Have an LLM emulate human reviewers on traits like creativity, clarity, and helpfulness.
* **Creativity:** Rates originality in response generation.
* **Helpfulness:** Evaluates how effectively the response assists in resolving the user’s query.
* **Clarity:** Measures how clearly the message is conveyed.
### **Key capabilities of Monitor**
* **Automated evaluation**: Assess responses using LLM-based and non-LLM-based metrics.
* **Performance tracking**: Track success/failure trends.
* **Query-level monitoring**: Configure evaluations at the individual query level within a session.
* **Agent and app support**: Monitor both AI apps and AI agents.
* **Input flexibility**: Monitor responses for .txt, PDF, image, and other file types.
* **Notification alerts**: Enable real-time notifications for event status updates when an event succeeds or fails.
### **Monitor interface navigation**
The monitor module consists of four main sections, accessible from the left navigation panel:
* **Events**: View and manage all configured monitoring events
* **Monitor logs**: Review detailed execution results and metrics
* **Event settings**: Configure evaluation metrics and parameters
* **User management:** Configure role-based user permissions
Together, these capabilities provide a single dashboard for validating fixes, identifying quality drift, and ensuring that every user interaction meets your organization’s standards.
#### Accessing Monitor documentation
ZBrain Builder includes a context-aware documentation panel within the Monitor module.
* Click the '?' icon in the top-right corner of the Monitor interface.
* When clicked, it opens a side panel that displays documentation specifically tailored to the Monitor module, all without requiring you to navigate away from your current screen.
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://docs.zbrain.ai/zbrain-documentation/zbrain-builder/monitor.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
