# How to configure a monitoring event for apps {% embed url="" %} Quick 'How-to' video with steps to monitor ZBrain apps {% endembed %} ### **Step 1: Access the monitoring configuration** To set up monitoring for an application: **Access the app session:** * Navigate to the Apps page * Click on your desired application * Go to the query history section of the app * Select a specific user session from the list to monitor (shows session ID, user info, prompt count)
**Review the conversation:** * View the session details and chat history. You can configure monitoring events at the query level. For instance, in a conversation containing multiple queries, each query can have its own monitoring event * Examine the response options (copy, conversation logs, edit annotation reply and feedback)
**Access conversation logs:** * Click 'Conversation Log' to see the interaction details of a specific query * Review status, time, and token usage * Check the input, output, and metadata
**Enable monitoring:** * Click the ‘Monitor’ button in the overview tab * Click ‘Configure now’ when prompted with ‘Added for monitoring’
### **Step 2: Configure event settings** You will be redirected to the Events > Monitor page. In the last status column, click ‘Configure’ to open the event settings page. On the event settings screen:
1. **Review entity information** * **Entity name:** The name of your application * **Entity type:** The type of entity being monitored (e.g., App) 2. **Verify monitored content** * **Monitored input:** The query or prompt being evaluated * **Monitored output:** The response being assessed 3. **Set evaluation frequency** * Click the dropdown menu under "Frequency of evaluation." * Select the desired interval (Hourly, Every 30 minutes, Every 6 hours, Daily, Weekly, or Monthly)
4. **Configure evaluation conditions** * Click ‘Add metric’ in the Evaluation Conditions section * Select a metric type from the available metric types: * Non-LLM * LLM-based * LLM-as-a-judge * Performance
* Choose the evaluation method (is less than, is greater than, or equals to) * Set the threshold value (0.1 to 5.0) * Click ‘Add’ to save the metric
* Set the "Mark evaluation as" dropdown to fail or success
5. **Configure notifications**
* Toggle the ‘Send Notification’ option to enable alerting for this monitoring event.
* Click ‘+ Add a Flow from the Flow Library’. This opens the ‘Add a Flow’ panel. This interface allows you to configure notification flows through a dual-tabbed layout. * The side panel includes two tabs: * Default: Contains predefined ZBrain template Flows. * Custom: Displays custom-built Flows created. * Choose a Flow from either the Default or Custom tab based on your notification needs. * The interface prevents duplication by restricting the addition of the same Flow across multiple notifications. * Flows that have already been assigned will be unavailable for selection, ensuring each notification uses a unique Flow.
* In the panel, search for the desired notification flow and select it. * You can configure multiple notification flows for a single monitoring event, allowing you to send alerts simultaneously across various channels. * The system prevents adding the same flow more than once to avoid redundant alerts or conflicts. * Upon meeting the evaluation trigger conditions, all attached flows are executed in parallel.
* You can directly edit linked notification flows from the **Event Settings** screen, allowing you to quickly update or troubleshoot notification logic without needing to navigate away from the monitoring configuration, as it directly redirects you to the Flow page. * Click on the pencil icon ![memo](https://leewayhertz.atlassian.net/gateway/api/emoji/86e7216a-2b63-4398-aefe-391cdd46a4ec/1f4dd/path?scale=MDPI) beside the selected Flow. * It will redirect you to the defined Flow page, and here you can modify the logic.
Click the Play ▶️ button to run a delivery test.
**Note:** Users cannot update the event settings until a valid notification flow passes the delivery test.\ Once the flow passes the delivery test, notifications will be sent via the chosen communication channel whenever the associated event fails. ### Test Evaluation Settings in event monitoring You can test the evaluation settings configuration from the **Test Evaluation Settings** panel within the page. This panel allows you to choose between using your own manual test input or relying on system-generated LLM outputs when running evaluation tests. The feature provides flexibility in simulating different test scenarios across **Agents, Apps, and Reasoning** configurations. **Note: Apps & Reasoning**: Manual input is optional and turned off by default. To test the settings * Navigate to Monitor and select an event. * Click **Event Settings,** which will open **Event Monitoring Settings** * Once you configure all settings, click the **Test** button to open the **Test Evaluation Settings** panel on the right. * Toggle **Manually input test value** to ON. A **Test Message** input field will appear. * When **Manual Input is** ON, the evaluation compares the LLM output with your custom message using configured metrics. * Enter your custom reference message for comparison. * When **Manual Input is** OFF, the evaluation runs only against the system-generated LLM output and metrics.
* Once you click 'Test', it validates the configuration in monitoring tests by ensuring that the model and temperature values match the exact configuration defined in the associated app. The metrics now dynamically align with the actual model selected in the entity, making evaluation results more realistic and aligned with runtime behavior.
* To receive a notification for the test evaluation report immediately after a test, enable the Send Notification toggle in the Test Evaluation Settings panel of the monitoring module. This toggle is available on the right side of the Test Evaluation Settings panel.
* Select a Flow to send an email notification. If no flow is selected, it still appears but gives a validation error*.*
* Hovering over the (i) icon displays the message: *‘If enabled, you’ll receive a notification with the evaluation report,’* ensuring clarity for the user.
* Click the ‘**Reset’** button to clear the input if needed. * Run the Test and review results * Save your configuration and click ‘Update’ to save and activate your monitoring event