How to manage and monitor prompts
Once a prompt is created, it will be listed on the main interface. Follow the steps below to manage and monitor your prompts effectively.
Accessing prompts
Navigating to the prompt manager
Locate the main navigation bar at the top of the ZBrain interface
Click on the ‘Prompts’ tab to access the Prompt Manager
The prompt manager displays all your available prompts in a tabular format
Browsing existing prompts
The prompt manager provides a comprehensive view of all your prompts with the following information:
ID: Unique identifier for each prompt
Prompt name: Descriptive name of the prompt
Total runs: Number of times the prompt has been executed
Credits used: Number of credits consumed by the prompt
Total cost: Accumulated cost from all executions
Last updated: Timestamp of the most recent modification
Status: Current state of the prompt (Published/ Draft)
Actions: Access additional options via the three-dot menu to edit or delete a prompt.
Searching and filtering
Search: Use the search bar at the top to find prompts by name
Filter: Click the ‘Filter’ button and select show all to view all prompts, or filter by status (Published/Draft) to narrow your results.
Monitoring prompt performance
Accessing detailed analytics
From the prompt manager dashboard, click on a prompt name
This opens the prompt details page
Use the left sidebar navigation to access different pages:
Prompt overview: Performance metrics and usage statistics
Prompt editor: Modify prompt content and settings
User management: Control access permissions
Understanding the analytics dashboard
The prompt overview page provides rich performance data:
Usage metrics
Run over time: Graph showing execution frequency by date
Total logs: Number of recorded executions
Total tokens: Cumulative token usage across all executions
Total cost: Aggregate expense from all executions
Performance metrics
Average time: Mean processing duration in seconds
Median time: Middle value of all processing durations
Median cost: Middle value of all execution costs
Version management
Current version information is displayed near the prompt name (e.g., "Version: 3 of 3")
Use the version dropdown menu to switch between different iterations of the prompt
Each version maintains separate analytics, allowing for performance comparison
Reviewing execution logs
Prompt logs table
The detailed logs section provides a comprehensive record of each prompt execution:
Request ID
Unique identifier for each execution instance
Date and Time
When the prompt was executed (e.g., "2 mins ago")
Entity
The context where the prompt was used (e.g., "App", "Playground")
Entity Name
Specific entity identifier (e.g., "HR policy query app")
Evaluations
Any evaluation metrics
Duration
Execution time in seconds
Tokens
Number of tokens consumed in this execution
Cost
The financial cost of this specific execution
You can sort these logs by clicking on any column header to organize entries by that attribute. (e.g., by date, duration, tokens, or cost)
Best practices for prompt monitoring
Regular performance review
Monitor total cost and token usage to optimize budget allocation
Track execution times to identify performance bottlenecks
Compare performance across different versions to validate improvements
Cost optimization
Identify high-cost prompts and consider optimization opportunities
Analyze token usage patterns to refine prompt efficiency
Monitor median costs to establish baseline expectations for similar prompts
Usage pattern analysis
Use the ‘Run over time’ graph to identify usage patterns and peaks
Track which entities are most frequently using specific prompts
Identify underutilized prompts that might need upgrading or removal.
Troubleshooting
Investigate abnormal duration times that might indicate performance issues
Monitor for sudden increases in token usage that could signal inefficiencies
Check entity usage patterns to ensure prompts are being used as intended
By effectively leveraging these monitoring capabilities, teams can ensure their prompts remain performant, cost-effective, and aligned with business objectives.
Last updated