Monitor OpenAI API with Python Agent

AppDynamics Python Agent helps you monitor the OpenAI API usage and related metrics. It monitors the ChatCompletion and Completion type requests created using the openai-python framework to the OpenAI backend.

Python Agent supports OpenAI <= 0.28.1.

The following metrics are reported at the application, tier, and node levels:

• Calls per minute
• Cost
• Errors per minute
• Tokens

The following are the example metric paths:

• Per model metric path - Application Infrastructure Performance|OpenAI_Tier|OpenAI|ada|Cost
• All model cumulative metrics path - Application Infrastructure Performance|OpenAI_Tier|OpenAI|Cost - All Models
• Net application level metrics path - Overall Application Summary|OpenAI - Cost - ada

These metrics are reported even if the exit call is outside the business transaction. However, the Average Response Time(ms) metric is not reported when the exit call is outside the business transaction.

Prerequisites

Ensure that you meet the following prerequisites to monitor the OpenAI API exit calls with AppDynamics:

• Install Python Agent 23.8.0 or later. See Install Python Agent.
• Import custom dashboards and templates.

Create a Custom Dashboard

1. Download Custom_Dashboard_v1.0.zip. It contains the following four files:
   

   Filename	Description
   template_app.json	The template to create the application-level dashboard.
   template_tier.json	The template to create the tier-level dashboard.
   main.py	Python script to create the dashboards.
   readme.md	Description of the solution.

   
   

2. Run the following command:

   cd Custom_dashboard_script
   python3 main.py

   CODE

   The preceding command prompts you to specify the dashboard type (application or tier) that you want to create.

3. Based on your selected dashboard type, you will be prompted to specify the following dashboard details.
   

   Variable	Description	Default	Possible Values	Required
   Application Name	The application created in the Controller with OpenAI monitoring enabled.	none	openai_app_1	Yes
   OpenAI Endpoints Host Name	The hostname of the OpenAI backend.	api.openai.com	api.openai.com	Optional
   OpenAI Endpoints Port	The port of the OpenAI backend.	443	443	Optional
   Dashboard Name	The dashboard name to be displayed on Dashboard & Reports in the Controller UI.	openai_app_dashboard	openai_app_dashboard	Optional
   Output File Name	The output dashboard .json filename.	app_dashboard.json	app_dashboard.json	Optional

   Variable	Description	Default	Possible Values	Required
   Application Name	The application created in the Controller with OpenAI monitoring enabled.	none	openai_app_1	Yes
   Tier Name	Name of the Tier exposed to OpenAI backend calls.	none	openai_tier_1	Yes
   Business Transaction Name	The name of the business transaction reported to the host.	openai_tier_bt_name	openai_bt	Optional
   OpenAI Endpoints Host Name	The hostname of the OpenAI backend.	api.openai.com	api.openai.com	Optional
   OpenAI Endpoints Port	The port of the OpenAI backend.	443	443	Optional
   Dashboard Name	The dashboard name to be displayed on Dashboard & Reports in the Controller UI.	openai_tier_dashboard	openai_tier_dashboard	Optional
   Output File Name	The output dashboard .json filename.	tier_dashboard.json	tier_dashboard.json	Optional

   
   

4. Import the dashboard in the Controller UI. See Create and Manage Custom Dashboards and Templates.