Add centralized OpenTelemetry tracing for MCP tool execution#3111
Add centralized OpenTelemetry tracing for MCP tool execution#3111
Conversation
Co-authored-by: souvikghosh04 <210500244+souvikghosh04@users.noreply.github.com>
Co-authored-by: souvikghosh04 <210500244+souvikghosh04@users.noreply.github.com>
…EntityName property Co-authored-by: souvikghosh04 <210500244+souvikghosh04@users.noreply.github.com>
Co-authored-by: souvikghosh04 <210500244+souvikghosh04@users.noreply.github.com>
Co-authored-by: souvikghosh04 <210500244+souvikghosh04@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds centralized OpenTelemetry tracing around MCP tool execution so both built-in and dynamic custom MCP tools emit consistent spans across stdio and HTTP transports.
Changes:
- Added
McpTelemetryHelper.ExecuteWithTelemetryAsyncto start/finish spans, set tags, and record exceptions with MCP-specific error codes. - Wired telemetry wrapper into both MCP dispatch entry points (
McpStdioServerand HTTPCallToolHandler). - Added unit tests for telemetry helpers and exposed
DynamicCustomTool.EntityNameto avoid reflection.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Azure.DataApiBuilder.Mcp/Utils/McpTelemetryHelper.cs | Central telemetry wrapper + operation inference + error-code mapping + custom-tool metadata extraction |
| src/Azure.DataApiBuilder.Mcp/Utils/McpTelemetryErrorCodes.cs | Defines MCP telemetry error-code constants |
| src/Core/Telemetry/TelemetryTracesHelper.cs | Adds Activity extension methods for MCP tool tags/status/exception recording |
| src/Azure.DataApiBuilder.Mcp/Core/McpStdioServer.cs | Wraps stdio tool execution with telemetry helper |
| src/Azure.DataApiBuilder.Mcp/Core/McpServerConfiguration.cs | Wraps HTTP tool execution with telemetry helper |
| src/Azure.DataApiBuilder.Mcp/Core/DynamicCustomTool.cs | Exposes EntityName property and updates internal usage |
| src/Service.Tests/UnitTests/McpTelemetryTests.cs | Adds unit tests validating MCP telemetry tagging/status/error mapping |
| src/Service.Tests/Azure.DataApiBuilder.Service.Tests.csproj | Adds project reference to MCP project for unit testing |
| src/Azure.DataApiBuilder.Mcp/Azure.DataApiBuilder.Mcp.csproj | Adds InternalsVisibleTo for Azure.DataApiBuilder.Service.Tests |
|
@copilot please go through the open review comments which are in Unresolved state and work on the fixes or implementation.
|
- Add AUTHORIZATION_FAILED error code separate from AUTHENTICATION_FAILED - Use tool type to infer operations: built-in tools mapped by name, custom tools always "execute" - Map describe_entities to "describe" operation (not "read") - Extract error.code and error.message from CallToolResult when IsError=true - Catch all exceptions in ExtractCustomToolMetadata for best-effort metadata - Update MapExceptionToErrorCode to handle DataApiBuilderException auth/authz cases - Make _recordedActivities readonly in tests - Use McpTelemetryErrorCodes constants in tests - Rename FakeMcpTool to MockMcpTool - Test all DML tools (read, create, update, delete, describe, execute) - Focus tests on ExecuteWithTelemetryAsync rather than individual TrackXX methods - Custom tools (stored procedures) now correctly return "execute" operation Co-authored-by: souvikghosh04 <210500244+souvikghosh04@users.noreply.github.com>
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
…ub.com/Azure/data-api-builder into copilot/add-telemetry-for-custom-tools
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
|
@copilot I want to run the DAB locally in Aspire. the project already supports as it has inbuilt references. could you run it and make some MCP calls so that we can see the logs and telemetry info in Aspire? |
|
@copilot I want to run the DAB locally in Aspire. the project already supports as it has inbuilt references. could you run it here in my VS code locally and make some MCP calls so that we can see the logs and telemetry info in Aspire? |
I've reviewed the Aspire setup in
However, this would require:
Would you prefer:
The unit tests in commit 98105c3 already verify that |
|
@copilot this is not a follow to any comments. |
Why make this change?
MCP tool executions (both built-in DML tools and dynamic custom stored-procedure tools) currently lack explicit OpenTelemetry tracing. While
ILoggerlogging exists, there are no tracing spans emitted for MCP tool invocations. This makes it difficult to observe and diagnose tool execution in distributed tracing systems.What is this change?
Added centralized OpenTelemetry tracing at the MCP dispatcher level (where tools are resolved and
ExecuteAsyncis invoked), ensuring consistent spans for all MCP tools without code duplication.Span Attributes
mcp.tool.namedab.entitydab.operationdb.procedureOperation Inference
read_records→readcreate_record→createupdate_record→updatedelete_record→deletedescribe_entities→describeexecute_entity→executeexecute(fallback)executeoperation (determined by ToolType.Custom)Outcome Recording
ActivityStatusCode.OkActivityStatusCode.ErrorwithRecordException()anderror.code/error.messagetagsIsError=true, error code/message extracted from JSON inCallToolResult.Contentand set as span tags for diagnosabilityError Codes
EXECUTION_FAILEDAUTHENTICATION_FAILEDAUTHORIZATION_FAILEDDATABASE_ERRORINVALID_REQUESTOPERATION_CANCELLEDCoverage
Spans cover all paths:
Implementation Details
TelemetryTracesHelper.DABActivitySource("DataApiBuilder")ExecutionHelperwhereDABActivitySource.StartActivity()wraps query executionMcpTelemetryHelper.ExecuteWithTelemetryAsyncwrapper at the MCP dispatcher levelHow was this tested?
Sample Request(s)
N/A - Telemetry instrumentation is transparent to API consumers. Spans are emitted to configured OpenTelemetry exporters with the attributes and error codes documented above.
Example telemetry span for a successful built-in tool:
Example telemetry span for a custom stored procedure:
Example telemetry span for an authorization failure:
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.