How to add custom fields (e.g. user_email) to BigQueryAgentAnalyticsPlugin events?

[rohan-kavari]

I am using BigQueryAgentAnalyticsPlugin for production observability and would like to enrich the BigQuery events with custom per request fields such as user_email, country, or tenant_id.

Current setup

I am already using the plugin successfully with the default schema. Below is a simplified version of my current code.

# my_bq_agent/agent.py
import os
import google.auth
from google.adk.apps import App
from google.adk.plugins.bigquery_agent_analytics_plugin import (
    BigQueryAgentAnalyticsPlugin,
    BigQueryLoggerConfig,
)
from google.adk.agents import Agent
from google.adk.models.google_llm import Gemini

# --- OpenTelemetry Initialization (Optional) ---
try:
    from opentelemetry import trace
    from opentelemetry.sdk.trace import TracerProvider
    trace.set_tracer_provider(TracerProvider())
except ImportError:
    pass

PROJECT_ID = os.environ.get("GOOGLE_CLOUD_PROJECT", "your-gcp-project-id")
DATASET_ID = os.environ.get("BIG_QUERY_DATASET_ID", "your-big-query-dataset-id")
LOCATION = os.environ.get("GOOGLE_CLOUD_LOCATION", "US")

bq_config = BigQueryLoggerConfig(
    enabled=True,
    log_multi_modal_content=False,
    max_content_length=500 * 1024,
    batch_size=1,
    shutdown_timeout=10.0,
)

bq_logging_plugin = BigQueryAgentAnalyticsPlugin(
    project_id=PROJECT_ID,
    dataset_id=DATASET_ID,
    table_id="agent_events",
    config=bq_config,
    location=LOCATION,
)

llm = Gemini(model="gemini-2.5-flash")

root_agent = Agent(
    model=llm,
    name="dad_joke_agent",
    instruction="You are a helpful assistant which only replies dad jokes.",
)

app = App(
    name="app",
    root_agent=root_agent,
    plugins=[bq_logging_plugin],
)

What I want to achieve

At request start, I store metadata like this in the session state:

state["user_email"] = "user@example.com"
state["country"] = "IN"

I would like these fields to appear as first class columns in BigQuery so that I can:

• Filter traces by user or tenant
• Build dashboards
• Run analytics and audits

Questions

1. What is the recommended way to attach custom fields to events written by BigQueryAgentAnalyticsPlugin?

2. Is there a supported hook or callback to enrich the event payload before it is flushed to BigQuery?

3. Should this be done by:

   • Subclassing BigQueryAgentAnalyticsPlugin
   • Overriding a specific callback (e.g. before_tool_callback, after_agent_callback)
   • Mutating a known event.metadata or event.attributes structure

4. If the BigQuery schema needs to be extended:

   • Which event fields are safe to add custom columns to?
   • Is there an example of mapping ToolContext.state → BigQuery columns?

A minimal example of extending the plugin or enriching the event would be extremely helpful.

[caohy1988]

How to Enrich BigQuery Agent Analytics Events

The recommended way to attach custom fields (like user_email, tenant_id) to your BigQuery events is to leverage the attributes JSON column rather than extending the BigQuery table schema.

This approach is flexible, avoids schema migration headaches, and is fully queryable in BigQuery.

1. Recommended Approach: Subclassing the Plugin

You can subclass BigQueryAgentAnalyticsPlugin and override the internal _log_event method. This method acts as a central hook for all events (tool calls, user messages, model responses, etc.), allowing you to inject custom data from the session state into the event attributes.

Implementation Example

from typing import Any
from google.adk.plugins.bigquery_agent_analytics_plugin import BigQueryAgentAnalyticsPlugin
from google.adk.agents.callback_context import CallbackContext

class EnrichedBigQueryPlugin(BigQueryAgentAnalyticsPlugin):
    """A custom BQ plugin that enriches events with session state metadata."""

    async def _log_event(
        self, 
        event_type: str, 
        callback_context: CallbackContext, 
        **kwargs: Any
    ) -> None:
        """Injects custom context from session state into event attributes."""
        
        # Safely access the session state
        if hasattr(callback_context, "session") and callback_context.session:
            state = callback_context.session.state
            
            # extract specific fields you want to promote to top-level attributes
            # These will appear in the 'attributes' JSON column in BigQuery
            if "user_email" in state:
                kwargs["user_email"] = state["user_email"]
            
            if "tenant_id" in state:
                kwargs["tenant_id"] = state["tenant_id"]
                
            if "country" in state:
                kwargs["country"] = state["country"]

        # Call the parent method to handle the actual logging
        await super()._log_event(event_type, callback_context, **kwargs)

Usage

Replace the standard plugin with your custom class in your application setup:

# Initialize your custom plugin
bq_logging_plugin = EnrichedBigQueryPlugin(
    project_id=PROJECT_ID,
    dataset_id=DATASET_ID,
    table_id="agent_events",  # You can use the same table
    config=bq_config,
    location=LOCATION,
)

# ... setup app ...
app = App(
    name="app",
    root_agent=root_agent,
    plugins=[bq_logging_plugin],
)

2. Querying the Data

Data injected into kwargs will automatically be serialized into the attributes JSON column. You can query these fields as if they were first-class columns using BigQuery's JSON_VALUE function.

SELECT
  timestamp,
  event_type,
  JSON_VALUE(attributes.user_email) AS user_email,
  JSON_VALUE(attributes.tenant_id) AS tenant_id,
  JSON_VALUE(attributes.country) AS country,
  -- You can still filter on them efficiently
FROM
  `your-project.your_dataset.agent_events`
WHERE
  JSON_VALUE(attributes.tenant_id) = 'tenant-123'

Answers to your specific questions

• Recommended way: Use the attributes JSON column. It creates a "schema-on-read" pattern that is perfect for observability data where metadata might evolve.
• Hook/Callback: Override _log_event in a subclass. This is safer than overriding individual callbacks (on_tool_start, etc.) because it catches every event type centrally.
• Schema Extension: We strongly discourage adding new top-level columns (e.g., modifying the table schema to add a user_email column).
  • Why? The plugin's internal BatchProcessor and PyArrow serialization are tightly coupled to the defined schema. Modifying it requires overriding private methods (_bq_schema, _prepare_arrow_batch) and makes upgrades difficult.
  • The attributes column is designed specifically for this "custom fields" use case.