Middleware and observability

After this chapter, you will be able to add cross-cutting behavior with Microsoft Agent Framework middleware and turn on OpenTelemetry tracing without burying operational concerns inside agent instructions or tool code.

The examples target the verified Python package agent-framework==1.9.0 and run without live model keys by short-circuiting network calls or disabling exporters.

Middleware is a request/response pipeline around work the framework already knows how to do: running an agent, calling a chat client, or invoking a function tool. A middleware component can inspect the context before the call, continue with await call_next(), inspect or transform the result afterward, or deliberately stop the pipeline with a local result.

Observability is the companion operational overlay: traces, metrics, and logs that help you answer what happened, where time was spent, which path a workflow took, and why a run failed. Microsoft Learn describes Agent Framework observability as built-in monitoring support for understanding agent behavior and diagnosing issues.^{Microsoft Learn}

This is the same idea Chapter 2 called operational overlays. Middleware and telemetry do not choose your agentic pattern. They make a single agent, tool loop, or workflow safer to operate by keeping policy, logging, redaction, retries, and diagnostics outside the business logic.

Agent Framework Python 1.9.0 exposes three middleware layers plus OpenTelemetry setup. Microsoft Learn documents middleware as a way to intercept and customize agent behavior, and the verified package API supplies both class-based middleware and decorator-based middleware functions.^{Microsoft Learn}

Middleware receives a mutable context and a call_next callback. If the middleware sets context.result and raises MiddlewareTermination, agent and chat pipelines can return without making the remaining call. Use this deliberately for fakes, cache hits, or policy blocks; use ordinary exceptions for real failures.

from agent_framework import Agent, ChatResponse, Content, Message, MiddlewareTermination
from agent_framework import agent_middleware, chat_middleware
from agent_framework.openai import OpenAIChatClient

@agent_middleware
async def log_agent_run(context, call_next):
    print(f"agent middleware saw {len(context.messages)} message(s)")
    context.metadata["chapter"] = "09"
    await call_next()
    print(f"agent middleware result: {context.result.text}")

@chat_middleware
async def local_chat_response(context, call_next):
    context.result = ChatResponse(
        messages=Message(role="assistant", contents=[Content.from_text("local chat response")]),
        model="middleware-local",
    )
    raise MiddlewareTermination(result=context.result)

client = OpenAIChatClient(model="not-called", api_key="placeholder")
agent = Agent(client=client, name="middleware-demo", middleware=[log_agent_run, local_chat_response])

For observability, use the installed setup function configure_otel_providers. The no-network example below disables console exporters and sensitive telemetry. Supplying real OTLP exporters, Azure Monitor, Langfuse, or other exporter destinations is a credential/exporter-requiring production step.

from agent_framework.observability import (
    OBSERVABILITY_SETTINGS,
    configure_otel_providers,
    enable_instrumentation,
    get_meter,
    get_tracer,
)

configure_otel_providers(enable_console_exporters=False, enable_sensitive_data=False)
enable_instrumentation(enable_sensitive_data=False, force=True)

tracer = get_tracer("playbook.ch09")
meter = get_meter("playbook.ch09")
counter = meter.create_counter("playbook_ch09_observability_runs")
counter.add(1, {"example": "observability_setup"})

with tracer.start_as_current_span("playbook.ch09.no_network_probe") as span:
    span.set_attribute("example.chapter", "09")

print(f"sensitive telemetry enabled: {OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED}")

Common pitfalls

• Putting policy only in prompts: instructions can guide behavior, but middleware can enforce application-side checks before or after framework calls.
• Choosing the wrong layer: use agent middleware for whole-run concerns, chat middleware for provider request/response concerns, and function middleware for tool-call validation or audit.
• Hiding business behavior in middleware: middleware should be reusable cross-cutting logic, not the primary domain workflow.
• Leaking telemetry data: prompts, messages, arguments, and outputs can contain sensitive data. In the verified 1.9.0 package, sensitive telemetry is off by default; keep enable_sensitive_data=False in production unless you have explicit approval and controls.
• Forgetting exporter requirements: local provider setup can run without credentials, but real OTLP or cloud exporters need configured endpoints, credentials, and operational ownership.
• Configuring OpenTelemetry repeatedly: call configure_otel_providers once at startup, before agent or workflow execution.

The chapter examples are intentionally runnable without live keys. middleware_demo.py constructs an OpenAIChatClient with a placeholder key, but the chat middleware raises MiddlewareTermination with a local ChatResponse, so the provider is never called. observability_setup.py configures local OpenTelemetry providers with exporters disabled and sensitive telemetry disabled.

1. Run python backend\examples\ch09\middleware_demo.py to see the agent middleware wrap a chat middleware short-circuit.
2. Run python backend\examples\ch09\observability_setup.py to verify tracing and metrics setup without exporting telemetry.
3. For production observability, add a real exporter or environment variables such as OTEL_EXPORTER_OTLP_*; that step requires an endpoint and credentials.

Expected behavior: the middleware demo prints that the agent middleware saw one message, the chat middleware short-circuited the request, and the final response is local chat response. The observability demo prints that instrumentation is enabled and sensitive telemetry is disabled.

• Middleware is the request/response pipeline for cross-cutting behavior around agent runs, chat/model calls, and function/tool invocations.
• Use AgentMiddleware, ChatMiddleware, and FunctionMiddleware or their decorators when the concern should stay outside agent instructions and domain tools.
• Use configure_otel_providers once at startup to wire OpenTelemetry providers; keep sensitive telemetry disabled unless explicitly approved.
• The verified examples for this chapter run without live credentials; real exporters and cloud destinations require configuration and credentials.

Next, Chapter 10 moves from local code to declarative agents, hosting, and DevUI so the same agent behaviors can be configured, run, and debugged in richer environments.