Exa Observability

Overview

Monitor Exa AI search API performance, result quality, and cost efficiency. Key metrics include search latency (Exa neural search typically takes 500-2000ms), result relevance (measured by click-through or downstream usage), search volume by type (neural vs keyword vs auto), per-search cost tracking, and cache hit rates for repeated queries.

Prerequisites

• Exa API integration in production

• Metrics backend (Prometheus, Datadog, or equivalent)

• Request logging infrastructure

Instructions

Step 1: Instrument the Exa Client

import Exa from 'exa-js';

async function trackedSearch(exa: Exa, query: string, options: any) { const start = performance.now(); try { const results = await exa.search(query, options); const duration = performance.now() - start; emitHistogram('exa_search_duration_ms', duration, { type: options.type || 'auto' }); emitCounter('exa_searches_total', 1, { type: options.type || 'auto', status: 'success' }); emitGauge('exa_results_count', results.results.length, { type: options.type || 'auto' }); return results; } catch (err: any) { emitCounter('exa_searches_total', 1, { status: 'error', code: err.status }); throw err; } }

Step 2: Track Result Quality

// Measure whether search results are actually used by downstream consumers function trackResultUsage(searchId: string, resultIndex: number, action: 'clicked' | 'used_in_context' | 'discarded') { emitCounter('exa_result_usage', 1, { action, position: String(resultIndex) }); // Results at position 0-2 should have high usage; if not, query needs tuning }

Step 3: Monitor Search Budget

set -euo pipefail

Check remaining search quota

curl -s https://api.exa.ai/v1/usage
-H "x-api-key: $EXA_API_KEY" |
jq '{searches_today, searches_this_month, monthly_limit, budget_remaining_pct: (1 - .searches_this_month / .monthly_limit) * 100}'

Step 4: Configure Alerts

groups:

• name: exa rules:
  • alert: ExaHighLatency expr: histogram_quantile(0.95, rate(exa_search_duration_ms_bucket[5m])) > 3000 # 3000: 3 seconds in ms annotations: { summary: "Exa search P95 latency exceeds 3 seconds" }
  • alert: ExaBudgetLow expr: exa_monthly_searches_remaining < 1000 # 1000: 1 second in ms annotations: { summary: "Exa monthly search budget nearly exhausted" }
  • alert: ExaLowResultQuality expr: rate(exa_result_usage{action="discarded"}[1h]) / rate(exa_result_usage[1h]) > 0.5 annotations: { summary: "Over 50% of Exa search results being discarded" }
  • alert: ExaApiErrors expr: rate(exa_searches_total{status="error"}[5m]) > 0.1 annotations: { summary: "Exa API errors detected" }

Step 5: Build a Search Efficiency Dashboard

Key panels: search volume by type (neural/keyword/auto), latency p50/p95, results per search distribution, result usage rate (used vs discarded), daily cost tracking, and cache hit rate. Low result counts with high latency indicate poorly formed queries.

Error Handling

Issue Cause Solution

429 Too Many Requests

Rate limit exceeded Implement exponential backoff and request queue

Zero results returned Query too specific or domain filter too narrow Broaden query, remove includeDomains filter

Latency spike to 5s+ Neural search on complex query Use type: "keyword" for simpler lookups

Monthly budget exhausted Uncapped search volume Add application-level search budget tracking

Examples

Basic usage: Apply exa observability to a standard project setup with default configuration options.

Advanced scenario: Customize exa observability for production environments with multiple constraints and team-specific requirements.

Output

• Configuration files or code changes applied to the project

• Validation report confirming correct implementation

• Summary of changes made and their rationale

Resources

• Official monitoring documentation

• Community best practices and patterns

• Related skills in this plugin pack