Elasticsearch
All Elasticsearch interaction is via REST API using curl . No SDK or client library required.
Authentication
Every request needs the cluster URL and an API key:
Set these for your session (or export in .env / shell profile)
ES_URL="https://your-cluster.es.cloud.elastic.co:443" ES_API_KEY="your-base64-api-key"
All requests follow this pattern:
curl -s "${ES_URL%/}/<endpoint>"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '<json-body>'
API key format: Base64-encoded id:api_key string. Pass as-is in the Authorization: ApiKey header.
If the user provides a URL and key, export them as ES_URL and ES_API_KEY before running commands.
Important — variable expansion in curl:
-
Always use $(printenv ES_API_KEY) instead of $ES_API_KEY in curl headers. The $ES_API_KEY variable may not expand correctly in the shell, resulting in empty Authorization headers and 401 errors.
-
Always use ${ES_URL%/} to strip any trailing slash from the URL, preventing double-slash path issues (e.g., //_cluster/health ).
Quick Health Check
Cluster health (green/yellow/red) — NOT available on serverless
curl -s "${ES_URL%/}/_cluster/health" -H "Authorization: ApiKey $(printenv ES_API_KEY)" | jq .
Node stats summary — NOT available on serverless
curl -s "${ES_URL%/}/_cat/nodes?v&h=name,heap.percent,ram.percent,cpu,load_1m,disk.used_percent"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
Index overview (works on both serverless and traditional)
curl -s "${ES_URL%/}/_cat/indices?v&s=store.size:desc&h=index,health,status,docs.count,store.size"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
Serverless Elasticsearch: If you get api_not_available_exception errors, the cluster is running in serverless mode. The following APIs are not available in serverless:
-
_cluster/health , _cluster/settings , _cluster/allocation/explain , _cluster/pending_tasks
-
_cat/nodes , _cat/shards
-
_nodes/hot_threads , _nodes/stats
-
ILM APIs (_ilm/* )
Use _cat/indices and _search APIs as the starting point instead — these work everywhere.
Search (Query DSL)
Simple match query
curl -s "${ES_URL%/}/my-index/_search"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{
"query": { "match": { "message": "error timeout" } },
"size": 10
}' | jq .
Bool query (must + filter + must_not)
curl -s "${ES_URL%/}/my-index/_search"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{
"query": {
"bool": {
"must": [ { "match": { "message": "error" } } ],
"filter": [ { "range": { "@timestamp": { "gte": "now-1h" } } } ],
"must_not": [ { "term": { "level": "debug" } } ]
}
},
"size": 20,
"sort": [ { "@timestamp": { "order": "desc" } } ]
}' | jq .
For full Query DSL reference (term, terms, range, wildcard, regexp, nested, exists, multi_match, etc.), see references/query-dsl.md.
Index & Document Operations
Create index with mappings
curl -s -X PUT "${ES_URL%/}/my-index"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{
"settings": { "number_of_shards": 1, "number_of_replicas": 1 },
"mappings": {
"properties": {
"message": { "type": "text" },
"@timestamp": { "type": "date" },
"level": { "type": "keyword" },
"count": { "type": "integer" }
}
}
}'
Index a document (auto-generate ID)
curl -s -X POST "${ES_URL%/}/my-index/_doc"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{ "message": "hello world", "@timestamp": "2026-01-31T12:00:00Z", "level": "info" }'
Index with specific ID
curl -s -X PUT "${ES_URL%/}/my-index/_doc/doc-123"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{ "message": "specific doc", "level": "warn" }'
Get document
curl -s "${ES_URL%/}/my-index/_doc/doc-123" -H "Authorization: ApiKey $(printenv ES_API_KEY)" | jq .
Update document (partial)
curl -s -X POST "${ES_URL%/}/my-index/_update/doc-123"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{ "doc": { "level": "error" } }'
Delete document
curl -s -X DELETE "${ES_URL%/}/my-index/_doc/doc-123"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
Bulk operations (newline-delimited JSON)
curl -s -X POST "${ES_URL%/}/_bulk"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/x-ndjson"
--data-binary @- << 'EOF'
{"index":{"_index":"my-index"}}
{"message":"bulk doc 1","level":"info","@timestamp":"2026-01-31T12:00:00Z"}
{"index":{"_index":"my-index"}}
{"message":"bulk doc 2","level":"warn","@timestamp":"2026-01-31T12:01:00Z"}
EOF
Aggregations
Terms aggregation (top values)
curl -s "${ES_URL%/}/my-index/_search?size=0"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{
"aggs": {
"levels": { "terms": { "field": "level", "size": 10 } }
}
}' | jq '.aggregations'
Date histogram + nested metric
curl -s "${ES_URL%/}/my-index/_search?size=0"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{
"query": { "range": { "@timestamp": { "gte": "now-24h" } } },
"aggs": {
"over_time": {
"date_histogram": { "field": "@timestamp", "fixed_interval": "1h" },
"aggs": {
"avg_count": { "avg": { "field": "count" } }
}
}
}
}' | jq '.aggregations'
For more aggregation types (cardinality, percentiles, composite, filters, significant_terms, etc.), see references/aggregations.md.
Mappings & Index Management
Get mapping
curl -s "${ES_URL%/}/my-index/_mapping" -H "Authorization: ApiKey $(printenv ES_API_KEY)" | jq .
Add field to existing mapping (mappings are additive — you can't change existing field types)
curl -s -X PUT "${ES_URL%/}/my-index/_mapping"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{ "properties": { "new_field": { "type": "keyword" } } }'
Reindex (change mappings, rename index, etc.)
curl -s -X POST "${ES_URL%/}/_reindex"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{
"source": { "index": "old-index" },
"dest": { "index": "new-index" }
}'
Delete index
curl -s -X DELETE "${ES_URL%/}/my-index" -H "Authorization: ApiKey $(printenv ES_API_KEY)"
Index aliases
curl -s -X POST "${ES_URL%/}/_aliases"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{
"actions": [
{ "add": { "index": "my-index-v2", "alias": "my-index" } },
{ "remove": { "index": "my-index-v1", "alias": "my-index" } }
]
}'
Index templates (for time-series / rollover patterns)
curl -s -X PUT "${ES_URL%/}/_index_template/my-template"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{
"index_patterns": ["logs-*"],
"template": {
"settings": { "number_of_shards": 1 },
"mappings": {
"properties": {
"message": { "type": "text" },
"@timestamp": { "type": "date" }
}
}
}
}'
Cluster & Troubleshooting
Note: Most APIs in this section are not available on serverless Elasticsearch. They only work on self-managed or traditional Elastic Cloud deployments.
Allocation explanation (why is a shard unassigned?) — NOT serverless
curl -s "${ES_URL%/}/_cluster/allocation/explain"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{ "index": "my-index", "shard": 0, "primary": true }' | jq .
Pending tasks
curl -s "${ES_URL%/}/_cluster/pending_tasks" -H "Authorization: ApiKey $(printenv ES_API_KEY)" | jq .
Hot threads (performance debugging)
curl -s "${ES_URL%/}/_nodes/hot_threads" -H "Authorization: ApiKey $(printenv ES_API_KEY)"
Shard allocation
curl -s "${ES_URL%/}/_cat/shards?v&s=store:desc&h=index,shard,prirep,state,docs,store,node"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
Task management (long-running operations)
curl -s "${ES_URL%/}/_tasks?actions=*search&detailed" -H "Authorization: ApiKey $(printenv ES_API_KEY)" | jq .
Cluster settings (persistent + transient)
curl -s "${ES_URL%/}/_cluster/settings?include_defaults=false"
-H "Authorization: ApiKey $(printenv ES_API_KEY)" | jq .
For Kibana API operations (dashboards, data views, saved objects, alerting rules), see references/kibana-api.md.
Data Streams & ILM
Note: ILM APIs (_ilm/* ) are not available on serverless. Data stream listing works on both.
List data streams
curl -s "${ES_URL%/}/_data_stream" -H "Authorization: ApiKey $(printenv ES_API_KEY)" | jq .
Create ILM policy
curl -s -X PUT "${ES_URL%/}/_ilm/policy/my-policy"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{
"policy": {
"phases": {
"hot": { "actions": { "rollover": { "max_age": "7d", "max_size": "50gb" } } },
"warm": { "min_age": "30d", "actions": { "shrink": { "number_of_shards": 1 } } },
"delete": { "min_age": "90d", "actions": { "delete": {} } }
}
}
}'
Check ILM status for an index
curl -s "${ES_URL%/}/my-index/_ilm/explain" -H "Authorization: ApiKey $(printenv ES_API_KEY)" | jq .
ES|QL (Elasticsearch Query Language)
For Elasticsearch 8.11+, ES|QL offers a pipe-based query syntax:
curl -s -X POST "${ES_URL%/}/_query"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{
"query": "FROM logs-* | WHERE level == "error" | STATS count = COUNT(*) BY service.name | SORT count DESC | LIMIT 10"
}' | jq .
For querying OpenTelemetry data (OTEL logs, traces, metrics, correlation patterns), see references/otel-data.md.
Ingest Pipelines
Create pipeline
curl -s -X PUT "${ES_URL%/}/_ingest/pipeline/my-pipeline"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{
"processors": [
{ "grok": { "field": "message", "patterns": ["%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} %{GREEDYDATA:msg}"] } },
{ "date": { "field": "timestamp", "formats": ["ISO8601"] } },
{ "remove": { "field": "timestamp" } }
]
}'
Test pipeline
curl -s -X POST "${ES_URL%/}/_ingest/pipeline/my-pipeline/_simulate"
-H "Authorization: ApiKey $(printenv ES_API_KEY)"
-H "Content-Type: application/json"
-d '{
"docs": [
{ "_source": { "message": "2026-01-31T12:00:00Z ERROR something broke" } }
]
}' | jq .
Tips
-
Always use jq to format JSON output — Elasticsearch responses are verbose.
-
?size=0 on search requests when you only want aggregations (skip hits).
-
_cat APIs (_cat/indices , _cat/shards , _cat/nodes ) give human-readable tabular output — add ?v for headers, ?format=json for JSON.
-
Scroll/PIT for large exports — don't use from /size beyond 10,000 hits. Use search_after + PIT instead.
-
Field types matter — keyword for exact match/aggs, text for full-text search. Check mappings before querying.
-
Date math in index names — logs-{now/d} resolves to today's date. Useful for time-based indices.