How well does your AI know Elasticsearch?

65 challenges + 12 skill-aligned scenarios across 7 domains. Benchmark any LLM on real Elasticsearch tasks. Compare models on the public leaderboard.

$OPENROUTER_API_KEY=sk-or-... npx elastic-quest benchmark --pick

Get your free API key at openrouter.ai/keys — one key, 300+ models.

How it works

Choose from 300+ models via OpenRouter. GPT-4o, Claude, Gemini, Llama, DeepSeek, Mistral, and more.

Each model gets Elasticsearch challenges. The query response is executed and validated against expected results.

Scores are broken down by domain and difficulty. Results are submitted to the public leaderboard automatically.

Challenge of the Weekexpert

Filters agg with success (2xx) vs failure (5xx) per service

Domain: observability

From beginner match queries to expert-level hybrid vector search with nested aggregations

match, bool, phrase, fuzzy, dis_max, boosting, nested queries, function_score, field boosting

Field types, sort, pagination, date ranges, conditional counts, terms queries

terms, avg, sum, stats, cardinality, date_histogram, percentiles, percentile_ranks, filters agg, 3-level nested aggs

Log filtering, error spikes, latency percentiles, trace correlation, SLO uptime, top errors

kNN search, filtered kNN, hybrid text+vector, semantic search with aggregations

IP range filtering, brute force detection, DNS threat hunting, alert triage, account correlation

FROM, WHERE, STATS, EVAL with CASE, time-bucketed analysis, multi-turn discovery queries

Aligned to Elastic Agent Skills. Run with and without skills to measure the uplift.

Basic filtering, stats aggregation, computed fields, time-bucketed analysis, multi-turn discovery

Skill: elasticsearch-esql

Error timeline, service breakdown, multi-turn root cause analysis across 2500 logs

Skill: logs-search

Brute force detection, post-compromise analysis, full attack chain investigation in 1500 events

Skill: alert-triage

$npx elastic-quest benchmark --scenarios --start-local --compare-skills -m openrouter:openai/gpt-4o

$OPENROUTER_API_KEY=sk-or-... npx elastic-quest benchmark --pick

Set your OpenRouter API key and pick models interactively.
Or run directly: npx elastic-quest benchmark -m openrouter:openai/gpt-4o