Challenge Insights
What do LLMs actually struggle with in Elasticsearch? Aggregate pass rates across 6 models.
Avg Pass Rate
84%
across all challenges
Hardest Challenge
kibana-5-audit-config
0% pass rate
Easiest Challenge
esql-2-stats-aggregation
100% pass rate
Models Tested
6
unique models
Pass Rate by Domain
unknown
84%
36 challenges
Pass Rate by Difficulty
beginner
0%
0 challenges
intermediate
0%
0 challenges
advanced
0%
0 challenges
expert
0%
0 challenges
10 Hardest Challenges
| Challenge | Domain | Difficulty | Pass Rate | Passed / Total |
|---|---|---|---|---|
| kibana-5-audit-config | unknown | unknown | 0% | 0/1 |
| kibana-6-vega-esql | unknown | unknown | 0% | 0/1 |
| kibana-8-connectors-preconfigured | unknown | unknown | 0% | 0/1 |
| esql-1-basic-filter | unknown | unknown | 33% | 2/6 |
| obs-esql-3-root-cause | unknown | unknown | 67% | 4/6 |
| sec-esql-2-post-compromise | unknown | unknown | 67% | 4/6 |
| sec-esql-3-attack-chain | unknown | unknown | 67% | 4/6 |
| cloud-1-setup | unknown | unknown | 80% | 4/5 |
| cloud-2-create-project | unknown | unknown | 80% | 4/5 |
| cloud-5-network-security | unknown | unknown | 80% | 4/5 |
10 Easiest Challenges
| Challenge | Domain | Difficulty | Pass Rate | Passed / Total |
|---|---|---|---|---|
| esql-2-stats-aggregation | unknown | unknown | 100% | 6/6 |
| esql-3-eval-computed | unknown | unknown | 100% | 6/6 |
| esql-4-log-error-analysis | unknown | unknown | 100% | 6/6 |
| esql-6-discover-and-query | unknown | unknown | 100% | 6/6 |
| obs-esql-1-error-investigation | unknown | unknown | 100% | 6/6 |
| obs-esql-2-error-breakdown | unknown | unknown | 100% | 6/6 |
| sec-esql-1-brute-force | unknown | unknown | 100% | 6/6 |
| cloud-3-manage-project | unknown | unknown | 100% | 5/5 |
| cloud-4-access-management | unknown | unknown | 100% | 5/5 |
| kibana-1-alerting-rules | unknown | unknown | 100% | 5/5 |