/bench
Builds Java and runs the full benchmark suite, then analyzes results.
Execution Steps
-
Run ./scripts/bench.sh
-
Extract metrics from tests/benchmark/prediction/opendataloader/evaluation.json using: jq '{summary, metrics, table_detection, speed}' tests/benchmark/prediction/opendataloader/evaluation.json
-
Output summary:
-
NID (reading order)
-
TEDS (table structure)
-
MHS (heading structure)
-
Table Detection F1/Precision/Recall
-
Speed (s/doc, total time)
-
Compare with thresholds in tests/benchmark/thresholds.json
-
Warn if regression detected
Options
- /bench --doc-id 01030000000189
- Run for a specific document only
Notes
-
Benchmark won't run if Java build fails
-
First run installs Python dependencies via uv sync