LLM-Server: Benchmark-Timeout auf 2700 s angehoben (Befund aus AB3-Phase 1)
Swift 1.0 xhigh Thinking lief auf einer AIME-Aufgabe ueber 30 Minuten und sprengte lm-evals Request-Timeout: die 32768-Token-Ausgabegrenze des Profils benoetigt bei ~18 t/s etwa 30,2 Minuten. 2700 s deckt Cap-Grenzen plus Prefill ab; max_retries auf 1 reduziert (sequenzielle Requests, ein Timeout-Wiederholungsversuch wuerde nur die gleiche Generierung wiederholen). Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
This commit is contained in:
@@ -30,7 +30,11 @@ benchmark_task_args:
|
||||
model: dummy-model-name
|
||||
base_url: "{{ benchmark_engine_url }}/chat/completions"
|
||||
num_concurrent: 1
|
||||
max_retries: 2
|
||||
# xhigh thinking can run to the 32768-token output cap, which takes ~30.2
|
||||
# minutes at ~18 t/s; 2700 s covers cap-bound generations plus prompt
|
||||
# evaluation. Retries are pointless for sequential requests (a timed-out
|
||||
# generation would just repeat); observed 02.11.2026 when Swift 1.0
|
||||
# exceeded the previous 1800 s timeout on AIME.
|
||||
max_retries: 1
|
||||
tokenized_requests: false
|
||||
# xhigh thinking on this hardware needs well over the 300 s default.
|
||||
timeout: 1800
|
||||
timeout: 2700
|
||||
|
||||
Reference in New Issue
Block a user