Trajectories — Poolside
Laguna S 2.1 Trajectories on Terminal-Bench 2.1
Evaluation configuration
Results calculated as pass@1 over k=4 attempts. If an attempt times out (it reaches a global task timeout) a trajectory may not be created.
Search tasks
100/178 shown
| Task | Variant | Result | Steps | Cost | Open |
|---|---|---|---|---|---|
| adaptive-rejection-sampler 4 trialsThinking | Thinking | 3/4 | 61 | $0.0915 | |
| adaptive-rejection-sampler 4 trialsNo thinking | No Thinking | 2/4 | 135 | $0.1007 | |
| bn-fit-modify 4 trialsThinking | Thinking | 4/4 | 40 | $0.0215 | |
| bn-fit-modify 4 trialsNo thinking | No Thinking | 2/4 | 37 | $0.0103 | |
| break-filter-js-from-html 4 trialsThinking | Thinking | 4/4 | 42 | $0.11 | |
| break-filter-js-from-html 4 trialsNo thinking | No Thinking | 4/4 | 144 | $0.1771 | |
| build-cython-ext 4 trialsThinking | Thinking | 4/4 | 57 | $0.0586 | |
| build-cython-ext 4 trialsNo thinking | No Thinking | 4/4 | 79 | $0.0333 | |
| build-pmars 4 trialsThinking | Thinking | 4/4 | 59 | $0.0359 | |
| build-pmars 4 trialsNo thinking | No Thinking | 4/4 | 45 | $0.0114 | |
| build-pov-ray 4 trialsThinking | Thinking | 3/4 | 51 | $0.0329 | |
| build-pov-ray 4 trialsNo thinking | No Thinking | 2/4 | 171 | $0.1698 | |
| caffe-cifar-10 4 trialsThinking | Thinking | 4/4 | 158 | $0.2132 | |
| caffe-cifar-10 4 trialsNo thinking | No Thinking | 3/4 | 151 | $0.0765 | |
| cancel-async-tasks 4 trialsThinking | Thinking | 1/4 | 23 | $0.0141 | |
| cancel-async-tasks 4 trialsNo thinking | No Thinking | 2/4 | 41 | $0.0099 | |
| chess-best-move 4 trialsThinking | Thinking | 2/4 | 99 | $0.2793 | |
| chess-best-move 4 trialsNo thinking | No Thinking | 0/4 | 117 | $0.1166 | |
| circuit-fibsqrt 4 trialsThinking | Thinking | 4/4 | 133 | $0.4494 | |
| circuit-fibsqrt 4 trialsNo thinking | No Thinking | 3/4 | 335 | $0.7771 | |
| cobol-modernization 4 trialsThinking | Thinking | 4/4 | 44 | $0.0806 | |
| cobol-modernization 4 trialsNo thinking | No Thinking | 4/4 | 117 | $0.08 | |
| code-from-image 4 trialsThinking | Thinking | 4/4 | 38 | $0.0406 | |
| code-from-image 3 trials | 1 timed outNo thinking | No Thinking | 2/4 | 210 | $0.6859 |
| compile-compcert 4 trialsThinking | Thinking | 4/4 | 128 | $0.1626 | |
| compile-compcert 4 trialsNo thinking | No Thinking | 4/4 | 161 | $0.0593 | |
| configure-git-webserver 4 trialsThinking | Thinking | 2/4 | 40 | $0.0241 | |
| configure-git-webserver 4 trialsNo thinking | No Thinking | 3/4 | 35 | $0.0048 | |
| constraints-scheduling 4 trialsThinking | Thinking | 4/4 | 19 | $0.0116 | |
| constraints-scheduling 4 trialsNo thinking | No Thinking | 4/4 | 20 | $0.0048 | |
| count-dataset-tokens 4 trialsThinking | Thinking | 3/4 | 39 | $0.0282 | |
| count-dataset-tokens 4 trialsNo thinking | No Thinking | 2/4 | 35 | $0.0089 | |
| crack-7z-hash 4 trialsThinking | Thinking | 2/4 | 75 | $0.0588 | |
| crack-7z-hash 4 trialsNo thinking | No Thinking | 4/4 | 80 | $0.0316 | |
| custom-memory-heap-crash 4 trialsThinking | Thinking | 4/4 | 52 | $0.0509 | |
| custom-memory-heap-crash 4 trialsNo thinking | No Thinking | 4/4 | 65 | $0.0263 | |
| db-wal-recovery 4 trialsThinking | Thinking | 0/4 | 47 | $0.1536 | |
| db-wal-recovery 3 trials | 1 timed outNo thinking | No Thinking | 2/4 | 51 | $0.0648 |
| distribution-search 4 trialsThinking | Thinking | 4/4 | 13 | $0.0108 | |
| distribution-search 4 trialsNo thinking | No Thinking | 4/4 | 33 | $0.0106 | |
| dna-assembly 3 trials | 1 timed outThinking | Thinking | 0/4 | 54 | $0.2217 |
| dna-assembly 4 trialsNo thinking | No Thinking | 0/4 | 144 | $0.3614 | |
| dna-insert 4 trialsThinking | Thinking | 0/4 | 59 | $0.1219 | |
| dna-insert 4 trialsNo thinking | No Thinking | 0/4 | 59 | $0.0482 | |
| extract-elf 4 trialsThinking | Thinking | 1/4 | 31 | $0.0583 | |
| extract-elf 4 trialsNo thinking | No Thinking | 2/4 | 40 | $0.0184 | |
| extract-moves-from-video 1 trials | 3 timed outThinking | Thinking | 0/4 | 183 | $0.6416 |
| extract-moves-from-video 3 trials | 1 timed outNo thinking | No Thinking | 0/4 | 440 | $1.0692 |
| feal-differential-cryptanalysis 4 trialsThinking | Thinking | 4/4 | 73 | $0.1188 | |
| feal-differential-cryptanalysis 3 trials | 1 timed outNo thinking | No Thinking | 3/4 | 162 | $0.337 |
| feal-linear-cryptanalysis 3 trials | 1 timed outThinking | Thinking | 3/4 | 41 | $0.0958 |
| feal-linear-cryptanalysis 2 trials | 2 timed outNo thinking | No Thinking | 1/4 | 275 | $0.4651 |
| filter-js-from-html 4 trialsThinking | Thinking | 0/4 | 40 | $0.0694 | |
| filter-js-from-html 4 trialsNo thinking | No Thinking | 1/4 | 87 | $0.0541 | |
| financial-document-processor 4 trialsThinking | Thinking | 4/4 | 51 | $0.0448 | |
| financial-document-processor 4 trialsNo thinking | No Thinking | 4/4 | 109 | $0.0826 | |
| fix-code-vulnerability 4 trialsThinking | Thinking | 4/4 | 34 | $0.0114 | |
| fix-code-vulnerability 4 trialsNo thinking | No Thinking | 4/4 | 34 | $0.0087 | |
| fix-git 4 trialsThinking | Thinking | 4/4 | 17 | $0.0048 | |
| fix-git 4 trialsNo thinking | No Thinking | 4/4 | 14 | $0.0017 | |
| fix-ocaml-gc 4 trialsThinking | Thinking | 4/4 | 62 | $0.0713 | |
| fix-ocaml-gc 4 trialsNo thinking | No Thinking | 4/4 | 93 | $0.0825 | |
| gcode-to-text 3 trials | 1 timed outThinking | Thinking | 0/4 | 136 | $0.5855 |
| gcode-to-text 4 trialsNo thinking | No Thinking | 0/4 | 155 | $0.5202 | |
| git-leak-recovery 4 trialsThinking | Thinking | 4/4 | 20 | $0.0073 | |
| git-leak-recovery 4 trialsNo thinking | No Thinking | 4/4 | 23 | $0.0035 | |
| git-multibranch 4 trialsThinking | Thinking | 4/4 | 30 | $0.0196 | |
| git-multibranch 4 trialsNo thinking | No Thinking | 3/4 | 61 | $0.0124 | |
| gpt2-codegolf 1 trials | 3 timed outThinking | Thinking | 0/4 | 125 | $0.6116 |
| gpt2-codegolf 4 trialsNo thinking | No Thinking | 0/4 | 501 | $1.7932 | |
| headless-terminal 4 trialsThinking | Thinking | 4/4 | 52 | $0.0603 | |
| headless-terminal 4 trialsNo thinking | No Thinking | 4/4 | 49 | $0.0158 | |
| hf-model-inference 4 trialsThinking | Thinking | 3/4 | 24 | $0.0041 | |
| hf-model-inference 4 trialsNo thinking | No Thinking | 3/4 | 24 | $0.0028 | |
| install-windows-3.11 4 trialsThinking | Thinking | 2/4 | 130 | $0.2185 | |
| install-windows-3.11 4 trialsNo thinking | No Thinking | 1/4 | 204 | $0.4145 | |
| kv-store-grpc 4 trialsThinking | Thinking | 4/4 | 26 | $0.0035 | |
| kv-store-grpc 4 trialsNo thinking | No Thinking | 2/4 | 25 | $0.0029 | |
| large-scale-text-editing 4 trialsThinking | Thinking | 4/4 | 31 | $0.0342 | |
| large-scale-text-editing 4 trialsNo thinking | No Thinking | 4/4 | 94 | $0.0295 | |
| largest-eigenval 4 trialsThinking | Thinking | 2/4 | 132 | $0.4405 | |
| largest-eigenval 4 trialsNo thinking | No Thinking | 1/4 | 266 | $0.5568 | |
| llm-inference-batching-scheduler 4 trialsThinking | Thinking | 4/4 | 77 | $0.2132 | |
| llm-inference-batching-scheduler 4 trialsNo thinking | No Thinking | 3/4 | 118 | $0.1422 | |
| log-summary-date-ranges 4 trialsThinking | Thinking | 4/4 | 14 | $0.0025 | |
| log-summary-date-ranges 4 trialsNo thinking | No Thinking | 4/4 | 9 | $0.0017 | |
| mailman 4 trialsThinking | Thinking | 4/4 | 127 | $0.2728 | |
| mailman 4 trialsNo thinking | No Thinking | 1/4 | 132 | $0.0878 | |
| make-doom-for-mips 1 trials | 3 timed outThinking | Thinking | 0/4 | 273 | $1.3287 |
| make-doom-for-mips 4 trialsNo thinking | No Thinking | 0/4 | 476 | $1.3159 | |
| make-mips-interpreter 3 trials | 1 timed outThinking | Thinking | 0/4 | 250 | $1.2843 |
| make-mips-interpreter 4 trialsNo thinking | No Thinking | 0/4 | 460 | $1.6009 | |
| mcmc-sampling-stan 4 trialsThinking | Thinking | 4/4 | 92 | $0.0561 | |
| mcmc-sampling-stan 4 trialsNo thinking | No Thinking | 3/4 | 49 | $0.0106 | |
| merge-diff-arc-agi-task 4 trialsThinking | Thinking | 2/4 | 31 | $0.0198 | |
| merge-diff-arc-agi-task 4 trialsNo thinking | No Thinking | 3/4 | 41 | $0.008 | |
| model-extraction-relu-logits 4 trialsThinking | Thinking | 2/4 | 55 | $0.103 | |
| model-extraction-relu-logits 4 trialsNo thinking | No Thinking | 1/4 | 98 | $0.08 | |
| modernize-scientific-stack 4 trialsThinking | Thinking | 4/4 | 18 | $0.0044 | |
| modernize-scientific-stack 4 trialsNo thinking | No Thinking | 4/4 | 17 | $0.0026 |