# Laguna S 2.1 Trajectories on Terminal-Bench 2.1

**Evaluation configuration**

Results calculated as pass@1 over k=4 attempts. If an attempt times out (it reaches a global task timeout) a trajectory may not be created.

## Search tasks

100/178 shown

| Task | Variant | Result | Steps | Cost | Open |
| --- | --- | --- | --- | --- | --- |
| **adaptive-rejection-sampler** 4 trialsThinking | Thinking | 3/4 | 61 | $0.0915 |  |
| **adaptive-rejection-sampler** 4 trialsNo thinking | No Thinking | 2/4 | 135 | $0.1007 |  |
| **bn-fit-modify** 4 trialsThinking | Thinking | 4/4 | 40 | $0.0215 |  |
| **bn-fit-modify** 4 trialsNo thinking | No Thinking | 2/4 | 37 | $0.0103 |  |
| **break-filter-js-from-html** 4 trialsThinking | Thinking | 4/4 | 42 | $0.11 |  |
| **break-filter-js-from-html** 4 trialsNo thinking | No Thinking | 4/4 | 144 | $0.1771 |  |
| **build-cython-ext** 4 trialsThinking | Thinking | 4/4 | 57 | $0.0586 |  |
| **build-cython-ext** 4 trialsNo thinking | No Thinking | 4/4 | 79 | $0.0333 |  |
| **build-pmars** 4 trialsThinking | Thinking | 4/4 | 59 | $0.0359 |  |
| **build-pmars** 4 trialsNo thinking | No Thinking | 4/4 | 45 | $0.0114 |  |
| **build-pov-ray** 4 trialsThinking | Thinking | 3/4 | 51 | $0.0329 |  |
| **build-pov-ray** 4 trialsNo thinking | No Thinking | 2/4 | 171 | $0.1698 |  |
| **caffe-cifar-10** 4 trialsThinking | Thinking | 4/4 | 158 | $0.2132 |  |
| **caffe-cifar-10** 4 trialsNo thinking | No Thinking | 3/4 | 151 | $0.0765 |  |
| **cancel-async-tasks** 4 trialsThinking | Thinking | 1/4 | 23 | $0.0141 |  |
| **cancel-async-tasks** 4 trialsNo thinking | No Thinking | 2/4 | 41 | $0.0099 |  |
| **chess-best-move** 4 trialsThinking | Thinking | 2/4 | 99 | $0.2793 |  |
| **chess-best-move** 4 trialsNo thinking | No Thinking | 0/4 | 117 | $0.1166 |  |
| **circuit-fibsqrt** 4 trialsThinking | Thinking | 4/4 | 133 | $0.4494 |  |
| **circuit-fibsqrt** 4 trialsNo thinking | No Thinking | 3/4 | 335 | $0.7771 |  |
| **cobol-modernization** 4 trialsThinking | Thinking | 4/4 | 44 | $0.0806 |  |
| **cobol-modernization** 4 trialsNo thinking | No Thinking | 4/4 | 117 | $0.08 |  |
| **code-from-image** 4 trialsThinking | Thinking | 4/4 | 38 | $0.0406 |  |
| **code-from-image** 3 trials | 1 timed outNo thinking | No Thinking | 2/4 | 210 | $0.6859 |  |
| **compile-compcert** 4 trialsThinking | Thinking | 4/4 | 128 | $0.1626 |  |
| **compile-compcert** 4 trialsNo thinking | No Thinking | 4/4 | 161 | $0.0593 |  |
| **configure-git-webserver** 4 trialsThinking | Thinking | 2/4 | 40 | $0.0241 |  |
| **configure-git-webserver** 4 trialsNo thinking | No Thinking | 3/4 | 35 | $0.0048 |  |
| **constraints-scheduling** 4 trialsThinking | Thinking | 4/4 | 19 | $0.0116 |  |
| **constraints-scheduling** 4 trialsNo thinking | No Thinking | 4/4 | 20 | $0.0048 |  |
| **count-dataset-tokens** 4 trialsThinking | Thinking | 3/4 | 39 | $0.0282 |  |
| **count-dataset-tokens** 4 trialsNo thinking | No Thinking | 2/4 | 35 | $0.0089 |  |
| **crack-7z-hash** 4 trialsThinking | Thinking | 2/4 | 75 | $0.0588 |  |
| **crack-7z-hash** 4 trialsNo thinking | No Thinking | 4/4 | 80 | $0.0316 |  |
| **custom-memory-heap-crash** 4 trialsThinking | Thinking | 4/4 | 52 | $0.0509 |  |
| **custom-memory-heap-crash** 4 trialsNo thinking | No Thinking | 4/4 | 65 | $0.0263 |  |
| **db-wal-recovery** 4 trialsThinking | Thinking | 0/4 | 47 | $0.1536 |  |
| **db-wal-recovery** 3 trials | 1 timed outNo thinking | No Thinking | 2/4 | 51 | $0.0648 |  |
| **distribution-search** 4 trialsThinking | Thinking | 4/4 | 13 | $0.0108 |  |
| **distribution-search** 4 trialsNo thinking | No Thinking | 4/4 | 33 | $0.0106 |  |
| **dna-assembly** 3 trials | 1 timed outThinking | Thinking | 0/4 | 54 | $0.2217 |  |
| **dna-assembly** 4 trialsNo thinking | No Thinking | 0/4 | 144 | $0.3614 |  |
| **dna-insert** 4 trialsThinking | Thinking | 0/4 | 59 | $0.1219 |  |
| **dna-insert** 4 trialsNo thinking | No Thinking | 0/4 | 59 | $0.0482 |  |
| **extract-elf** 4 trialsThinking | Thinking | 1/4 | 31 | $0.0583 |  |
| **extract-elf** 4 trialsNo thinking | No Thinking | 2/4 | 40 | $0.0184 |  |
| **extract-moves-from-video** 1 trials | 3 timed outThinking | Thinking | 0/4 | 183 | $0.6416 |  |
| **extract-moves-from-video** 3 trials | 1 timed outNo thinking | No Thinking | 0/4 | 440 | $1.0692 |  |
| **feal-differential-cryptanalysis** 4 trialsThinking | Thinking | 4/4 | 73 | $0.1188 |  |
| **feal-differential-cryptanalysis** 3 trials | 1 timed outNo thinking | No Thinking | 3/4 | 162 | $0.337 |  |
| **feal-linear-cryptanalysis** 3 trials | 1 timed outThinking | Thinking | 3/4 | 41 | $0.0958 |  |
| **feal-linear-cryptanalysis** 2 trials | 2 timed outNo thinking | No Thinking | 1/4 | 275 | $0.4651 |  |
| **filter-js-from-html** 4 trialsThinking | Thinking | 0/4 | 40 | $0.0694 |  |
| **filter-js-from-html** 4 trialsNo thinking | No Thinking | 1/4 | 87 | $0.0541 |  |
| **financial-document-processor** 4 trialsThinking | Thinking | 4/4 | 51 | $0.0448 |  |
| **financial-document-processor** 4 trialsNo thinking | No Thinking | 4/4 | 109 | $0.0826 |  |
| **fix-code-vulnerability** 4 trialsThinking | Thinking | 4/4 | 34 | $0.0114 |  |
| **fix-code-vulnerability** 4 trialsNo thinking | No Thinking | 4/4 | 34 | $0.0087 |  |
| **fix-git** 4 trialsThinking | Thinking | 4/4 | 17 | $0.0048 |  |
| **fix-git** 4 trialsNo thinking | No Thinking | 4/4 | 14 | $0.0017 |  |
| **fix-ocaml-gc** 4 trialsThinking | Thinking | 4/4 | 62 | $0.0713 |  |
| **fix-ocaml-gc** 4 trialsNo thinking | No Thinking | 4/4 | 93 | $0.0825 |  |
| **gcode-to-text** 3 trials | 1 timed outThinking | Thinking | 0/4 | 136 | $0.5855 |  |
| **gcode-to-text** 4 trialsNo thinking | No Thinking | 0/4 | 155 | $0.5202 |  |
| **git-leak-recovery** 4 trialsThinking | Thinking | 4/4 | 20 | $0.0073 |  |
| **git-leak-recovery** 4 trialsNo thinking | No Thinking | 4/4 | 23 | $0.0035 |  |
| **git-multibranch** 4 trialsThinking | Thinking | 4/4 | 30 | $0.0196 |  |
| **git-multibranch** 4 trialsNo thinking | No Thinking | 3/4 | 61 | $0.0124 |  |
| **gpt2-codegolf** 1 trials | 3 timed outThinking | Thinking | 0/4 | 125 | $0.6116 |  |
| **gpt2-codegolf** 4 trialsNo thinking | No Thinking | 0/4 | 501 | $1.7932 |  |
| **headless-terminal** 4 trialsThinking | Thinking | 4/4 | 52 | $0.0603 |  |
| **headless-terminal** 4 trialsNo thinking | No Thinking | 4/4 | 49 | $0.0158 |  |
| **hf-model-inference** 4 trialsThinking | Thinking | 3/4 | 24 | $0.0041 |  |
| **hf-model-inference** 4 trialsNo thinking | No Thinking | 3/4 | 24 | $0.0028 |  |
| **install-windows-3.11** 4 trialsThinking | Thinking | 2/4 | 130 | $0.2185 |  |
| **install-windows-3.11** 4 trialsNo thinking | No Thinking | 1/4 | 204 | $0.4145 |  |
| **kv-store-grpc** 4 trialsThinking | Thinking | 4/4 | 26 | $0.0035 |  |
| **kv-store-grpc** 4 trialsNo thinking | No Thinking | 2/4 | 25 | $0.0029 |  |
| **large-scale-text-editing** 4 trialsThinking | Thinking | 4/4 | 31 | $0.0342 |  |
| **large-scale-text-editing** 4 trialsNo thinking | No Thinking | 4/4 | 94 | $0.0295 |  |
| **largest-eigenval** 4 trialsThinking | Thinking | 2/4 | 132 | $0.4405 |  |
| **largest-eigenval** 4 trialsNo thinking | No Thinking | 1/4 | 266 | $0.5568 |  |
| **llm-inference-batching-scheduler** 4 trialsThinking | Thinking | 4/4 | 77 | $0.2132 |  |
| **llm-inference-batching-scheduler** 4 trialsNo thinking | No Thinking | 3/4 | 118 | $0.1422 |  |
| **log-summary-date-ranges** 4 trialsThinking | Thinking | 4/4 | 14 | $0.0025 |  |
| **log-summary-date-ranges** 4 trialsNo thinking | No Thinking | 4/4 | 9 | $0.0017 |  |
| **mailman** 4 trialsThinking | Thinking | 4/4 | 127 | $0.2728 |  |
| **mailman** 4 trialsNo thinking | No Thinking | 1/4 | 132 | $0.0878 |  |
| **make-doom-for-mips** 1 trials | 3 timed outThinking | Thinking | 0/4 | 273 | $1.3287 |  |
| **make-doom-for-mips** 4 trialsNo thinking | No Thinking | 0/4 | 476 | $1.3159 |  |
| **make-mips-interpreter** 3 trials | 1 timed outThinking | Thinking | 0/4 | 250 | $1.2843 |  |
| **make-mips-interpreter** 4 trialsNo thinking | No Thinking | 0/4 | 460 | $1.6009 |  |
| **mcmc-sampling-stan** 4 trialsThinking | Thinking | 4/4 | 92 | $0.0561 |  |
| **mcmc-sampling-stan** 4 trialsNo thinking | No Thinking | 3/4 | 49 | $0.0106 |  |
| **merge-diff-arc-agi-task** 4 trialsThinking | Thinking | 2/4 | 31 | $0.0198 |  |
| **merge-diff-arc-agi-task** 4 trialsNo thinking | No Thinking | 3/4 | 41 | $0.008 |  |
| **model-extraction-relu-logits** 4 trialsThinking | Thinking | 2/4 | 55 | $0.103 |  |
| **model-extraction-relu-logits** 4 trialsNo thinking | No Thinking | 1/4 | 98 | $0.08 |  |
| **modernize-scientific-stack** 4 trialsThinking | Thinking | 4/4 | 18 | $0.0044 |  |
| **modernize-scientific-stack** 4 trialsNo thinking | No Thinking | 4/4 | 17 | $0.0026 |  |