Skip to content

Commit 85392f5

Browse files
chaliychaliy
committed
feat(security): charge buffer growth before allocation
1 parent 225a5d7 commit 85392f5

10 files changed

Lines changed: 5973 additions & 174 deletions

File tree

‎crates/bashkit-bench/results/bench-mykhailosmin-macos-aarch64-1785975172.json‎

Lines changed: 4847 additions & 0 deletions
Large diffs are not rendered by default.

‎crates/bashkit-bench/results/bench-mykhailosmin-macos-aarch64-1785975172.md‎

Lines changed: 305 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,305 @@
1+
# Bashkit Benchmark Report
2+
3+
## System Information
4+
5+
- **Moniker**: `mykhailosmin-macos-aarch64`
6+
- **Hostname**: Mykhailos-Mini.lan
7+
- **OS**: macos
8+
- **Architecture**: aarch64
9+
- **CPUs**: 12
10+
- **Timestamp**: 1785975172
11+
- **Iterations**: 10
12+
- **Warmup**: 2
13+
- **Prewarm cases**: 3
14+
15+
## Summary
16+
17+
Benchmarked 96 cases across 2 runners.
18+
19+
| Runner | Total Time (ms) | Avg/Case (ms) | Errors | Error Rate | Output Match |
20+
|--------|-----------------|---------------|--------|------------|-------------|
21+
| bashkit | 50.58 | 0.527 | 0 | 0.0% | 95.8% |
22+
| bash | 3351.84 | 34.915 | 20 | 2.1% | 100.0% |
23+
24+
## Performance Comparison
25+
26+
**Bashkit is 66.3x faster** than bash on average.
27+
28+
## Results by Category
29+
30+
### Arithmetic
31+
32+
| Benchmark | Runner | Mean (ms) | StdDev | Errors | Match |
33+
|-----------|--------|-----------|--------|--------|-------|
34+
| arith_basic | bashkit | 0.047 | ±0.043 | - | ✓ |
35+
| arith_basic | bash | 11.833 | ±7.702 | - | ✓ |
36+
| arith_complex | bashkit | 0.051 | ±0.030 | - | ✓ |
37+
| arith_complex | bash | 6.589 | ±3.560 | - | ✓ |
38+
| arith_variables | bashkit | 0.020 | ±0.001 | - | ✓ |
39+
| arith_variables | bash | 8.838 | ±5.153 | - | ✓ |
40+
| arith_increment | bashkit | 0.142 | ±0.355 | - | ✓ |
41+
| arith_increment | bash | 9.888 | ±3.258 | - | ✓ |
42+
| arith_modulo | bashkit | 0.046 | ±0.007 | - | ✓ |
43+
| arith_modulo | bash | 6.592 | ±3.534 | - | ✓ |
44+
| arith_loop_sum | bashkit | 2.462 | ±7.261 | - | ✓ |
45+
| arith_loop_sum | bash | 8.514 | ±4.484 | - | ✓ |
46+
47+
### Arrays
48+
49+
| Benchmark | Runner | Mean (ms) | StdDev | Errors | Match |
50+
|-----------|--------|-----------|--------|--------|-------|
51+
| arr_create | bashkit | 0.019 | ±0.001 | - | ✓ |
52+
| arr_create | bash | 7.440 | ±3.381 | - | ✓ |
53+
| arr_all | bashkit | 0.019 | ±0.001 | - | ✓ |
54+
| arr_all | bash | 5.457 | ±2.946 | - | ✓ |
55+
| arr_length | bashkit | 0.018 | ±0.001 | - | ✓ |
56+
| arr_length | bash | 9.074 | ±4.784 | - | ✓ |
57+
| arr_iterate | bashkit | 0.072 | ±0.037 | - | ✓ |
58+
| arr_iterate | bash | 7.745 | ±5.876 | - | ✓ |
59+
| arr_slice | bashkit | 0.030 | ±0.018 | - | ✓ |
60+
| arr_slice | bash | 6.289 | ±1.616 | - | ✓ |
61+
| arr_assign_index | bashkit | 0.020 | ±0.001 | - | ✓ |
62+
| arr_assign_index | bash | 6.118 | ±3.239 | - | ✓ |
63+
64+
### Complex
65+
66+
| Benchmark | Runner | Mean (ms) | StdDev | Errors | Match |
67+
|-----------|--------|-----------|--------|--------|-------|
68+
| complex_fibonacci | bashkit | 3.356 | ±1.934 | - | ✓ |
69+
| complex_fibonacci | bash | 192.573 | ±39.435 | - | ✓ |
70+
| complex_fibonacci_iter | bashkit | 0.055 | ±0.014 | - | ✓ |
71+
| complex_fibonacci_iter | bash | 4.267 | ±2.244 | - | ✓ |
72+
| complex_nested_subst | bashkit | 0.078 | ±0.052 | - | ✓ |
73+
| complex_nested_subst | bash | 10.043 | ±3.156 | - | ✓ |
74+
| complex_loop_compute | bashkit | 0.050 | ±0.001 | - | ✓ |
75+
| complex_loop_compute | bash | 8.075 | ±5.651 | - | ✓ |
76+
| complex_string_build | bashkit | 0.025 | ±0.001 | - | ✓ |
77+
| complex_string_build | bash | 4.672 | ±3.285 | - | ✓ |
78+
| complex_json_transform | bashkit | 0.248 | ±0.012 | - | ✓ |
79+
| complex_json_transform | bash | 12.190 | ±3.440 | - | ✓ |
80+
| complex_pipeline_text | bashkit | 0.068 | ±0.003 | - | ✓ |
81+
| complex_pipeline_text | bash | 11.880 | ±4.765 | - | ✓ |
82+
83+
### Control
84+
85+
| Benchmark | Runner | Mean (ms) | StdDev | Errors | Match |
86+
|-----------|--------|-----------|--------|--------|-------|
87+
| ctrl_if_simple | bashkit | 0.018 | ±0.001 | - | ✓ |
88+
| ctrl_if_simple | bash | 7.616 | ±3.636 | - | ✓ |
89+
| ctrl_if_else | bashkit | 0.020 | ±0.001 | - | ✓ |
90+
| ctrl_if_else | bash | 13.184 | ±8.063 | - | ✓ |
91+
| ctrl_for_list | bashkit | 0.056 | ±0.029 | - | ✓ |
92+
| ctrl_for_list | bash | 6.790 | ±2.747 | - | ✓ |
93+
| ctrl_for_range | bashkit | 0.074 | ±0.004 | - | ✓ |
94+
| ctrl_for_range | bash | 10.580 | ±8.626 | - | ✓ |
95+
| ctrl_while | bashkit | 0.049 | ±0.002 | - | ✓ |
96+
| ctrl_while | bash | 8.485 | ±5.118 | - | ✓ |
97+
| ctrl_case | bashkit | 0.021 | ±0.001 | - | ✓ |
98+
| ctrl_case | bash | 4.695 | ±2.309 | - | ✓ |
99+
| ctrl_function | bashkit | 0.021 | ±0.002 | - | ✓ |
100+
| ctrl_function | bash | 6.677 | ±2.754 | - | ✓ |
101+
| ctrl_function_return | bashkit | 0.026 | ±0.002 | - | ✓ |
102+
| ctrl_function_return | bash | 8.239 | ±4.338 | - | ✓ |
103+
| ctrl_nested_loops | bashkit | 0.040 | ±0.004 | - | ✓ |
104+
| ctrl_nested_loops | bash | 14.515 | ±21.225 | - | ✓ |
105+
106+
### Io
107+
108+
| Benchmark | Runner | Mean (ms) | StdDev | Errors | Match |
109+
|-----------|--------|-----------|--------|--------|-------|
110+
| io_redirect_write | bashkit | 0.031 | ±0.002 | - | ✓ |
111+
| io_redirect_write | bash | 30.515 | ±13.023 | - | ✓ |
112+
| io_append | bashkit | 0.040 | ±0.006 | - | ✓ |
113+
| io_append | bash | 11.167 | ±4.241 | - | ✓ |
114+
| io_dev_null | bashkit | 0.018 | ±0.001 | - | ✓ |
115+
| io_dev_null | bash | 4.252 | ±2.416 | - | ✓ |
116+
| io_stderr_redirect | bashkit | 0.018 | ±0.001 | - | ✓ |
117+
| io_stderr_redirect | bash | 8.458 | ±2.933 | - | ✓ |
118+
| io_read_lines | bashkit | 0.050 | ±0.020 | - | ✓ |
119+
| io_read_lines | bash | 7.903 | ±2.437 | - | ✓ |
120+
| io_multiline_heredoc | bashkit | 0.118 | ±0.119 | - | ✓ |
121+
| io_multiline_heredoc | bash | 9.570 | ±1.853 | - | ✓ |
122+
123+
### Large
124+
125+
| Benchmark | Runner | Mean (ms) | StdDev | Errors | Match |
126+
|-----------|--------|-----------|--------|--------|-------|
127+
| large_loop_1000 | bashkit | 2.988 | ±1.128 | - | ✓ |
128+
| large_loop_1000 | bash | 12.302 | ±5.935 | - | ✓ |
129+
| large_string_append_100 | bashkit | 0.180 | ±0.002 | - | ✓ |
130+
| large_string_append_100 | bash | 6.234 | ±1.817 | - | ✓ |
131+
| large_array_fill_200 | bashkit | 0.775 | ±0.013 | - | ✓ |
132+
| large_array_fill_200 | bash | 10.175 | ±4.574 | - | ✓ |
133+
| large_nested_loops | bashkit | 1.327 | ±0.472 | - | ✓ |
134+
| large_nested_loops | bash | 12.297 | ±8.177 | - | ✓ |
135+
| large_fibonacci_12 | bashkit | 20.147 | ±16.629 | - | ✓ |
136+
| large_fibonacci_12 | bash | 1263.744 | ±562.827 | - | ✓ |
137+
| large_function_calls_500 | bashkit | 10.035 | ±8.573 | - | ✓ |
138+
| large_function_calls_500 | bash | 1032.523 | ±403.901 | - | ✓ |
139+
| large_multiline_script | bashkit | 0.220 | ±0.027 | - | ✗ |
140+
| large_multiline_script | bash | 4.159 | ±0.937 | - | ✓ |
141+
| large_pipeline_chain | bashkit | 0.468 | ±0.007 | - | ✓ |
142+
| large_pipeline_chain | bash | 7.702 | ±1.396 | - | ✓ |
143+
| large_assoc_array | bashkit | 0.023 | ±0.001 | - | ✗ |
144+
| large_assoc_array | bash | 3.870 | ±1.609 | - | ✓ |
145+
146+
### Pipes
147+
148+
| Benchmark | Runner | Mean (ms) | StdDev | Errors | Match |
149+
|-----------|--------|-----------|--------|--------|-------|
150+
| pipe_simple | bashkit | 0.024 | ±0.002 | - | ✓ |
151+
| pipe_simple | bash | 11.578 | ±5.362 | - | ✓ |
152+
| pipe_multi | bashkit | 0.035 | ±0.001 | - | ✓ |
153+
| pipe_multi | bash | 11.292 | ±3.853 | - | ✓ |
154+
| pipe_command_subst | bashkit | 0.021 | ±0.002 | - | ✓ |
155+
| pipe_command_subst | bash | 3.927 | ±1.124 | - | ✓ |
156+
| pipe_heredoc | bashkit | 0.022 | ±0.001 | - | ✓ |
157+
| pipe_heredoc | bash | 8.089 | ±4.187 | - | ✓ |
158+
| pipe_herestring | bashkit | 0.021 | ±0.001 | - | ✓ |
159+
| pipe_herestring | bash | 6.274 | ±1.308 | - | ✓ |
160+
| pipe_discard | bashkit | 0.020 | ±0.001 | - | ✓ |
161+
| pipe_discard | bash | 6.603 | ±3.417 | - | ✓ |
162+
163+
### Startup
164+
165+
| Benchmark | Runner | Mean (ms) | StdDev | Errors | Match |
166+
|-----------|--------|-----------|--------|--------|-------|
167+
| startup_empty | bashkit | 0.015 | ±0.001 | - | ✓ |
168+
| startup_empty | bash | 5.180 | ±3.605 | - | ✓ |
169+
| startup_true | bashkit | 0.018 | ±0.006 | - | ✓ |
170+
| startup_true | bash | 9.358 | ±5.640 | - | ✓ |
171+
| startup_echo | bashkit | 0.016 | ±0.001 | - | ✓ |
172+
| startup_echo | bash | 5.336 | ±2.543 | - | ✓ |
173+
| startup_exit | bashkit | 0.039 | ±0.002 | - | ✓ |
174+
| startup_exit | bash | 5.481 | ±5.334 | - | ✓ |
175+
176+
### Strings
177+
178+
| Benchmark | Runner | Mean (ms) | StdDev | Errors | Match |
179+
|-----------|--------|-----------|--------|--------|-------|
180+
| str_concat | bashkit | 0.019 | ±0.001 | - | ✓ |
181+
| str_concat | bash | 9.470 | ±4.095 | - | ✓ |
182+
| str_printf | bashkit | 0.017 | ±0.001 | - | ✓ |
183+
| str_printf | bash | 6.343 | ±7.660 | - | ✓ |
184+
| str_printf_pad | bashkit | 0.040 | ±0.002 | - | ✓ |
185+
| str_printf_pad | bash | 6.030 | ±2.633 | - | ✓ |
186+
| str_echo_escape | bashkit | 0.016 | ±0.001 | - | ✓ |
187+
| str_echo_escape | bash | 5.497 | ±1.638 | - | ✓ |
188+
| str_prefix_strip | bashkit | 0.040 | ±0.021 | - | ✓ |
189+
| str_prefix_strip | bash | 8.906 | ±7.406 | - | ✓ |
190+
| str_suffix_strip | bashkit | 0.020 | ±0.005 | - | ✓ |
191+
| str_suffix_strip | bash | 5.320 | ±3.266 | - | ✓ |
192+
| str_uppercase | bashkit | 0.018 | ±0.001 | - | ✗ |
193+
| str_uppercase | bash | 7.271 | ±3.867 | 10 | ✓ |
194+
| str_lowercase | bashkit | 0.100 | ±0.129 | - | ✗ |
195+
| str_lowercase | bash | 7.972 | ±5.049 | 10 | ✓ |
196+
197+
### Subshell
198+
199+
| Benchmark | Runner | Mean (ms) | StdDev | Errors | Match |
200+
|-----------|--------|-----------|--------|--------|-------|
201+
| subshell_simple | bashkit | 0.017 | ±0.001 | - | ✓ |
202+
| subshell_simple | bash | 3.553 | ±0.599 | - | ✓ |
203+
| subshell_isolation | bashkit | 0.022 | ±0.001 | - | ✓ |
204+
| subshell_isolation | bash | 6.233 | ±6.070 | - | ✓ |
205+
| subshell_nested | bashkit | 0.027 | ±0.001 | - | ✓ |
206+
| subshell_nested | bash | 8.139 | ±3.945 | - | ✓ |
207+
| subshell_pipeline | bashkit | 0.047 | ±0.004 | - | ✓ |
208+
| subshell_pipeline | bash | 51.381 | ±17.176 | - | ✓ |
209+
| subshell_capture_loop | bashkit | 0.045 | ±0.002 | - | ✓ |
210+
| subshell_capture_loop | bash | 11.158 | ±2.938 | - | ✓ |
211+
| subshell_process_subst | bashkit | 0.029 | ±0.001 | - | ✓ |
212+
| subshell_process_subst | bash | 7.270 | ±1.816 | - | ✓ |
213+
214+
### Tools
215+
216+
| Benchmark | Runner | Mean (ms) | StdDev | Errors | Match |
217+
|-----------|--------|-----------|--------|--------|-------|
218+
| tool_grep_simple | bashkit | 0.022 | ±0.002 | - | ✓ |
219+
| tool_grep_simple | bash | 13.882 | ±6.248 | - | ✓ |
220+
| tool_grep_case | bashkit | 1.787 | ±2.795 | - | ✓ |
221+
| tool_grep_case | bash | 13.085 | ±6.850 | - | ✓ |
222+
| tool_grep_count | bashkit | 0.023 | ±0.002 | - | ✓ |
223+
| tool_grep_count | bash | 11.065 | ±4.521 | - | ✓ |
224+
| tool_grep_invert | bashkit | 0.068 | ±0.083 | - | ✓ |
225+
| tool_grep_invert | bash | 14.331 | ±5.223 | - | ✓ |
226+
| tool_grep_regex | bashkit | 0.066 | ±0.035 | - | ✓ |
227+
| tool_grep_regex | bash | 14.363 | ±3.163 | - | ✓ |
228+
| tool_sed_replace | bashkit | 0.057 | ±0.003 | - | ✓ |
229+
| tool_sed_replace | bash | 9.419 | ±2.948 | - | ✓ |
230+
| tool_sed_global | bashkit | 0.057 | ±0.003 | - | ✓ |
231+
| tool_sed_global | bash | 9.460 | ±2.834 | - | ✓ |
232+
| tool_sed_delete | bashkit | 0.020 | ±0.001 | - | ✓ |
233+
| tool_sed_delete | bash | 10.257 | ±4.791 | - | ✓ |
234+
| tool_sed_lines | bashkit | 0.022 | ±0.007 | - | ✓ |
235+
| tool_sed_lines | bash | 9.648 | ±5.290 | - | ✓ |
236+
| tool_sed_backrefs | bashkit | 0.078 | ±0.007 | - | ✓ |
237+
| tool_sed_backrefs | bash | 10.159 | ±4.331 | - | ✓ |
238+
| tool_awk_print | bashkit | 0.021 | ±0.002 | - | ✓ |
239+
| tool_awk_print | bash | 8.059 | ±2.451 | - | ✓ |
240+
| tool_awk_sum | bashkit | 0.023 | ±0.001 | - | ✓ |
241+
| tool_awk_sum | bash | 9.403 | ±4.043 | - | ✓ |
242+
| tool_awk_pattern | bashkit | 0.031 | ±0.002 | - | ✓ |
243+
| tool_awk_pattern | bash | 10.155 | ±2.492 | - | ✓ |
244+
| tool_awk_fieldsep | bashkit | 0.020 | ±0.001 | - | ✓ |
245+
| tool_awk_fieldsep | bash | 10.116 | ±4.357 | - | ✓ |
246+
| tool_awk_nf | bashkit | 0.020 | ±0.001 | - | ✓ |
247+
| tool_awk_nf | bash | 12.800 | ±5.745 | - | ✓ |
248+
| tool_awk_compute | bashkit | 0.020 | ±0.001 | - | ✓ |
249+
| tool_awk_compute | bash | 10.985 | ±2.700 | - | ✓ |
250+
| tool_jq_identity | bashkit | 0.249 | ±0.014 | - | ✓ |
251+
| tool_jq_identity | bash | 11.116 | ±4.503 | - | ✓ |
252+
| tool_jq_field | bashkit | 0.244 | ±0.013 | - | ✓ |
253+
| tool_jq_field | bash | 9.068 | ±2.884 | - | ✓ |
254+
| tool_jq_array | bashkit | 0.650 | ±0.263 | - | ✓ |
255+
| tool_jq_array | bash | 12.901 | ±5.862 | - | ✓ |
256+
| tool_jq_filter | bashkit | 0.250 | ±0.018 | - | ✓ |
257+
| tool_jq_filter | bash | 10.880 | ±3.260 | - | ✓ |
258+
| tool_jq_map | bashkit | 0.382 | ±0.240 | - | ✓ |
259+
| tool_jq_map | bash | 10.667 | ±3.244 | - | ✓ |
260+
261+
### Variables
262+
263+
| Benchmark | Runner | Mean (ms) | StdDev | Errors | Match |
264+
|-----------|--------|-----------|--------|--------|-------|
265+
| var_assign_simple | bashkit | 0.064 | ±0.139 | - | ✓ |
266+
| var_assign_simple | bash | 10.405 | ±6.939 | - | ✓ |
267+
| var_assign_many | bashkit | 1.554 | ±3.485 | - | ✓ |
268+
| var_assign_many | bash | 7.201 | ±4.621 | - | ✓ |
269+
| var_default | bashkit | 0.302 | ±0.786 | - | ✓ |
270+
| var_default | bash | 5.340 | ±1.782 | - | ✓ |
271+
| var_length | bashkit | 0.017 | ±0.001 | - | ✓ |
272+
| var_length | bash | 7.034 | ±4.333 | - | ✓ |
273+
| var_substring | bashkit | 0.018 | ±0.001 | - | ✓ |
274+
| var_substring | bash | 7.026 | ±2.972 | - | ✓ |
275+
| var_replace | bashkit | 0.063 | ±0.046 | - | ✓ |
276+
| var_replace | bash | 7.640 | ±5.310 | - | ✓ |
277+
| var_nested | bashkit | 0.019 | ±0.001 | - | ✓ |
278+
| var_nested | bash | 7.555 | ±3.479 | - | ✓ |
279+
| var_export | bashkit | 0.159 | ±0.315 | - | ✓ |
280+
| var_export | bash | 6.428 | ±2.431 | - | ✓ |
281+
282+
## Runner Descriptions
283+
284+
| Runner | Type | Description |
285+
|--------|------|-------------|
286+
| bashkit | in-process | Rust library call, no fork/exec |
287+
| bashkit-cli | subprocess | bashkit binary, new process per run |
288+
| bashkit-js | persistent child | Node.js + @everruns/bashkit, warm interpreter |
289+
| bashkit-py | persistent child | Python + bashkit package, warm interpreter |
290+
| bash | subprocess | /bin/bash, new process per run |
291+
| gbash | subprocess | gbash binary (Go), new process per run |
292+
| gbash-server | persistent child | gbash JSON-RPC server, warm interpreter |
293+
| just-bash | subprocess | just-bash CLI, new process per run |
294+
| just-bash-inproc | persistent child | Node.js + just-bash library, warm interpreter |
295+
296+
## Assumptions & Notes
297+
298+
- Times measured in nanoseconds, displayed in milliseconds
299+
- Prewarm phase runs first few cases to warm up JIT/compilation
300+
- Per-benchmark warmup iterations excluded from timing
301+
- Output match compares against bash output when available
302+
- Errors include execution failures and exit code mismatches
303+
- In-process: interpreter runs inside the benchmark process
304+
- Subprocess: new process spawned per benchmark run
305+
- Persistent child: long-lived child process, amortizes startup cost

‎crates/bashkit/docs/threat-model.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,7 @@ let bash = Bash::builder()
157157
| Contradictory execution-profile limits (TM-DOS-097) | Host config silently requests ineffective or impossible limits | Validate profile cross-field invariants before `BashBuilder` accepts it | MITIGATED |
158158
| Suspended host-call retention (TM-DOS-098) | Script repeats event-backed calls or host never resumes one | Capacity-one channel, normal execution limits, and handle-owned session released on drop | MITIGATED |
159159
| `time` report amplification (TM-DOS-099) | Attacker-controlled `-f` format expands repeatedly or targets the VFS with `-o` | Incremental rendering is capped by the stderr limit before emission or file replacement | MITIGATED |
160+
| Post-allocation charging (TM-DOS-100) | Archive/compression buffers grow before their live-byte check | Owning budget-aware string/vector/byte builders charge before reserve, roll back errors, and release on drop | MITIGATED |
160161

161162
### Sandbox Escape (TM-ESC-*)
162163

0 commit comments

Comments
 (0)