ci: replace use of cargo cross for centos7 tests#1675
Conversation
299f9a9 to
0635890
Compare
BenchmarksComparisonBenchmark execution time: 2026-03-07 02:53:10 Comparing candidate commit 1eb41be in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 2 unstable metrics.
|
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| redis/obfuscate_redis_string | execution_time | 32.695µs | 33.528µs ± 1.141µs | 32.817µs ± 0.064µs | 34.824µs | 35.480µs | 35.948µs | 36.212µs | 10.35% | 0.990 | -0.856 | 3.39% | 0.081µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| redis/obfuscate_redis_string | execution_time | [33.370µs; 33.686µs] or [-0.472%; +0.472%] | None | None | None |
Group 2
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| benching serializing traces from their internal representation to msgpack | execution_time | 13.958ms | 14.024ms ± 0.032ms | 14.019ms ± 0.015ms | 14.034ms | 14.073ms | 14.140ms | 14.204ms | 1.32% | 2.129 | 7.154 | 0.22% | 0.002ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| benching serializing traces from their internal representation to msgpack | execution_time | [14.020ms; 14.029ms] or [-0.031%; +0.031%] | None | None | None |
Group 3
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | execution_time | 185.334µs | 185.757µs ± 0.410µs | 185.701µs ± 0.142µs | 185.871µs | 186.098µs | 186.240µs | 190.822µs | 2.76% | 9.523 | 114.978 | 0.22% | 0.029µs | 1 | 200 |
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | throughput | 5240481.778op/s | 5383403.325op/s ± 11635.577op/s | 5385007.506op/s ± 4108.061op/s | 5388567.044op/s | 5391894.702op/s | 5394923.761op/s | 5395656.766op/s | 0.20% | -9.350 | 112.115 | 0.22% | 822.760op/s | 1 | 200 |
| normalization/normalize_name/normalize_name/bad-name | execution_time | 17.828µs | 17.917µs ± 0.046µs | 17.912µs ± 0.033µs | 17.952µs | 17.985µs | 18.019µs | 18.075µs | 0.91% | 0.279 | 0.048 | 0.25% | 0.003µs | 1 | 200 |
| normalization/normalize_name/normalize_name/bad-name | throughput | 55325986.290op/s | 55811839.606op/s ± 141643.846op/s | 55828232.964op/s ± 101941.433op/s | 55909991.493op/s | 56067192.300op/s | 56086509.152op/s | 56091356.322op/s | 0.47% | -0.264 | 0.023 | 0.25% | 10015.732op/s | 1 | 200 |
| normalization/normalize_name/normalize_name/good | execution_time | 10.269µs | 10.346µs ± 0.040µs | 10.340µs ± 0.022µs | 10.366µs | 10.426µs | 10.460µs | 10.473µs | 1.28% | 0.754 | 0.587 | 0.39% | 0.003µs | 1 | 200 |
| normalization/normalize_name/normalize_name/good | throughput | 95486338.893op/s | 96659270.824op/s ± 373483.964op/s | 96711513.249op/s ± 208107.466op/s | 96888689.719op/s | 97197755.432op/s | 97297353.491op/s | 97382564.804op/s | 0.69% | -0.731 | 0.542 | 0.39% | 26409.304op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | execution_time | [185.700µs; 185.814µs] or [-0.031%; +0.031%] | None | None | None |
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | throughput | [5381790.746op/s; 5385015.904op/s] or [-0.030%; +0.030%] | None | None | None |
| normalization/normalize_name/normalize_name/bad-name | execution_time | [17.911µs; 17.924µs] or [-0.035%; +0.035%] | None | None | None |
| normalization/normalize_name/normalize_name/bad-name | throughput | [55792209.131op/s; 55831470.080op/s] or [-0.035%; +0.035%] | None | None | None |
| normalization/normalize_name/normalize_name/good | execution_time | [10.340µs; 10.351µs] or [-0.054%; +0.054%] | None | None | None |
| normalization/normalize_name/normalize_name/good | throughput | [96607509.539op/s; 96711032.109op/s] or [-0.054%; +0.054%] | None | None | None |
Group 4
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| two way interface | execution_time | 18.005µs | 25.938µs ± 9.977µs | 18.493µs ± 0.447µs | 33.785µs | 43.102µs | 57.661µs | 73.154µs | 295.58% | 1.239 | 2.018 | 38.37% | 0.705µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| two way interface | execution_time | [24.555µs; 27.321µs] or [-5.331%; +5.331%] | None | None | None |
Group 5
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| normalization/normalize_trace/test_trace | execution_time | 240.540ns | 249.472ns ± 12.105ns | 244.066ns ± 2.254ns | 249.907ns | 283.545ns | 288.585ns | 296.015ns | 21.28% | 2.149 | 3.630 | 4.84% | 0.856ns | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| normalization/normalize_trace/test_trace | execution_time | [247.794ns; 251.149ns] or [-0.673%; +0.673%] | None | None | None |
Group 6
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| concentrator/add_spans_to_concentrator | execution_time | 10.639ms | 10.664ms ± 0.015ms | 10.662ms ± 0.008ms | 10.671ms | 10.689ms | 10.716ms | 10.741ms | 0.74% | 1.470 | 4.398 | 0.14% | 0.001ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| concentrator/add_spans_to_concentrator | execution_time | [10.662ms; 10.666ms] or [-0.019%; +0.019%] | None | None | None |
Group 7
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| benching string interning on wordpress profile | execution_time | 160.690µs | 161.215µs ± 0.428µs | 161.122µs ± 0.120µs | 161.251µs | 161.922µs | 163.134µs | 164.443µs | 2.06% | 3.783 | 20.674 | 0.26% | 0.030µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| benching string interning on wordpress profile | execution_time | [161.156µs; 161.274µs] or [-0.037%; +0.037%] | None | None | None |
Group 8
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| profile_add_sample2_frames_x1000 | execution_time | 738.975µs | 739.932µs ± 0.546µs | 739.934µs ± 0.335µs | 740.216µs | 740.778µs | 741.386µs | 743.760µs | 0.52% | 1.942 | 10.948 | 0.07% | 0.039µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| profile_add_sample2_frames_x1000 | execution_time | [739.856µs; 740.008µs] or [-0.010%; +0.010%] | None | None | None |
Group 9
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| single_flag_killswitch/rules-based | execution_time | 187.572ns | 189.783ns ± 1.994ns | 189.673ns ± 1.455ns | 190.724ns | 193.597ns | 196.602ns | 199.477ns | 5.17% | 1.554 | 3.465 | 1.05% | 0.141ns | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| single_flag_killswitch/rules-based | execution_time | [189.507ns; 190.059ns] or [-0.146%; +0.146%] | None | None | None |
Group 10
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ip_address/quantize_peer_ip_address_benchmark | execution_time | 4.924µs | 4.999µs ± 0.049µs | 4.982µs ± 0.039µs | 5.048µs | 5.080µs | 5.084µs | 5.090µs | 2.17% | 0.440 | -1.226 | 0.97% | 0.003µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| ip_address/quantize_peer_ip_address_benchmark | execution_time | [4.993µs; 5.006µs] or [-0.135%; +0.135%] | None | None | None |
Group 11
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| receiver_entry_point/report/2597 | execution_time | 3.104ms | 3.133ms ± 0.028ms | 3.123ms ± 0.008ms | 3.137ms | 3.196ms | 3.226ms | 3.262ms | 4.43% | 1.984 | 3.818 | 0.88% | 0.002ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| receiver_entry_point/report/2597 | execution_time | [3.130ms; 3.137ms] or [-0.123%; +0.123%] | None | None | None |
Group 12
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | execution_time | 534.271µs | 535.347µs ± 0.862µs | 535.131µs ± 0.349µs | 535.537µs | 537.378µs | 538.608µs | 539.354µs | 0.79% | 2.138 | 5.306 | 0.16% | 0.061µs | 1 | 200 |
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | throughput | 1854070.273op/s | 1867951.828op/s ± 2996.266op/s | 1868700.282op/s ± 1218.181op/s | 1869725.552op/s | 1870890.558op/s | 1871313.125op/s | 1871707.859op/s | 0.16% | -2.124 | 5.237 | 0.16% | 211.868op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | execution_time | 380.893µs | 381.580µs ± 0.298µs | 381.580µs ± 0.211µs | 381.785µs | 382.066µs | 382.213µs | 382.406µs | 0.22% | 0.086 | -0.432 | 0.08% | 0.021µs | 1 | 200 |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | throughput | 2615023.515op/s | 2620685.463op/s ± 2047.557op/s | 2620681.679op/s ± 1448.580op/s | 2622162.414op/s | 2624027.887op/s | 2625057.308op/s | 2625405.939op/s | 0.18% | -0.082 | -0.433 | 0.08% | 144.784op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | execution_time | 189.996µs | 190.339µs ± 0.151µs | 190.336µs ± 0.097µs | 190.413µs | 190.594µs | 190.773µs | 190.816µs | 0.25% | 0.667 | 0.695 | 0.08% | 0.011µs | 1 | 200 |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | throughput | 5240646.454op/s | 5253799.667op/s ± 4159.610op/s | 5253877.463op/s ± 2689.860op/s | 5256696.982op/s | 5259851.783op/s | 5261081.672op/s | 5263258.756op/s | 0.18% | -0.661 | 0.684 | 0.08% | 294.129op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/[empty string] | execution_time | 37.338µs | 37.529µs ± 0.068µs | 37.528µs ± 0.045µs | 37.568µs | 37.643µs | 37.687µs | 37.761µs | 0.62% | 0.159 | 0.423 | 0.18% | 0.005µs | 1 | 200 |
| normalization/normalize_service/normalize_service/[empty string] | throughput | 26482508.146op/s | 26645829.192op/s ± 48368.185op/s | 26646882.414op/s ± 32096.814op/s | 26679306.517op/s | 26719588.643op/s | 26755422.456op/s | 26782579.972op/s | 0.51% | -0.146 | 0.415 | 0.18% | 3420.147op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/test_ASCII | execution_time | 45.816µs | 45.898µs ± 0.108µs | 45.884µs ± 0.028µs | 45.915µs | 45.972µs | 46.038µs | 47.272µs | 3.03% | 10.522 | 131.111 | 0.23% | 0.008µs | 1 | 200 |
| normalization/normalize_service/normalize_service/test_ASCII | throughput | 21154038.306op/s | 21787555.741op/s ± 49896.708op/s | 21794089.443op/s ± 13532.043op/s | 21805286.336op/s | 21817315.034op/s | 21820676.382op/s | 21826466.588op/s | 0.15% | -10.365 | 128.376 | 0.23% | 3528.230op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | execution_time | [535.228µs; 535.467µs] or [-0.022%; +0.022%] | None | None | None |
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | throughput | [1867536.574op/s; 1868367.082op/s] or [-0.022%; +0.022%] | None | None | None |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | execution_time | [381.538µs; 381.621µs] or [-0.011%; +0.011%] | None | None | None |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | throughput | [2620401.691op/s; 2620969.235op/s] or [-0.011%; +0.011%] | None | None | None |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | execution_time | [190.318µs; 190.359µs] or [-0.011%; +0.011%] | None | None | None |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | throughput | [5253223.185op/s; 5254376.149op/s] or [-0.011%; +0.011%] | None | None | None |
| normalization/normalize_service/normalize_service/[empty string] | execution_time | [37.520µs; 37.539µs] or [-0.025%; +0.025%] | None | None | None |
| normalization/normalize_service/normalize_service/[empty string] | throughput | [26639125.827op/s; 26652532.557op/s] or [-0.025%; +0.025%] | None | None | None |
| normalization/normalize_service/normalize_service/test_ASCII | execution_time | [45.883µs; 45.913µs] or [-0.033%; +0.033%] | None | None | None |
| normalization/normalize_service/normalize_service/test_ASCII | throughput | [21780640.537op/s; 21794470.944op/s] or [-0.032%; +0.032%] | None | None | None |
Group 13
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| sdk_test_data/rules-based | execution_time | 144.215µs | 145.912µs ± 1.744µs | 145.653µs ± 0.528µs | 146.119µs | 147.504µs | 152.328µs | 161.831µs | 11.11% | 5.475 | 40.789 | 1.19% | 0.123µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| sdk_test_data/rules-based | execution_time | [145.671µs; 146.154µs] or [-0.166%; +0.166%] | None | None | None |
Group 14
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| sql/obfuscate_sql_string | execution_time | 85.193µs | 85.542µs ± 0.172µs | 85.519µs ± 0.040µs | 85.561µs | 85.657µs | 86.068µs | 87.314µs | 2.10% | 7.100 | 64.752 | 0.20% | 0.012µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| sql/obfuscate_sql_string | execution_time | [85.518µs; 85.565µs] or [-0.028%; +0.028%] | None | None | None |
Group 15
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| write only interface | execution_time | 1.240µs | 3.208µs ± 1.396µs | 2.998µs ± 0.025µs | 3.026µs | 3.675µs | 13.715µs | 14.721µs | 390.98% | 7.312 | 54.824 | 43.41% | 0.099µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| write only interface | execution_time | [3.015µs; 3.402µs] or [-6.031%; +6.031%] | None | None | None |
Group 16
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| profile_add_sample_frames_x1000 | execution_time | 4.202ms | 4.206ms ± 0.008ms | 4.205ms ± 0.001ms | 4.207ms | 4.211ms | 4.215ms | 4.307ms | 2.43% | 11.553 | 148.511 | 0.18% | 0.001ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| profile_add_sample_frames_x1000 | execution_time | [4.205ms; 4.207ms] or [-0.025%; +0.025%] | None | None | None |
Group 17
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| benching deserializing traces from msgpack to their internal representation | execution_time | 48.231ms | 48.743ms ± 1.098ms | 48.576ms ± 0.061ms | 48.656ms | 48.938ms | 55.403ms | 60.298ms | 24.13% | 8.413 | 75.099 | 2.25% | 0.078ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| benching deserializing traces from msgpack to their internal representation | execution_time | [48.591ms; 48.895ms] or [-0.312%; +0.312%] | None | None | None |
Group 18
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| credit_card/is_card_number/ | execution_time | 3.893µs | 3.913µs ± 0.003µs | 3.913µs ± 0.002µs | 3.915µs | 3.919µs | 3.921µs | 3.921µs | 0.20% | -0.878 | 7.075 | 0.08% | 0.000µs | 1 | 200 |
| credit_card/is_card_number/ | throughput | 255033190.221op/s | 255535616.453op/s ± 209439.270op/s | 255540925.790op/s ± 126978.171op/s | 255672949.802op/s | 255807100.051op/s | 255882956.415op/s | 256870713.966op/s | 0.52% | 0.898 | 7.209 | 0.08% | 14809.593op/s | 1 | 200 |
| credit_card/is_card_number/ 3782-8224-6310-005 | execution_time | 78.860µs | 79.642µs ± 0.411µs | 79.583µs ± 0.259µs | 79.881µs | 80.391µs | 80.712µs | 80.924µs | 1.69% | 0.559 | -0.133 | 0.51% | 0.029µs | 1 | 200 |
| credit_card/is_card_number/ 3782-8224-6310-005 | throughput | 12357302.577op/s | 12556581.917op/s ± 64639.419op/s | 12565531.500op/s ± 40735.385op/s | 12601791.979op/s | 12645338.851op/s | 12668203.664op/s | 12680703.803op/s | 0.92% | -0.535 | -0.170 | 0.51% | 4570.697op/s | 1 | 200 |
| credit_card/is_card_number/ 378282246310005 | execution_time | 72.304µs | 73.000µs ± 0.393µs | 72.936µs ± 0.272µs | 73.235µs | 73.718µs | 74.059µs | 74.175µs | 1.70% | 0.572 | -0.063 | 0.54% | 0.028µs | 1 | 200 |
| credit_card/is_card_number/ 378282246310005 | throughput | 13481693.631op/s | 13699072.783op/s ± 73515.243op/s | 13710671.025op/s ± 51032.483op/s | 13756565.285op/s | 13805691.473op/s | 13812329.200op/s | 13830559.095op/s | 0.87% | -0.546 | -0.110 | 0.54% | 5198.313op/s | 1 | 200 |
| credit_card/is_card_number/37828224631 | execution_time | 3.895µs | 3.912µs ± 0.003µs | 3.912µs ± 0.001µs | 3.913µs | 3.916µs | 3.917µs | 3.930µs | 0.47% | 0.360 | 16.357 | 0.07% | 0.000µs | 1 | 200 |
| credit_card/is_card_number/37828224631 | throughput | 254427734.116op/s | 255624804.496op/s ± 179052.168op/s | 255631375.787op/s ± 81409.138op/s | 255711411.123op/s | 255855634.430op/s | 255924332.268op/s | 256764030.141op/s | 0.44% | -0.322 | 16.330 | 0.07% | 12660.900op/s | 1 | 200 |
| credit_card/is_card_number/378282246310005 | execution_time | 69.163µs | 69.736µs ± 0.371µs | 69.698µs ± 0.260µs | 69.957µs | 70.377µs | 70.663µs | 71.229µs | 2.20% | 0.771 | 0.640 | 0.53% | 0.026µs | 1 | 200 |
| credit_card/is_card_number/378282246310005 | throughput | 14039188.640op/s | 14340206.806op/s ± 76045.960op/s | 14347691.906op/s ± 53448.266op/s | 14401249.701op/s | 14441069.246op/s | 14453212.508op/s | 14458564.415op/s | 0.77% | -0.739 | 0.537 | 0.53% | 5377.261op/s | 1 | 200 |
| credit_card/is_card_number/37828224631000521389798 | execution_time | 52.148µs | 52.233µs ± 0.104µs | 52.224µs ± 0.021µs | 52.241µs | 52.288µs | 52.410µs | 53.227µs | 1.92% | 8.110 | 72.768 | 0.20% | 0.007µs | 1 | 200 |
| credit_card/is_card_number/37828224631000521389798 | throughput | 18787466.685op/s | 19144949.575op/s ± 37483.184op/s | 19148408.444op/s ± 7750.336op/s | 19156530.480op/s | 19168004.766op/s | 19170904.836op/s | 19176213.824op/s | 0.15% | -8.056 | 72.092 | 0.20% | 2650.461op/s | 1 | 200 |
| credit_card/is_card_number/x371413321323331 | execution_time | 6.028µs | 6.039µs ± 0.013µs | 6.036µs ± 0.003µs | 6.039µs | 6.072µs | 6.110µs | 6.126µs | 1.48% | 3.826 | 16.762 | 0.22% | 0.001µs | 1 | 200 |
| credit_card/is_card_number/x371413321323331 | throughput | 163250724.129op/s | 165586919.105op/s ± 363058.904op/s | 165673824.257op/s ± 81862.767op/s | 165745607.517op/s | 165818079.793op/s | 165882659.906op/s | 165888978.601op/s | 0.13% | -3.799 | 16.505 | 0.22% | 25672.141op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/ | execution_time | 3.893µs | 3.912µs ± 0.003µs | 3.912µs ± 0.002µs | 3.914µs | 3.916µs | 3.918µs | 3.920µs | 0.18% | -1.845 | 14.463 | 0.07% | 0.000µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/ | throughput | 255130726.809op/s | 255595702.595op/s ± 173166.251op/s | 255594631.142op/s ± 99462.326op/s | 255695218.502op/s | 255821684.143op/s | 255963419.891op/s | 256895101.703op/s | 0.51% | 1.871 | 14.688 | 0.07% | 12244.703op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | execution_time | 64.136µs | 64.434µs ± 0.147µs | 64.407µs ± 0.092µs | 64.521µs | 64.693µs | 64.863µs | 64.902µs | 0.77% | 0.715 | 0.530 | 0.23% | 0.010µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | throughput | 15407833.501op/s | 15519863.293op/s ± 35283.053op/s | 15526342.256op/s ± 22275.981op/s | 15543694.135op/s | 15569019.843op/s | 15582027.039op/s | 15591901.566op/s | 0.42% | -0.701 | 0.502 | 0.23% | 2494.889op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 378282246310005 | execution_time | 57.798µs | 58.069µs ± 0.161µs | 58.026µs ± 0.087µs | 58.147µs | 58.380µs | 58.547µs | 58.631µs | 1.04% | 1.105 | 1.047 | 0.28% | 0.011µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 378282246310005 | throughput | 17055697.599op/s | 17221058.643op/s ± 47622.536op/s | 17233641.862op/s ± 25777.790op/s | 17254981.323op/s | 17277846.781op/s | 17289398.789op/s | 17301771.701op/s | 0.40% | -1.089 | 1.001 | 0.28% | 3367.422op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631 | execution_time | 3.892µs | 3.912µs ± 0.003µs | 3.911µs ± 0.002µs | 3.913µs | 3.916µs | 3.919µs | 3.921µs | 0.24% | -1.183 | 11.828 | 0.07% | 0.000µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631 | throughput | 255067203.774op/s | 255645989.029op/s ± 180003.737op/s | 255667888.114op/s ± 98969.346op/s | 255749461.946op/s | 255854218.689op/s | 255923712.130op/s | 256925508.985op/s | 0.49% | 1.209 | 12.018 | 0.07% | 12728.186op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/378282246310005 | execution_time | 54.586µs | 54.976µs ± 0.284µs | 54.965µs ± 0.226µs | 55.141µs | 55.442µs | 55.780µs | 55.853µs | 1.61% | 0.751 | 0.228 | 0.52% | 0.020µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/378282246310005 | throughput | 17904294.463op/s | 18190094.009op/s ± 93628.570op/s | 18193255.768op/s ± 74697.280op/s | 18278332.250op/s | 18307636.085op/s | 18315998.019op/s | 18319638.229op/s | 0.69% | -0.726 | 0.164 | 0.51% | 6620.540op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | execution_time | 52.150µs | 52.217µs ± 0.041µs | 52.206µs ± 0.021µs | 52.235µs | 52.294µs | 52.326µs | 52.446µs | 0.46% | 1.573 | 4.505 | 0.08% | 0.003µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | throughput | 19067200.760op/s | 19150797.601op/s ± 14838.976op/s | 19154754.555op/s ± 7783.641op/s | 19161236.818op/s | 19168622.881op/s | 19170342.657op/s | 19175459.247op/s | 0.11% | -1.564 | 4.443 | 0.08% | 1049.274op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/x371413321323331 | execution_time | 6.028µs | 6.037µs ± 0.011µs | 6.034µs ± 0.002µs | 6.037µs | 6.054µs | 6.074µs | 6.121µs | 1.44% | 4.654 | 25.884 | 0.19% | 0.001µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/x371413321323331 | throughput | 163374171.292op/s | 165657409.620op/s ± 305718.413op/s | 165733308.917op/s ± 58757.933op/s | 165785804.590op/s | 165842259.925op/s | 165857642.165op/s | 165884335.596op/s | 0.09% | -4.620 | 25.490 | 0.18% | 21617.556op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| credit_card/is_card_number/ | execution_time | [3.913µs; 3.914µs] or [-0.011%; +0.011%] | None | None | None |
| credit_card/is_card_number/ | throughput | [255506590.185op/s; 255564642.722op/s] or [-0.011%; +0.011%] | None | None | None |
| credit_card/is_card_number/ 3782-8224-6310-005 | execution_time | [79.585µs; 79.699µs] or [-0.072%; +0.072%] | None | None | None |
| credit_card/is_card_number/ 3782-8224-6310-005 | throughput | [12547623.515op/s; 12565540.319op/s] or [-0.071%; +0.071%] | None | None | None |
| credit_card/is_card_number/ 378282246310005 | execution_time | [72.945µs; 73.054µs] or [-0.075%; +0.075%] | None | None | None |
| credit_card/is_card_number/ 378282246310005 | throughput | [13688884.277op/s; 13709261.288op/s] or [-0.074%; +0.074%] | None | None | None |
| credit_card/is_card_number/37828224631 | execution_time | [3.912µs; 3.912µs] or [-0.010%; +0.010%] | None | None | None |
| credit_card/is_card_number/37828224631 | throughput | [255599989.587op/s; 255649619.404op/s] or [-0.010%; +0.010%] | None | None | None |
| credit_card/is_card_number/378282246310005 | execution_time | [69.685µs; 69.787µs] or [-0.074%; +0.074%] | None | None | None |
| credit_card/is_card_number/378282246310005 | throughput | [14329667.567op/s; 14350746.045op/s] or [-0.073%; +0.073%] | None | None | None |
| credit_card/is_card_number/37828224631000521389798 | execution_time | [52.219µs; 52.248µs] or [-0.028%; +0.028%] | None | None | None |
| credit_card/is_card_number/37828224631000521389798 | throughput | [19139754.766op/s; 19150144.383op/s] or [-0.027%; +0.027%] | None | None | None |
| credit_card/is_card_number/x371413321323331 | execution_time | [6.037µs; 6.041µs] or [-0.031%; +0.031%] | None | None | None |
| credit_card/is_card_number/x371413321323331 | throughput | [165536602.633op/s; 165637235.577op/s] or [-0.030%; +0.030%] | None | None | None |
| credit_card/is_card_number_no_luhn/ | execution_time | [3.912µs; 3.913µs] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number_no_luhn/ | throughput | [255571703.418op/s; 255619701.772op/s] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | execution_time | [64.414µs; 64.454µs] or [-0.032%; +0.032%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | throughput | [15514973.402op/s; 15524753.185op/s] or [-0.032%; +0.032%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 378282246310005 | execution_time | [58.047µs; 58.091µs] or [-0.038%; +0.038%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 378282246310005 | throughput | [17214458.618op/s; 17227658.669op/s] or [-0.038%; +0.038%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631 | execution_time | [3.911µs; 3.912µs] or [-0.010%; +0.010%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631 | throughput | [255621042.242op/s; 255670935.816op/s] or [-0.010%; +0.010%] | None | None | None |
| credit_card/is_card_number_no_luhn/378282246310005 | execution_time | [54.937µs; 55.016µs] or [-0.072%; +0.072%] | None | None | None |
| credit_card/is_card_number_no_luhn/378282246310005 | throughput | [18177117.990op/s; 18203070.029op/s] or [-0.071%; +0.071%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | execution_time | [52.212µs; 52.223µs] or [-0.011%; +0.011%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | throughput | [19148741.062op/s; 19152854.141op/s] or [-0.011%; +0.011%] | None | None | None |
| credit_card/is_card_number_no_luhn/x371413321323331 | execution_time | [6.035µs; 6.038µs] or [-0.026%; +0.026%] | None | None | None |
| credit_card/is_card_number_no_luhn/x371413321323331 | throughput | [165615039.988op/s; 165699779.252op/s] or [-0.026%; +0.026%] | None | None | None |
Group 19
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 1eb41be | 1772851054 | ekump/APMSP-2688-use-nextest-with-cross-for-centos-test |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| tags/replace_trace_tags | execution_time | 2.321µs | 2.394µs ± 0.018µs | 2.397µs ± 0.005µs | 2.403µs | 2.414µs | 2.420µs | 2.421µs | 1.01% | -2.351 | 5.983 | 0.74% | 0.001µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| tags/replace_trace_tags | execution_time | [2.392µs; 2.397µs] or [-0.103%; +0.103%] | None | None | None |
Baseline
Omitted due to size.
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1675 +/- ##
=======================================
Coverage 71.37% 71.38%
=======================================
Files 427 427
Lines 62903 62903
=======================================
+ Hits 44899 44902 +3
+ Misses 18004 18001 -3
🚀 New features to boost your workflow:
|
run centOS7 tests. And switch to nextest as a test runner for CentOS7 so we can upload junit.xml files
052d18e to
1eb41be
Compare
@iunanua - In theory, yes it should just work. |
|
Thanks for this! |
# What does this PR do? Bump to 29.0.0 [feat(profiling)!: add Tracepoint sample type](#1676) [feat(obfuscation/redis): Reach feature parity on redis obfuscation [APMSP-2668]](#1632) [fix(sidecar): Handle backpressure more gracefully](#1682) [feat(trace-protobuf)!: Add two fields to ClientGroupedStats [SVLS-8627]](#1630) [chore: exclude libdatadog from ADMS auto generated PRs for dependency updates](#1688) [chore(ci): run crashtracking ffi example tests in CI](#1687) [fix(crashtracking): use libunwind to unwind frames](#1663) [feat: publish tracer metadata as OTel process ctx](#1658) [ci: run thread count test in own process](#1693) [feat(obfuscation/json): Init json obfuscation [APMSP-2665]](#1635) [chore(ci): add final_status property on junit XML [APMSP-2610]](#1681) [refactor(data-pipeline-ffi): move macro definitions for better reuse](#1699) [fix(obfuscation/memcached): fuzzing fix](#1695) [ci: replace use of cargo cross for centos7 tests](#1675) #1702 (comment) [feat(stats_exporter)!: add process tags to CSS payloads](#1709) # Motivation I mainly am creating this to use the new unwinding from ucontext for crashtracking # Additional Notes Anything else we should know when reviewing? # How to test the change? Describe here in detail how the change can be validated. [APMSP-2668]: https://datadoghq.atlassian.net/browse/APMSP-2668?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SVLS-8627]: https://datadoghq.atlassian.net/browse/SVLS-8627?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMSP-2665]: https://datadoghq.atlassian.net/browse/APMSP-2665?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: gyuheon.oh <gyuheon.oh@datadoghq.com>
What does this PR do?
The Centos7 tests we run uses cargo cross. However, we don't actually do any cross-compilation. The host is
x86_64and the build target isx86_64-unknown-linux-gnu, the same architecture. What this test is doing is running tests for thex86_64-unknown-linux-gnubuild in a CentOS7 environment with the older 2.17 glibc. Cross makes it easier to leverage the centOS7 docker container to run tests, but isn't really necessary. Using cargo cross means we can can't use nextest as the test runner. Without nextest we don't get process isolation for tests and we can't generate a junit.xml file to upload to CI Viz / test optimization. We can just handle spinning up the docker container ourselves.Motivation
What inspired you to submit this pull request?
Additional Notes
The nextest binary needs to be the musl variant since the regular version requires glibc 2.25+, which isn't available in CentOS7
As a future enhancement we should look into not building the docker image on every run.
How to test the change?
Observe the CI runs