[BUG FIX] Various minor bug fixes. #2217

Benchmark Comparison

🔴 Regressions detected. See tables below.

Baselines considered: 5 commits

Commit 1: 71b4f23
Commit 2: 32cd539
Commit 3: 1480102
Commit 4: 63721ad
Commit 5: 00e77b7

Thresholds: runtime ± 8%, compile ± 16%

Runtime FPS

status	env	batch_size	use_contact_island	gjk_collision	dtype	backend	constraint_solver	current FPS	baseline FPS [last (mean ± std)] (*1)	Δ FPS (*2)
✅	anymal_c	0	False	-	field	cpu	-	8,015	7,856 (7,922 ± 204)	+2.0%
✅	anymal_c	0	False	-	ndarray	cpu	-	4,025	3,993 (3,967 ± 37)	+0.8%
✅	anymal_c	30000	False	-	field	cuda	-	16,882,850	16,906,475 (16,887,201 ± 45,375)	-0.1%
✅	anymal_c	30000	False	-	ndarray	cuda	-	12,107,071	12,064,619 (12,078,602 ± 35,690)	+0.4%
✅	box_pyramid_3	4096	False	-	field	cuda	-	342,100	343,264 (342,769 ± 1,216)	-0.3%
✅	box_pyramid_3	4096	False	-	ndarray	cuda	-	307,510	302,378 (305,641 ± 2,228)	+1.7%
✅	box_pyramid_4	4096	False	-	field	cuda	-	110,476	110,473 (110,764 ± 454)	+0.0%
✅	box_pyramid_4	4096	False	-	ndarray	cuda	-	95,545	95,123 (95,053 ± 106)	+0.4%
✅	box_pyramid_5	4096	False	-	field	cuda	-	42,638	42,636 (42,497 ± 146)	+0.0%
✅	box_pyramid_5	4096	False	-	ndarray	cuda	-	30,782	30,926 (30,886 ± 46)	-0.5%
✅	box_pyramid_6	4096	False	False	field	cuda	-	9,030	9,055 (9,038 ± 11)	-0.3%
✅	box_pyramid_6	4096	False	False	ndarray	cuda	-	5,171	5,183 (5,182 ± 7)	-0.2%
✅	box_pyramid_6	4096	False	True	field	cuda	-	10,362	10,367 (10,382 ± 33)	-0.0%
✅	box_pyramid_6	4096	False	True	ndarray	cuda	-	6,513	6,460 (6,462 ± 12)	+0.8%
✅	duck_in_box_easy	30000	False	False	field	cuda	-	21,035,503	20,955,647 (20,992,237 ± 38,772)	+0.4%
✅	duck_in_box_easy	30000	False	False	ndarray	cuda	-	16,034,145	16,006,239 (15,971,473 ± 51,716)	+0.2%
✅	duck_in_box_easy	30000	False	True	field	cuda	-	8,997,995	9,003,943 (8,999,697 ± 13,417)	-0.1%
✅	duck_in_box_easy	30000	False	True	ndarray	cuda	-	8,637,292	8,608,955 (8,623,391 ± 10,374)	+0.3%
✅	duck_in_box_hard	0	False	-	field	cpu	-	5,713	5,434 (5,471 ± 87)	+5.1%
✅	duck_in_box_hard	0	False	-	ndarray	cpu	-	3,092	3,112 (3,043 ± 55)	-0.6%
✅	duck_in_box_hard	30000	False	False	field	cuda	-	3,091,161	3,075,602 (3,068,606 ± 17,840)	+0.5%
✅	duck_in_box_hard	30000	False	False	ndarray	cuda	-	2,941,464	2,924,738 (2,935,135 ± 21,205)	+0.6%
✅	duck_in_box_hard	30000	False	True	field	cuda	-	1,767,195	1,765,384 (1,765,676 ± 6,759)	+0.1%
✅	duck_in_box_hard	30000	False	True	ndarray	cuda	-	1,316,320	1,318,100 (1,316,254 ± 10,628)	-0.1%
✅	franka	0	False	-	field	cpu	-	6,844	6,826 (6,894 ± 46)	+0.3%
✅	franka	0	False	-	ndarray	cpu	-	3,590	3,556 (3,599 ± 37)	+1.0%
✅	franka	30000	False	False	field	cuda	-	18,524,296	18,524,229 (18,538,864 ± 42,449)	+0.0%
✅	franka	30000	False	False	ndarray	cuda	-	14,117,909	14,084,813 (14,098,259 ± 34,991)	+0.2%
✅	franka	30000	False	True	field	cuda	-	11,634,502	11,585,685 (11,661,706 ± 66,804)	+0.4%
✅	franka	30000	False	True	ndarray	cuda	-	8,203,875	8,145,970 (8,171,051 ± 32,355)	+0.7%
✅	franka	30000	False	-	field	cuda	CG	20,870,182	20,923,664 (20,890,163 ± 49,248)	-0.3%
✅	franka	30000	False	-	field	cuda	Newton	18,495,453	18,506,961 (18,527,282 ± 52,599)	-0.1%
✅	franka	30000	False	-	ndarray	cuda	CG	15,597,594	15,645,569 (15,633,712 ± 44,478)	-0.3%
✅	franka	30000	False	-	ndarray	cuda	Newton	14,125,642	13,990,761 (14,046,710 ± 38,190)	+1.0%
✅	franka_accessors	0	False	-	field	cpu	-	1,943	1,969 (1,965 ± 16)	-1.3%
✅	franka_accessors	0	False	-	ndarray	cpu	-	1,473	1,445 (1,456 ± 35)	+1.9%
✅	franka_accessors	30000	False	-	field	cuda	-	23,530,348	23,016,755 (22,956,176 ± 88,825)	+2.2%
✅	franka_accessors	30000	False	-	ndarray	cuda	-	16,167,438	15,831,693 (16,005,155 ± 153,776)	+2.1%
✅	franka_free	30000	False	-	field	cuda	-	38,313,412	38,227,988 (37,970,631 ± 236,246)	+0.2%
✅	franka_free	30000	False	-	ndarray	cuda	-	24,648,300	24,808,807 (24,688,603 ± 111,319)	-0.6%
✅	go2	4096	False	False	field	cuda	CG	3,124,652	3,125,794 (3,118,867 ± 13,140)	-0.0%
✅	go2	4096	False	False	field	cuda	Newton	3,317,131	3,305,551 (3,314,540 ± 10,656)	+0.4%
✅	go2	4096	False	False	ndarray	cuda	CG	2,158,281	2,133,989 (2,140,287 ± 4,400)	+1.1%
✅	go2	4096	False	False	ndarray	cuda	Newton	2,314,426	2,321,314 (2,315,684 ± 8,119)	-0.3%
✅	go2	4096	False	True	field	cuda	-	2,302,683	2,304,623 (2,306,981 ± 6,848)	-0.1%
✅	go2	4096	False	True	ndarray	cuda	-	1,584,260	1,583,709 (1,582,259 ± 2,747)	+0.0%
✅	random	30000	False	-	field	cuda	-	5,914,794	5,908,071 (5,911,053 ± 9,116)	+0.1%
✅	random	30000	False	-	ndarray	cuda	-	4,517,285	4,515,869 (4,513,329 ± 5,754)	+0.0%

Compile Time

status	env	batch_size	use_contact_island	gjk_collision	dtype	backend	constraint_solver	current compile	baseline compile [last (mean ± std)] (*1)	Δ compile (*2)
🔴	anymal_c	0	False	-	field	cpu	-	68.60	58.10 (58.27 ± 2.47)	+18.1%
✅	anymal_c	0	False	-	ndarray	cpu	-	33.80	35.30 (35.41 ± 1.64)	-4.3%
✅	anymal_c	30000	False	-	field	cuda	-	69.20	69.10 (69.92 ± 2.26)	+0.1%
✅	anymal_c	30000	False	-	ndarray	cuda	-	41.90	43.64 (43.24 ± 1.03)	-4.0%
✅	box_pyramid_3	4096	False	-	field	cuda	-	60.70	62.61 (64.40 ± 4.38)	-3.1%
✅	box_pyramid_3	4096	False	-	ndarray	cuda	-	37.40	37.63 (38.61 ± 0.98)	-0.6%
✅	box_pyramid_4	4096	False	-	field	cuda	-	61.90	68.14 (64.03 ± 3.24)	-9.2%
✅	box_pyramid_4	4096	False	-	ndarray	cuda	-	37	38 (38 ± 0)	-4.2%
✅	box_pyramid_5	4096	False	-	field	cuda	-	66.90	72.88 (70.34 ± 4.30)	-8.2%
✅	box_pyramid_5	4096	False	-	ndarray	cuda	-	36	36 (37 ± 0)	-0.9%
✅	box_pyramid_6	4096	False	False	field	cuda	-	64	60 (63 ± 5)	+5.5%
✅	box_pyramid_6	4096	False	False	ndarray	cuda	-	41.30	42.58 (41.35 ± 0.93)	-3.0%
✅	box_pyramid_6	4096	False	True	field	cuda	-	37.60	38.13 (39.12 ± 0.92)	-1.4%
✅	box_pyramid_6	4096	False	True	ndarray	cuda	-	27.30	27.60 (27.44 ± 0.33)	-1.1%
✅	duck_in_box_easy	30000	False	False	field	cuda	-	71	61 (64 ± 5)	+15.3%
✅	duck_in_box_easy	30000	False	False	ndarray	cuda	-	38.30	37.66 (37.87 ± 0.42)	+1.7%
✅	duck_in_box_easy	30000	False	True	field	cuda	-	39.70	38.69 (38.54 ± 0.57)	+2.6%
✅	duck_in_box_easy	30000	False	True	ndarray	cuda	-	25.20	25.12 (25.23 ± 0.54)	+0.3%
✅	duck_in_box_hard	0	False	-	field	cpu	-	52.60	58.38 (56.18 ± 2.71)	-9.9%
✅	duck_in_box_hard	0	False	-	ndarray	cpu	-	34.20	32.75 (32.44 ± 0.96)	+4.4%
✅	duck_in_box_hard	30000	False	False	field	cuda	-	60.80	65.07 (62.70 ± 1.37)	-6.6%
✅	duck_in_box_hard	30000	False	False	ndarray	cuda	-	36.30	36.10 (37.30 ± 1.12)	+0.5%
✅	duck_in_box_hard	30000	False	True	field	cuda	-	37.70	38.75 (37.57 ± 1.10)	-2.7%
✅	duck_in_box_hard	30000	False	True	ndarray	cuda	-	24.30	24.26 (23.95 ± 0.60)	+0.2%
✅	franka	0	False	-	field	cpu	-	59.10	51.80 (57.62 ± 4.03)	+14.1%
✅	franka	0	False	-	ndarray	cpu	-	31.70	31.67 (31.32 ± 0.56)	+0.1%
✅	franka	30000	False	False	field	cuda	-	63.30	64.01 (66.87 ± 5.53)	-1.1%
✅	franka	30000	False	False	ndarray	cuda	-	37.30	37.02 (37.04 ± 0.95)	+0.8%
✅	franka	30000	False	True	field	cuda	-	38.40	35.22 (36.54 ± 1.72)	+9.0%
✅	franka	30000	False	True	ndarray	cuda	-	25.30	25.67 (25.66 ± 0.53)	-1.4%
✅	franka	30000	False	-	field	cuda	CG	64	65 (64 ± 4)	-2.1%
🔴	franka	30000	False	-	field	cuda	Newton	75.20	64.29 (62.82 ± 1.96)	+17.0%
✅	franka	30000	False	-	ndarray	cuda	CG	43.30	40.67 (41.46 ± 0.96)	+6.5%
✅	franka	30000	False	-	ndarray	cuda	Newton	38.60	37.07 (37.61 ± 0.82)	+4.1%
✅	franka_accessors	0	False	-	field	cpu	-	61.90	60.24 (59.52 ± 5.11)	+2.8%
✅	franka_accessors	0	False	-	ndarray	cpu	-	31.60	32.16 (32.43 ± 0.86)	-1.8%
✅	franka_accessors	30000	False	-	field	cuda	-	57.90	61.76 (62.02 ± 2.40)	-6.3%
✅	franka_accessors	30000	False	-	ndarray	cuda	-	36.50	37.70 (37.13 ± 0.66)	-3.2%
✅	franka_free	30000	False	-	field	cuda	-	62.30	62.75 (62.07 ± 3.53)	-0.7%
✅	franka_free	30000	False	-	ndarray	cuda	-	38.40	37.22 (38.49 ± 0.87)	+3.2%
✅	go2	4096	False	False	field	cuda	CG	70.10	61.33 (63.02 ± 2.64)	+14.3%
✅	go2	4096	False	False	field	cuda	Newton	63.80	57.53 (62.86 ± 6.26)	+10.9%
✅	go2	4096	False	False	ndarray	cuda	CG	38.30	38.11 (37.84 ± 0.45)	+0.5%
✅	go2	4096	False	False	ndarray	cuda	Newton	38.50	38.30 (38.55 ± 1.30)	+0.5%
✅	go2	4096	False	True	field	cuda	-	41.60	42.51 (42.00 ± 1.09)	-2.1%
✅	go2	4096	False	True	ndarray	cuda	-	29.60	29.51 (30.13 ± 0.94)	+0.3%
✅	random	30000	False	-	field	cuda	-	67.70	75.89 (67.75 ± 6.26)	-10.8%
✅	random	30000	False	-	ndarray	cuda	-	38	38 (39 ± 0)	-1.2%

(*1) last: last commit on main, mean/std: stats over revs 5 commits if available.
(*2) Δ: relative difference between PR and last commit on main, i.e. (PR - main) / main * 100%.

Artifact: Download raw data

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[BUG FIX] Various minor bug fixes. #2217

Uh oh!

Uh oh!

[BUG FIX] Various minor bug fixes. #2217

Uh oh!

Benchmark Comparison

Details

Runtime FPS

Compile Time

Re-running jobs...