-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improved performance benchmark #168
Open
rafaelsc
wants to merge
17
commits into
pakrym:main
Choose a base branch
from
rafaelsc:betterPerformanceBenchmark
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
b64cc50
Add More Benchmarks
rafaelsc 8fadd9d
Update Packages
rafaelsc 1d9015d
Fix MixedBenchmark
rafaelsc 55fd777
Add ComplexBenchmark
rafaelsc 634140f
BugFix
rafaelsc d0bac8d
Add Deep Benchmarks
rafaelsc 757cf04
Rename Benchmarks
rafaelsc 5b74cd3
Add Startup Time Benchmarks
rafaelsc 9504593
Remove Old Benchmarks
rafaelsc 1d963cf
Update StartupBenchmark
rafaelsc cc8613d
Update Benchmark Results
rafaelsc b3f41c5
Fix Format
rafaelsc 2419476
Clean Up
rafaelsc 36a5d7a
Use ShortRunJob
rafaelsc a513881
Update benchmark results
rafaelsc 9f0ae7a
Merge
rafaelsc f6e4281
Update Results
rafaelsc File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
40 changes: 40 additions & 0 deletions
40
doc/benchmark/Jab.Performance.Basic.Complex.BasicComplexBenchmark-report-github.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
``` | ||
|
||
BenchmarkDotNet v0.13.10, Windows 10 (10.0.19045.3930/22H2/2022Update) | ||
AMD Ryzen 9 3900XT, 1 CPU, 24 logical and 12 physical cores | ||
.NET SDK 8.0.100 | ||
[Host] : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2 | ||
ShortRun : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2 | ||
|
||
Job=ShortRun IterationCount=3 LaunchCount=1 | ||
WarmupCount=3 | ||
|
||
``` | ||
| Method | NumbersOfCalls | NumbersOfClasses | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
|------- |--------------- |----------------- |------------:|--------------:|------------:|------:|--------:|--------:|-------:|----------:|------------:| | ||
| **Jab** | **1** | **1** | **279.6 ns** | **154.26 ns** | **8.46 ns** | **1.00** | **0.00** | **0.0563** | **-** | **472 B** | **1.00** | | ||
| MEDI | 1 | 1 | 149.2 ns | 20.02 ns | 1.10 ns | 0.53 | 0.02 | 0.0870 | 0.0002 | 728 B | 1.54 | | ||
| | | | | | | | | | | | | | ||
| **Jab** | **1** | **2** | **565.4 ns** | **80.38 ns** | **4.41 ns** | **1.00** | **0.00** | **0.1059** | **-** | **888 B** | **1.00** | | ||
| MEDI | 1 | 2 | 269.8 ns | 103.70 ns | 5.68 ns | 0.48 | 0.01 | 0.1364 | 0.0005 | 1144 B | 1.29 | | ||
| | | | | | | | | | | | | | ||
| **Jab** | **1** | **3** | **829.7 ns** | **353.08 ns** | **19.35 ns** | **1.00** | **0.00** | **0.1554** | **-** | **1304 B** | **1.00** | | ||
| MEDI | 1 | 3 | 384.2 ns | 25.62 ns | 1.40 ns | 0.46 | 0.01 | 0.1864 | 0.0010 | 1560 B | 1.20 | | ||
| | | | | | | | | | | | | | ||
| **Jab** | **10** | **1** | **2,846.8 ns** | **1,643.85 ns** | **90.10 ns** | **1.00** | **0.00** | **0.5608** | **-** | **4720 B** | **1.00** | | ||
| MEDI | 10 | 1 | 1,419.2 ns | 93.91 ns | 5.15 ns | 0.50 | 0.02 | 0.8698 | 0.0019 | 7280 B | 1.54 | | ||
| | | | | | | | | | | | | | ||
| **Jab** | **10** | **2** | **5,674.5 ns** | **1,663.99 ns** | **91.21 ns** | **1.00** | **0.00** | **1.0605** | **-** | **8880 B** | **1.00** | | ||
| MEDI | 10 | 2 | 2,742.9 ns | 1,163.85 ns | 63.79 ns | 0.48 | 0.02 | 1.3657 | 0.0038 | 11440 B | 1.29 | | ||
| | | | | | | | | | | | | | ||
| **Jab** | **10** | **3** | **7,961.3 ns** | **4,443.31 ns** | **243.55 ns** | **1.00** | **0.00** | **1.5564** | **-** | **13040 B** | **1.00** | | ||
| MEDI | 10 | 3 | 3,895.1 ns | 1,591.50 ns | 87.24 ns | 0.49 | 0.02 | 1.8616 | 0.0076 | 15600 B | 1.20 | | ||
| | | | | | | | | | | | | | ||
| **Jab** | **100** | **1** | **29,641.7 ns** | **30,778.23 ns** | **1,687.06 ns** | **1.00** | **0.00** | **5.6152** | **-** | **47200 B** | **1.00** | | ||
| MEDI | 100 | 1 | 14,530.2 ns | 7,006.36 ns | 384.04 ns | 0.49 | 0.01 | 8.6975 | 0.0153 | 72800 B | 1.54 | | ||
| | | | | | | | | | | | | | ||
| **Jab** | **100** | **2** | **54,178.3 ns** | **22,959.21 ns** | **1,258.47 ns** | **1.00** | **0.00** | **10.5591** | **-** | **88800 B** | **1.00** | | ||
| MEDI | 100 | 2 | 26,824.2 ns | 4,858.13 ns | 266.29 ns | 0.50 | 0.01 | 13.6719 | 0.0305 | 114400 B | 1.29 | | ||
| | | | | | | | | | | | | | ||
| **Jab** | **100** | **3** | **92,719.6 ns** | **111,920.74 ns** | **6,134.75 ns** | **1.00** | **0.00** | **15.5029** | **-** | **130400 B** | **1.00** | | ||
| MEDI | 100 | 3 | 39,650.9 ns | 26,436.59 ns | 1,449.08 ns | 0.43 | 0.01 | 18.6157 | 0.0610 | 156000 B | 1.20 | |
40 changes: 40 additions & 0 deletions
40
doc/benchmark/Jab.Performance.Basic.Mixed.BasicMixedBenchmark-report-github.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
``` | ||
|
||
BenchmarkDotNet v0.13.10, Windows 10 (10.0.19045.3930/22H2/2022Update) | ||
AMD Ryzen 9 3900XT, 1 CPU, 24 logical and 12 physical cores | ||
.NET SDK 8.0.100 | ||
[Host] : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2 | ||
ShortRun : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2 | ||
|
||
Job=ShortRun IterationCount=3 LaunchCount=1 | ||
WarmupCount=3 | ||
|
||
``` | ||
| Method | NumbersOfCalls | NumbersOfClasses | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio | | ||
|------- |--------------- |----------------- |-------------:|-------------:|-----------:|------:|--------:|-------:|----------:|------------:| | ||
| **Jab** | **1** | **1** | **25.09 ns** | **2.023 ns** | **0.111 ns** | **1.00** | **0.00** | **0.0105** | **88 B** | **1.00** | | ||
| MEDI | 1 | 1 | 52.97 ns | 5.454 ns | 0.299 ns | 2.11 | 0.02 | 0.0220 | 184 B | 2.09 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **1** | **2** | **48.33 ns** | **22.718 ns** | **1.245 ns** | **1.00** | **0.00** | **0.0172** | **144 B** | **1.00** | | ||
| MEDI | 1 | 2 | 71.94 ns | 23.674 ns | 1.298 ns | 1.49 | 0.02 | 0.0286 | 240 B | 1.67 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **1** | **3** | **66.67 ns** | **37.560 ns** | **2.059 ns** | **1.00** | **0.00** | **0.0238** | **200 B** | **1.00** | | ||
| MEDI | 1 | 3 | 111.83 ns | 108.495 ns | 5.947 ns | 1.68 | 0.04 | 0.0353 | 296 B | 1.48 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **10** | **1** | **259.18 ns** | **130.556 ns** | **7.156 ns** | **1.00** | **0.00** | **0.1049** | **880 B** | **1.00** | | ||
| MEDI | 10 | 1 | 533.29 ns | 295.258 ns | 16.184 ns | 2.06 | 0.01 | 0.2193 | 1840 B | 2.09 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **10** | **2** | **453.72 ns** | **208.118 ns** | **11.408 ns** | **1.00** | **0.00** | **0.1717** | **1440 B** | **1.00** | | ||
| MEDI | 10 | 2 | 715.44 ns | 228.174 ns | 12.507 ns | 1.58 | 0.01 | 0.2861 | 2400 B | 1.67 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **10** | **3** | **664.24 ns** | **323.259 ns** | **17.719 ns** | **1.00** | **0.00** | **0.2384** | **2000 B** | **1.00** | | ||
| MEDI | 10 | 3 | 1,060.66 ns | 691.051 ns | 37.879 ns | 1.60 | 0.10 | 0.3529 | 2960 B | 1.48 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **100** | **1** | **2,587.41 ns** | **1,476.947 ns** | **80.956 ns** | **1.00** | **0.00** | **1.0490** | **8800 B** | **1.00** | | ||
| MEDI | 100 | 1 | 5,068.08 ns | 395.160 ns | 21.660 ns | 1.96 | 0.06 | 2.1973 | 18400 B | 2.09 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **100** | **2** | **4,591.13 ns** | **2,233.109 ns** | **122.404 ns** | **1.00** | **0.00** | **1.7166** | **14400 B** | **1.00** | | ||
| MEDI | 100 | 2 | 7,234.70 ns | 1,964.384 ns | 107.675 ns | 1.58 | 0.06 | 2.8687 | 24000 B | 1.67 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **100** | **3** | **6,837.38 ns** | **2,789.230 ns** | **152.887 ns** | **1.00** | **0.00** | **2.3880** | **20000 B** | **1.00** | | ||
| MEDI | 100 | 3 | 11,699.66 ns | 1,556.090 ns | 85.295 ns | 1.71 | 0.04 | 3.5248 | 29600 B | 1.48 | |
40 changes: 40 additions & 0 deletions
40
doc/benchmark/Jab.Performance.Basic.Scoped.BasicScopedBenchmark-report-github.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
``` | ||
|
||
BenchmarkDotNet v0.13.10, Windows 10 (10.0.19045.3930/22H2/2022Update) | ||
AMD Ryzen 9 3900XT, 1 CPU, 24 logical and 12 physical cores | ||
.NET SDK 8.0.100 | ||
[Host] : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2 | ||
ShortRun : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2 | ||
|
||
Job=ShortRun IterationCount=3 LaunchCount=1 | ||
WarmupCount=3 | ||
|
||
``` | ||
| Method | NumbersOfCalls | NumbersOfClasses | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio | | ||
|------- |--------------- |----------------- |-------------:|------------:|-----------:|------:|--------:|-------:|----------:|------------:| | ||
| **Jab** | **1** | **1** | **23.67 ns** | **19.60 ns** | **1.074 ns** | **1.00** | **0.00** | **0.0095** | **80 B** | **1.00** | | ||
| MEDI | 1 | 1 | 101.19 ns | 73.78 ns | 4.044 ns | 4.28 | 0.02 | 0.0401 | 336 B | 4.20 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **1** | **2** | **34.37 ns** | **21.20 ns** | **1.162 ns** | **1.00** | **0.00** | **0.0124** | **104 B** | **1.00** | | ||
| MEDI | 1 | 2 | 152.44 ns | 48.25 ns | 2.645 ns | 4.44 | 0.08 | 0.0429 | 360 B | 3.46 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **1** | **3** | **46.67 ns** | **19.28 ns** | **1.057 ns** | **1.00** | **0.00** | **0.0153** | **128 B** | **1.00** | | ||
| MEDI | 1 | 3 | 217.12 ns | 207.52 ns | 11.375 ns | 4.66 | 0.35 | 0.0458 | 384 B | 3.00 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **10** | **1** | **213.94 ns** | **34.73 ns** | **1.904 ns** | **1.00** | **0.00** | **0.0956** | **800 B** | **1.00** | | ||
| MEDI | 10 | 1 | 836.62 ns | 30.44 ns | 1.669 ns | 3.91 | 0.04 | 0.4015 | 3360 B | 4.20 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **10** | **2** | **327.55 ns** | **154.36 ns** | **8.461 ns** | **1.00** | **0.00** | **0.1240** | **1040 B** | **1.00** | | ||
| MEDI | 10 | 2 | 1,429.73 ns | 1,090.76 ns | 59.788 ns | 4.36 | 0.12 | 0.4292 | 3600 B | 3.46 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **10** | **3** | **471.03 ns** | **225.45 ns** | **12.358 ns** | **1.00** | **0.00** | **0.1526** | **1280 B** | **1.00** | | ||
| MEDI | 10 | 3 | 2,067.34 ns | 244.00 ns | 13.374 ns | 4.39 | 0.14 | 0.4578 | 3840 B | 3.00 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **100** | **1** | **2,087.62 ns** | **1,052.17 ns** | **57.673 ns** | **1.00** | **0.00** | **0.9537** | **8000 B** | **1.00** | | ||
| MEDI | 100 | 1 | 8,277.90 ns | 214.87 ns | 11.778 ns | 3.97 | 0.11 | 4.0131 | 33600 B | 4.20 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **100** | **2** | **3,275.71 ns** | **2,345.95 ns** | **128.590 ns** | **1.00** | **0.00** | **1.2398** | **10400 B** | **1.00** | | ||
| MEDI | 100 | 2 | 13,711.81 ns | 3,013.26 ns | 165.167 ns | 4.19 | 0.17 | 4.3030 | 36000 B | 3.46 | | ||
| | | | | | | | | | | | | ||
| **Jab** | **100** | **3** | **4,289.60 ns** | **1,056.13 ns** | **57.890 ns** | **1.00** | **0.00** | **1.5259** | **12800 B** | **1.00** | | ||
| MEDI | 100 | 3 | 21,340.73 ns | 657.08 ns | 36.017 ns | 4.98 | 0.07 | 4.5776 | 38400 B | 3.00 | |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is quite interesting. Do you have an idea why complex resolution is so much slower in Jab?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried to find it and didn't understand why yet.
Some could be explained as the
lock {}
, but I removed it and didn't see a big improvement. See #169There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I managed to beat the MEDI performance for this case.
locks
and usingLazyInitializer.EnsureInitialized
TryAddDisposable()
- (this could be done in compilation time instead of runtime)this.GetService<T>()
for((IServiceProvider<T>)this).GetService()