Skip to content

Commit

Permalink
Configure remote profiles for running benchmarks (#2168)
Browse files Browse the repository at this point in the history
* Add benchmark profiles for asp.net lab machines

* Update Readme
  • Loading branch information
habbes authored Sep 2, 2021
1 parent 6fa0e0e commit 1073d52
Show file tree
Hide file tree
Showing 2 changed files with 40 additions and 1 deletion.
26 changes: 26 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,32 @@ The easiest way to run the perf benchmarks is to use the [Microsoft.Crank](https
crank --config benchmarks.yml --scenario UriParser --profile local
```
#### Run benchmarks on remote dedicated agents
The `local` profile is provided for testing purposes, but it's not ideal for running benchmarks.
For more stable results and results that we can more reliably compare, the following profiles are also
available and should be preferred whenever possible:
Profile | Machine Architecture | OS
--------------|----------------------|------
`lab-windows` | INTEL, 12 Cores | Windows Server 2016
`lab-linux` | INTEL, 12 Cores | Ubuntu 18.04, Kernel 4.x
Use the `--profile` argument to specify the profile you want to use. For example,
to run the components benchmark on the Windows agent, run the following command:
```
crank --config benchmarks.yml --scenario Components --profile lab-windows
```
And to run on the Linux agent:
```
crank --config benchmarks.yml --scenario Components --profile lab-linux
```
PS: We should not use these machines to run automated scheduled benchmarks.
#### Run benchmarks against the official repo
To run benchmarks against the official repo instead of your local repo, pass
Expand Down
15 changes: 14 additions & 1 deletion benchmarks.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

variables:
# set to true to run benchmarks against master branch of main/official repo
base: false
Expand Down Expand Up @@ -80,3 +79,17 @@ profiles:
application:
endpoints:
- http://localhost:5010
# Remote lab machines for benchmarking
# These are also used by ASP.NET team
# for more info visit: https://github.com/aspnet/Benchmarks/tree/main/scenarios
# and https://github.com/aspnet/Benchmarks/blob/main/scenarios/aspnet.profiles.yml
lab-windows:
jobs:
application:
endpoints:
- http://asp-perf-win:5001
lab-linux:
jobs:
application:
endpoints:
- http://asp-perf-lin:5001

0 comments on commit 1073d52

Please sign in to comment.