Performance depends on number of 'setoption Threads value X' commands

[vondele]

Describe the issue

The following two invocations:

printf "setoption name Threads value 288\nsetoption name Threads value 288\nspeedtest 288 36864 15\nucinewgame\n" | ./stockfish.master

and

printf "setoption name Threads value 288\nspeedtest 288 36864 15\nucinewgame\n" | ./stockfish.master

give different performance. An even number of setoption commands results in good performance, and odd number in bad performance. The difference is most visible on multisocket numa systems:
e.g.

Nodes/second               : 228918174

vs

Nodes/second               : 266462375

It is not a recent issue, as it was also present in SF 17.1

Expected behavior

same performance independent on the number of calls

Steps to reproduce

printf "setoption name Threads value 288\nsetoption name Threads value 288\nsetoption name Threads value 288\nspeedtest 288 36864 15\nucinewgame\n" | ./stockfish.master

Anything else?

No response

Operating system

Linux

Stockfish version

master, SF17.1

[vondele]

Can be traced back to the original NUMA commit a169c78 using:

$ cat find_slow.sh 
nodes1=$(printf "setoption name Threads value 288\nsetoption name Hash value 28800\ngo movetime 15000\nucinewgame\n" | ./stockfish | grep -B1 bestmove | grep -o 'nps [0-9]*' | grep -o '[0-9]*')
nodes2=$(printf "setoption name Threads value 288\nsetoption name Threads value 288\nsetoption name Hash value 28800\ngo movetime 15000\nucinewgame\n" | ./stockfish | grep -B1 bestmove | grep -o 'nps [0-9]*' | grep -o '[0-9]*')
printf "fast: $nodes1 slow: $nodes2 relative: $(((nodes1-nodes2)*100/nodes2))%%\n"

prior to it performance is 2x slower and noisy.

git checkout a169c78b6d3b082068deb49a39aaa1fd75464c7f
fast: 211522178 slow: 196517920 relative: 7 %
fast: 214774594 slow: 199651148 relative: 7 %
fast: 213260099 slow: 195395588 relative: 9 %
fast: 213074170 slow: 202112870 relative: 5 %
fast: 210645791 slow: 202391817 relative: 4 %
fast: 213034647 slow: 201675134 relative: 5 %
fast: 213742616 slow: 196888324 relative: 8 %


git checkout a169c78b6d3b082068deb49a39aaa1fd75464c7f^
fast: 113893282 slow: 128717864 relative: -11 %
fast: 117964673 slow: 122997084 relative: -4 %
fast: 114715560 slow: 108618132 relative: 5 %
fast: 137593900 slow: 124249965 relative: 10 %
fast: 126324574 slow: 117619710 relative: 7 %
fast: 112090931 slow: 117966704 relative: -4 %
fast: 109571960 slow: 130712665 relative: -16 %
fast: 120494535 slow: 132602971 relative: -9 %
fast: 135151422 slow: 124188581 relative: 8 %