Stockfish needlessly hangs the queen

[robertnurnberg]

Describe the issue

Stockfish playing as black hangs the queen in the position r5k1/1p2qpb1/2p1bnp1/r3p2p/7Q/P1NPPB1P/1BP3P1/1R3R1K b - - by suggesting as bestmove f6e8

Expected behavior

Stockfish should suggest any other available move, that does not blunder the position in a single move.

Steps to reproduce

These steps deterministically lead to the described issue. They only work for sf revision 0716b84

position fen r1bq1rk1/pppn1pbp/3p1np1/4P3/1P6/P3PN2/1BPP2PP/RN1QKB1R b KQ - 0 8
go nodes 110000
position fen r1bq1rk1/ppp2pbp/3p1np1/4N3/1P6/P3P3/1BPP2PP/RN1QKB1R b KQ - 0 9
go nodes 110000
position fen r1bq1rk1/ppp2pbp/5np1/4p3/1P6/P3P3/1BPPB1PP/RN1QK2R b KQ - 1 10
go nodes 110000
position fen r2q1rk1/ppp2pbp/5np1/4pb2/1P6/P3P3/1BPPB1PP/RN1Q1RK1 b - - 3 11
go nodes 110000
position fen r4rk1/ppp2pbp/3q1np1/4pb2/1P6/P1N1P3/1BPPB1PP/R2Q1RK1 b - - 5 12
go nodes 110000
position fen r4rk1/ppp2pb1/3q1np1/4pb1p/1P6/P1NPP3/1BP1B1PP/R2Q1RK1 b - - 0 13
go nodes 110000
position fen r4rk1/1pp2pb1/3q1np1/P3pb1p/8/P1NPP3/1BP1B1PP/R2Q1RK1 b - - 0 14
go nodes 110000
position fen 5rk1/1pp2pb1/3q1np1/r3pb1p/8/P1NPP3/1BP1B1PP/R3QRK1 b - - 1 15
go nodes 110000
position fen r5k1/1pp2pb1/3q1np1/r3pb1p/8/P1NPP3/1BP1B1PP/1R2QRK1 b - - 3 16
go nodes 110000
position fen r5k1/1p3pb1/2pq1np1/r3pb1p/8/P1NPP3/1BP1B1PP/1R2QR1K b - - 1 17
go nodes 110000
position fen r5k1/1p3pb1/2p2np1/r1q1pb1p/8/P1NPPB2/1BP3PP/1R2QR1K b - - 3 18
go nodes 110000
position fen r5k1/1p3pb1/2p1bnp1/r1q1p2p/8/P1NPPB1P/1BP3P1/1R2QR1K b - - 0 19
go nodes 110000
position fen r5k1/1p2qpb1/2p1bnp1/r3p2p/7Q/P1NPPB1P/1BP3P1/1R3R1K b - - 2 20
go nodes 400000

Anything else?

Losing games by hanging the queen in games on fishtest has been anecdotally reported by @dav1312 . The steps to reproduce this example blunder were initiated by his post on discord. See also here for a script to find the necessary commands to reproduce the "bug".

Operating system

All

Stockfish version

0716b84

[robertnurnberg]

For completeness, the output to the final go nodes command is

info depth 1 seldepth 8 multipv 1 score cp 81 nodes 78 nps 382 hashfull 0 tbhits 0 time 204 pv f6e8
info depth 2 seldepth 3 multipv 1 score cp 81 nodes 130 nps 637 hashfull 0 tbhits 0 time 204 pv f6e8
info depth 3 seldepth 3 multipv 1 score cp 81 nodes 180 nps 882 hashfull 0 tbhits 0 time 204 pv f6e8
info depth 4 seldepth 3 multipv 1 score cp 81 nodes 231 nps 1132 hashfull 0 tbhits 0 time 204 pv f6e8
info depth 5 seldepth 4 multipv 1 score cp 84 nodes 286 nps 1401 hashfull 0 tbhits 0 time 204 pv f6e8 h4e1
info depth 6 seldepth 5 multipv 1 score cp 80 nodes 466 nps 2284 hashfull 0 tbhits 0 time 204 pv f6e8 h4e1 f7f5 e1g3
info depth 7 seldepth 10 multipv 1 score cp 78 nodes 768 nps 3764 hashfull 0 tbhits 0 time 204 pv f6e8 h4e1 f7f5 c3e2 e8d6
info depth 8 seldepth 12 multipv 1 score cp 69 nodes 3066 nps 14669 hashfull 1 tbhits 0 time 209 pv f6e8 h4e1 f7f5 a3a4 a5a7 b2c1
info depth 9 seldepth 13 multipv 1 score cp 69 nodes 6902 nps 32102 hashfull 2 tbhits 0 time 215 pv e7c5 h4b4 f6d5 b4c5 a5c5
info depth 10 seldepth 15 multipv 1 score cp 69 nodes 9301 nps 42277 hashfull 4 tbhits 0 time 220 pv f6e8 h4b4 e7c7 c3e4 a5b5
info depth 11 seldepth 16 multipv 1 score cp 71 nodes 15718 nps 67750 hashfull 7 tbhits 0 time 232 pv f6e8 h4e1 f7f5 a3a4 g7f6 e1c1 e8d6
info depth 12 seldepth 11 multipv 1 score cp 72 nodes 18540 nps 78227 hashfull 8 tbhits 0 time 237 pv f6e8 h4e1 f7f5 a3a4 g7f6 e1c1 e8d6 b2a3 e5e4
info depth 13 seldepth 17 multipv 1 score cp 73 nodes 31013 nps 117030 hashfull 11 tbhits 0 time 265 pv f6e8 h4e1 f7f5 a3a4 g8h7 b2c1 a5a7 e3e4 f5f4 c1d2
info depth 14 seldepth 20 multipv 1 score cp 77 nodes 50784 nps 165420 hashfull 18 tbhits 0 time 307 pv f6e8 h4e1 f7f5 a3a4 g8h7 b2c1 a5a7 c1d2 e8d6 a4a5 e5e4 d3e4 e6c4 e4f5 c4f1
info depth 15 seldepth 20 multipv 1 score cp 80 nodes 105665 nps 257719 hashfull 40 tbhits 0 time 410 pv f6e8 h4e1 e8d6 a3a4 d6f5 b2c1 h5h4 h1g1 f5g3 f1f2 f7f5 c1d2 a5a7 e3e4 f5f4
info depth 16 seldepth 26 multipv 1 score cp 71 nodes 235967 nps 371017 hashfull 99 tbhits 0 time 636 pv f6e8 h4e1 e8d6 b2c1 f7f5 e3e4 f5f4 a3a4 d6f7 c1d2 e7c7 e1h4 c7c8 h4e1 a8a6
info depth 17 seldepth 25 multipv 1 score cp 74 nodes 343756 nps 418704 hashfull 148 tbhits 0 time 821 pv f6e8 h4e1 e8d6 e3e4 b7b5 e1g3 d6b7 b2c1 b7c5 b1a1 a5a6
info depth 18 seldepth 25 multipv 1 score cp 74 nodes 400287 nps 434150 hashfull 176 tbhits 0 time 922 pv f6e8 h4e1 e8d6 e3e4 b7b5 e1g3 d6b7 b2c1 b7c5 b1a1 a5a6
bestmove f6e8 ponder h4e1

[Sopel97]

This appears to be a hash collision issue. I can't replicate with 64-bit tt key (same number of tt entries)

[Craftyawesome]

Did you use the script, or just the one reproduction? If the latter isn't it possible that the collision is responsible for the divergence but not directly the bad move?

[Sopel97]

not sure what you mean by "script"? I followed the steps to reproduce

[Craftyawesome]

He made a script to search for a reproduction.

See also here for a script to find the necessary commands to reproduce the "bug".

[Sopel97]

ok, I'll try to investigate further tomorrow

[robertnurnberg]

There is no guarantee for the script to find the right parameters (needed to trigger the bug) in finite time....

I was a bit lucky the first time round, and could use the knowledge at roughly what nodes count the move was produced during the LTC fishtest game.

I believe the best course of action would be do definitely pin down the cause, and then try to find a remedy.

[robertnurnberg]

Btw, the issue still triggers with setoption name hash value 147. Running with that option should mean that we reduce the chances of general hash collisions, and so it might be easier to find the one responsible for the blunder (if indeed a hash collision is to blame).

Edit: Also the final command can be shortened to go nodes 1 to still trigger the blunder as best move.

[peregrineshahin]

I figured a smaller reproduction and there is only 6 collided positions, from the first glance it looks they are irrelevant
smaller repro for sf revision 0716b84

position fen r1bq1rk1/pppn1pbp/3p1np1/4P3/1P6/P3PN2/1BPP2PP/RN1QKB1R b KQ - 0 8
go nodes 109876
position fen r4rk1/ppp2pbp/3q1np1/4pb2/1P6/P1N1P3/1BPPB1PP/R2Q1RK1 b - - 5 12
go nodes 65986
position fen r4rk1/ppp2pb1/3q1np1/4pb1p/1P6/P1NPP3/1BP1B1PP/R2Q1RK1 b - - 0 13
go nodes 21611
position fen r5k1/1p2qpb1/2p1bnp1/r3p2p/7Q/P1NPPB1P/1BP3P1/1R3R1K b - - 2 20
go nodes 1

ucinewgame

results

posKey: 156379583
FENs:
Number of FENS: 2
 r1b1nrk1/pppn1pbp/8/6p1/1P1Np2q/P1N1P1P1/1BPPB2P/R2QK2R w KQ - 0 13
 4r1k1/1pp2pb1/rq3np1/4pb1p/4P3/P1NP4/1BP1B1PP/R3QR1K w - - 3 18
TT Moves: (none)
          (none)
          (none)
          (none)
          (none)


posKey: 1476590525
FENs:
Number of FENS: 2
 r2q1rk1/1pp2pbp/5np1/p3p3/1P6/P1N1P3/1BbPB1PP/R2Q1RK1 w - - 0 13
 4r1k1/2p2p2/1p3npb/r1q2b1p/4B3/P1NPP2P/2P3P1/1RB1QRK1 b - - 0 21
TT Moves: (none)
          (none)


posKey: -391743011
FENs:
Number of FENS: 2
 r1b2rk1/1ppnqpb1/5n2/p5pp/1P1Np3/P1N1P3/1BPPB1PP/R2Q1R1K w - - 0 14
 r4rk1/pppn1pbp/2q3p1/4p3/1P6/P1NPP3/1B1P2PP/R2Q1RK1 w - - 1 15
TT Moves: (none)
          (none)


posKey: 424872270
FENs:
Number of FENS: 2
 5r2/ppp1qpkp/6P1/8/1P6/P3P3/2P1B1PP/R2r1RK1 b - - 0 18
 3r1rk1/ppp2p2/3q2p1/7p/1P2P3/P3P3/2P2KPb/1R1BQR2 b - - 1 20
TT Moves: (none)
          (none)


posKey: -457919305
FENs:
Number of FENS: 2
 3rr1k1/ppp2pb1/3q2p1/3n1b1p/1P1Pp3/P1N1P3/1BP1B1PP/R3QR1K w - - 4 17
 3r1rk1/ppp2pbp/5np1/4pb2/1P2P3/P1N5/1BP1B1PP/2R2RK1 b - - 0 15
TT Moves: (none)
          (none)


posKey: -1351134240
FENs:
Number of FENS: 2
 6k1/1p3p2/2p1b1pb/2Bnp2p/r3N3/3PPB1P/2P3P1/6RK b - - 3 26
 r5k1/5pb1/4bnp1/rp1pQ2p/8/P2PqB1P/1BP3P1/1R3R1K b - - 0 22
TT Moves: (none)
          (none)

[peregrineshahin]

After thorough analysis, I have found the culprit which is this position (the position after the punishment of the blunder) of r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b
is never evaluated and is never hit in search, the only hits are at qsearch and we immediately cutoff with standpating based on a TT entry whereas we never saved this position ever into TT.

Analysis Code

$ git diff 0716b845fdef8a20102b07eaec074b8da8162523..ea20ac70a4b1b10a99d842570be4b89e2bb72b0a
diff --git a/src/evaluate.cpp b/src/evaluate.cpp
index bc705b85..d76e0e4d 100644
--- a/src/evaluate.cpp
+++ b/src/evaluate.cpp
@@ -85,6 +85,14 @@ Value Eval::evaluate(const Eval::NNUE::Networks& networks, const Position& pos,
     // Guarantee evaluation does not hit the tablebase range
     v = std::clamp(v, VALUE_TB_LOSS_IN_MAX_PLY + 1, VALUE_TB_WIN_IN_MAX_PLY - 1);

+
+    /// ever evaluated
+    if (pos.fen().find("r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b")
+        != std::string::npos)
+    {
+        assert(false);
+    }
+
     return v;
 }

diff --git a/src/input.txt b/src/input.txt
new file mode 100644
index 00000000..608d4629
--- /dev/null
+++ b/src/input.txt
@@ -0,0 +1,10 @@
+position fen r1bq1rk1/pppn1pbp/3p1np1/4P3/1P6/P3PN2/1BPP2PP/RN1QKB1R b KQ - 0 8
+go nodes 109876
+position fen r4rk1/ppp2pbp/3q1np1/4pb2/1P6/P1N1P3/1BPPB1PP/R2Q1RK1 b - - 5 12
+go nodes 65986
+position fen r4rk1/ppp2pb1/3q1np1/4pb1p/1P6/P1NPP3/1BP1B1PP/R2Q1RK1 b - - 0 13
+go nodes 21611
+position fen r5k1/1p2qpb1/2p1bnp1/r3p2p/7Q/P1NPPB1P/1BP3P1/1R3R1K b - - 2 20
+go nodes 1
+
+ucinewgame
\ No newline at end of file
diff --git a/src/search.cpp b/src/search.cpp
index 3f882aab..babdcd01 100644
--- a/src/search.cpp
+++ b/src/search.cpp
@@ -46,6 +46,11 @@

 namespace Stockfish {

+int globalSearch      = 0;
+int globalQsearch     = 0;
+int positionEverSaved = 0;
+int everEvalIsSkipped = 0;
+
 namespace TB = Tablebases;

 using Eval::evaluate;
@@ -204,6 +209,13 @@ void Search::Worker::start_searching() {
         sync_cout << main_manager()->pv(*bestThread, threads, tt, bestThread->completedDepth)
                   << sync_endl;

+    std::cerr << std::endl;
+    std::cerr << "everEvalIsSkipped: " << everEvalIsSkipped << std::endl;
+    std::cerr << "positionEverSaved: " << positionEverSaved << std::endl;
+    std::cerr << "globalSearch: " << globalSearch << std::endl;
+    std::cerr << "globalQsearch: " << globalQsearch << std::endl;
+    std::cerr << std::endl << std::endl;
+
     sync_cout << "bestmove " << UCI::move(bestThread->rootMoves[0].pv[0], rootPos.is_chess960());

     if (bestThread->rootMoves[0].pv.size() > 1
@@ -510,6 +522,10 @@ template<NodeType nodeType>
 Value Search::Worker::search(
   Position& pos, Stack* ss, Value alpha, Value beta, Depth depth, bool cutNode) {

+    if (pos.fen().find("r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b")
+        != std::string::npos)
+        globalSearch++;
+
     constexpr bool PvNode   = nodeType != NonPV;
     constexpr bool rootNode = nodeType == Root;

@@ -597,7 +613,13 @@ Value Search::Worker::search(
     // Step 4. Transposition table lookup.
     excludedMove = ss->excludedMove;
     posKey       = pos.key();
-    tte          = tt.probe(posKey, ss->ttHit);
+
+    if (posKey == 11258407522955116289)
+    {
+        std::cerr << pos.fen() << std::endl;
+    }
+
+    tte       = tt.probe(posKey, ss->ttHit);
     ttValue   = ss->ttHit ? value_from_tt(tte->value(), ss->ply, pos.rule50_count()) : VALUE_NONE;
     ttMove    = rootNode  ? thisThread->rootMoves[thisThread->pvIdx].pv[0]
               : ss->ttHit ? tte->move()
@@ -671,6 +693,12 @@ Value Search::Worker::search(

                 if (b == BOUND_EXACT || (b == BOUND_LOWER ? value >= beta : value <= alpha))
                 {
+                    if (pos.fen().find("r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b")
+                        != std::string::npos)
+                    {
+                        positionEverSaved++;
+                    }
+
                     tte->save(posKey, value_to_tt(value, ss->ply), ss->ttPv, b,
                               std::min(MAX_PLY - 1, depth + 6), Move::none(), VALUE_NONE,
                               tt.generation());
@@ -725,6 +753,12 @@ Value Search::Worker::search(
         unadjustedStaticEval = evaluate(networks, pos, thisThread->optimism[us]);
         ss->staticEval = eval = to_corrected_static_eval(unadjustedStaticEval, *thisThread, pos);

+        if (pos.fen().find("r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b")
+            != std::string::npos)
+        {
+            positionEverSaved++;
+        }
+
         // Static evaluation is saved as it was before adjustment by correction history
         tte->save(posKey, VALUE_NONE, ss->ttPv, BOUND_NONE, DEPTH_NONE, Move::none(),
                   unadjustedStaticEval, tt.generation());
@@ -871,6 +905,11 @@ Value Search::Worker::search(

                 if (value >= probCutBeta)
                 {
+                    if (pos.fen().find("r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b")
+                        != std::string::npos)
+                    {
+                        positionEverSaved++;
+                    }
                     // Save ProbCut data into transposition table
                     tte->save(posKey, value_to_tt(value, ss->ply), ss->ttPv, BOUND_LOWER, depth - 3,
                               move, unadjustedStaticEval, tt.generation());
@@ -1341,11 +1380,20 @@ moves_loop:  // When in check, search starts here
     // Write gathered information in transposition table
     // Static evaluation is saved as it was before correction history
     if (!excludedMove && !(rootNode && thisThread->pvIdx))
+    {
+
+        if (pos.fen().find("r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b")
+            != std::string::npos)
+        {
+            positionEverSaved++;
+        }
+
         tte->save(posKey, value_to_tt(bestValue, ss->ply), ss->ttPv,
                   bestValue >= beta    ? BOUND_LOWER
                   : PvNode && bestMove ? BOUND_EXACT
                                        : BOUND_UPPER,
                   depth, bestMove, unadjustedStaticEval, tt.generation());
+    }

     // Adjust correction history
     if (!ss->inCheck && (!bestMove || !pos.capture(bestMove))
@@ -1372,6 +1420,11 @@ Value Search::Worker::qsearch(Position& pos, Stack* ss, Value alpha, Value beta,
     static_assert(nodeType != Root);
     constexpr bool PvNode = nodeType == PV;

+
+    if (pos.fen().find("r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b")
+        != std::string::npos)
+        globalQsearch++;
+
     assert(alpha >= -VALUE_INFINITE && alpha < beta && beta <= VALUE_INFINITE);
     assert(PvNode || (alpha == beta - 1));
     assert(depth <= 0);
@@ -1432,6 +1485,11 @@ Value Search::Worker::qsearch(Position& pos, Stack* ss, Value alpha, Value beta,
     ttMove  = ss->ttHit ? tte->move() : Move::none();
     pvHit   = ss->ttHit && tte->is_pv();

+    if (posKey == 11258407522955116289)
+    {
+        std::cerr << std::endl << pos.fen() << std::endl;
+    }
+
     // At non-PV nodes we check for an early TT cutoff
     if (!PvNode && tte->depth() >= ttDepth
         && ttValue != VALUE_NONE  // Only in case of TT access race or if !ttHit
@@ -1457,6 +1515,15 @@ Value Search::Worker::qsearch(Position& pos, Stack* ss, Value alpha, Value beta,
             if (ttValue != VALUE_NONE
                 && (tte->bound() & (ttValue > bestValue ? BOUND_LOWER : BOUND_UPPER)))
                 bestValue = ttValue;
+
+
+            if (pos.fen().find("r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b")
+                != std::string::npos)
+            {
+                std::cerr << std::endl
+                          << "bestValueOfQueenBlunder: " << bestValue << std::endl
+                          << std::endl;
+            }
         }
         else
         {
@@ -1466,11 +1533,18 @@ Value Search::Worker::qsearch(Position& pos, Stack* ss, Value alpha, Value beta,
                                    : -(ss - 1)->staticEval;
             ss->staticEval       = bestValue =
               to_corrected_static_eval(unadjustedStaticEval, *thisThread, pos);
+
+            if (pos.fen().find("r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b")
+                != std::string::npos)
+            {
+                everEvalIsSkipped++;
+            }
         }

         // Stand pat. Return immediately if static value is at least beta
         if (bestValue >= beta)
         {
+            // no check for saving here as the position is never evaluated
             if (!ss->ttHit)
                 tte->save(posKey, value_to_tt(bestValue, ss->ply), false, BOUND_LOWER, DEPTH_NONE,
                           Move::none(), unadjustedStaticEval, tt.generation());
@@ -1618,6 +1692,12 @@ Value Search::Worker::qsearch(Position& pos, Stack* ss, Value alpha, Value beta,

     // Save gathered info in transposition table
     // Static evaluation is saved as it was before adjustment by correction history
+    if (pos.fen().find("r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b")
+        != std::string::npos)
+    {
+        positionEverSaved++;
+    }
+
     tte->save(posKey, value_to_tt(bestValue, ss->ply), pvHit,
               bestValue >= beta ? BOUND_LOWER : BOUND_UPPER, ttDepth, bestMove,
               unadjustedStaticEval, tt.generation());

Analysis Output

$ more input.txt | ./stockfish.exe
Stockfish dev-20240415-51e383e9 by the Stockfish developers (see AUTHORS file)
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue

r1bq1rk1/ppp2pbp/5np1/3p4/1P6/P3PN2/1BPP2PP/RN1QKB1R w KQ - 0 10
info depth 1 seldepth 5 multipv 1 score cp 56 nodes 44 nps 11000 hashfull 0 tbhits 0 time 4 pv d7e5
info depth 2 seldepth 3 multipv 1 score cp 101 nodes 81 nps 16200 hashfull 0 tbhits 0 time 5 pv d6e5
info depth 3 seldepth 3 multipv 1 score cp 101 nodes 113 nps 22600 hashfull 0 tbhits 0 time 5 pv d6e5
info depth 4 seldepth 3 multipv 1 score cp 101 nodes 142 nps 23666 hashfull 0 tbhits 0 time 6 pv d6e5
info depth 5 seldepth 5 multipv 1 score cp 101 nodes 173 nps 28833 hashfull 0 tbhits 0 time 6 pv d6e5 f3e5
info depth 6 seldepth 5 multipv 1 score cp 110 nodes 229 nps 32714 hashfull 0 tbhits 0 time 7 pv d6e5 f3e5 f8e8
info depth 7 seldepth 5 multipv 1 score cp 137 nodes 309 nps 38625 hashfull 0 tbhits 0 time 8 pv d6e5 f3e5 f8e8
info depth 8 seldepth 7 multipv 1 score cp 181 nodes 563 nps 46916 hashfull 0 tbhits 0 time 12 pv d6e5 f3e5 f6e4
info depth 9 seldepth 10 multipv 1 score cp 164 nodes 1659 nps 61444 hashfull 0 tbhits 0 time 27 pv d6e5 d2d3 a7a5 e3e4 a5b4 a3b4 a8a1 b2a1
info depth 10 seldepth 10 multipv 1 score cp 174 nodes 2246 nps 66058 hashfull 1 tbhits 0 time 34 pv d6e5 b1c3 e5e4 f3g5 f8e8 g5h3
info depth 11 seldepth 10 multipv 1 score cp 173 nodes 3139 nps 68239 hashfull 1 tbhits 0 time 46 pv d6e5 b1c3 e5e4 f3d4 d7e5
info depth 12 seldepth 16 multipv 1 score cp 125 nodes 20699 nps 79611 hashfull 8 tbhits 0 time 260 pv d6e5 f1e2 c7c5 e1g1 e5e4 f3e1
info depth 13 seldepth 21 multipv 1 score cp 106 nodes 75847 nps 82352 hashfull 31 tbhits 0 time 921 pv d7e5 b1c3 f6g4 f1d3 d6d5 e1g1
info depth 14 seldepth 20 multipv 1 score cp 95 upperbound nodes 110018 nps 83032 hashfull 44 tbhits 0 time 1325 pv d7e5 b1c3

blunderKey: 11258407522955116289
everEvalIsSkipped: 0
positionEverSaved: 0
globalSearch: 0
globalQsearch: 0


bestmove d7e5 ponder b1c3
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue
info depth 1 seldepth 3 multipv 1 score cp 45 nodes 54 nps 40 hashfull 0 tbhits 0 time 1329 pv a8d8
info depth 2 seldepth 3 multipv 1 score cp 53 nodes 117 nps 87 hashfull 0 tbhits 0 time 1330 pv c7c5
info depth 3 seldepth 3 multipv 1 score cp 66 nodes 173 nps 129 hashfull 0 tbhits 0 time 1331 pv e5e4
info depth 4 seldepth 2 multipv 1 score cp 66 nodes 222 nps 166 hashfull 0 tbhits 0 time 1332 pv e5e4
info depth 5 seldepth 3 multipv 1 score cp 66 nodes 275 nps 206 hashfull 0 tbhits 0 time 1332 pv e5e4 g1h1
info depth 6 seldepth 6 multipv 1 score cp 75 nodes 476 nps 356 hashfull 0 tbhits 0 time 1335 pv d6d8
info depth 7 seldepth 6 multipv 1 score cp 87 nodes 550 nps 411 hashfull 0 tbhits 0 time 1336 pv d6d8
info depth 8 seldepth 6 multipv 1 score cp 88 nodes 706 nps 527 hashfull 0 tbhits 0 time 1338 pv c7c6
info depth 9 seldepth 6 multipv 1 score cp 136 nodes 1065 nps 793 hashfull 1 tbhits 0 time 1343 pv a8d8
info depth 10 seldepth 8 multipv 1 score cp 157 nodes 1285 nps 953 hashfull 1 tbhits 0 time 1347 pv a8d8 e2f3 d6d2 d1d2 d8d2
info depth 11 seldepth 10 multipv 1 score cp 154 nodes 2145 nps 1578 hashfull 1 tbhits 0 time 1359 pv a8d8 d2d3 e5e4 c3e4 f5e4 d3e4
info depth 12 seldepth 14 multipv 1 score cp 90 nodes 9614 nps 6566 hashfull 5 tbhits 0 time 1464 pv h7h5 d2d3 a8d8
info depth 13 seldepth 15 multipv 1 score cp 83 nodes 25701 nps 15566 hashfull 13 tbhits 0 time 1651 pv a8d8 g2g4 f5c8 g4g5 f6d5 c3e4 d6e7 e2d3 h7h6 h2h4 h6g5 h4g5
info depth 14 seldepth 19 multipv 1 score cp 86 nodes 41367 nps 22531 hashfull 18 tbhits 0 time 1836 pv h7h5 d1e1 a8d8 d2d3 e5e4 c3e4 f5e4 d3e4 f6g4 e2g4 g7b2 a1d1 d6c6 d1d8 f8d8
info depth 15 seldepth 20 multipv 1 score cp 89 lowerbound nodes 66052 nps 31010 hashfull 27 tbhits 0 time 2130 pv a8d8

blunderKey: 11258407522955116289
everEvalIsSkipped: 0
positionEverSaved: 0
globalSearch: 0
globalQsearch: 0


bestmove a8d8 ponder g2g4
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue
info depth 1 seldepth 5 multipv 1 score cp 89 nodes 53 nps 65 hashfull 0 tbhits 0 time 808 pv a7a5
info depth 2 seldepth 3 multipv 1 score cp 89 nodes 111 nps 137 hashfull 0 tbhits 0 time 808 pv a7a5
info depth 3 seldepth 3 multipv 1 score cp 89 nodes 158 nps 195 hashfull 0 tbhits 0 time 809 pv a7a5
info depth 4 seldepth 3 multipv 1 score cp 89 nodes 205 nps 253 hashfull 0 tbhits 0 time 809 pv a7a5
info depth 5 seldepth 5 multipv 1 score cp 89 nodes 261 nps 322 hashfull 0 tbhits 0 time 810 pv a7a5 e3e4
info depth 6 seldepth 9 multipv 1 score cp 89 nodes 452 nps 556 hashfull 0 tbhits 0 time 812 pv a7a5 e3e4 a5b4
info depth 7 seldepth 9 multipv 1 score cp 98 nodes 823 nps 1007 hashfull 0 tbhits 0 time 817 pv a7a5 b4a5 d6c5 d1d2
info depth 8 seldepth 10 multipv 1 score cp 101 nodes 2616 nps 3117 hashfull 0 tbhits 0 time 839 pv a7a5 b4a5 a8a5 b2c1 d6b6 g1h1
info depth 9 seldepth 12 multipv 1 score cp 92 nodes 5004 nps 5699 hashfull 1 tbhits 0 time 878 pv a7a5 b4a5 a8a5 d1e1 d6b6 c3d1 a5b5 a1b1 f8a8 h2h3
info depth 10 seldepth 12 multipv 1 score cp 93 nodes 5476 nps 6194 hashfull 2 tbhits 0 time 884 pv a7a5 b4a5 a8a5 d1e1 d6b6 c3d1 a5b5 b2c3 b5c5 g1h1
info depth 11 seldepth 14 multipv 1 score cp 92 nodes 6981 nps 7705 hashfull 2 tbhits 0 time 906 pv a7a5 b4a5 a8a5 d1e1 d6b6 c3d1 f8d8 h2h3 a5b5
info depth 12 seldepth 16 multipv 1 score cp 88 nodes 11793 nps 12033 hashfull 4 tbhits 0 time 980 pv a7a5 b4a5 d6c5 d1d2 f8d8 c3d1 a8a5 g1h1 a5a4 a1c1
info depth 13 seldepth 16 multipv 1 score cp 94 nodes 20281 nps 18353 hashfull 8 tbhits 0 time 1105 pv a7a5 b4a5 d6c5 d1d2 g7h6 b2c1 a8a5 g1h1
info depth 14 seldepth 16 multipv 1 score cp 94 nodes 21639 nps 19251 hashfull 8 tbhits 0 time 1124 pv a7a5 b4a5 d6c5 d1d2 g7h6 b2c1 a8a5 g1h1

blunderKey: 11258407522955116289
everEvalIsSkipped: 0
positionEverSaved: 0
globalSearch: 0
globalQsearch: 0


bestmove a7a5 ponder b4a5

bestValueOfQueenBlunder: 1516


bestValueOfQueenBlunder: 1516

info depth 1 seldepth 6 multipv 1 score cp 27 nodes 73 nps 224 hashfull 0 tbhits 0 time 325 pv f6e8

bestValueOfQueenBlunder: 1516

info depth 2 seldepth 3 multipv 1 score cp 27 nodes 105 nps 322 hashfull 0 tbhits 0 time 326 pv f6e8

blunderKey: 11258407522955116289
everEvalIsSkipped: 0
positionEverSaved: 0
globalSearch: 0
globalQsearch: 3


bestmove f6e8

Conclusion

This proves that there was a hash collision that caused the queen's blunder.

[peregrineshahin]

here is the output for the collided positions using in qsearch and search for that position

    if (uint16_t(posKey) == uint16_t(2069647028597739265uLL))
    {
        sync_cout << std::endl << pos.fen() << sync_endl;
    }

outout

Stockfish dev-20240416-ea20ac70 by the Stockfish developers (see AUTHORS file)
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue
info depth 1 seldepth 5 multipv 1 score cp 56 nodes 44 nps 11000 hashfull 0 tbhits 0 time 4 pv d7e5
info depth 2 seldepth 3 multipv 1 score cp 101 nodes 81 nps 16200 hashfull 0 tbhits 0 time 5 pv d6e5
info depth 3 seldepth 3 multipv 1 score cp 101 nodes 113 nps 22600 hashfull 0 tbhits 0 time 5 pv d6e5
info depth 4 seldepth 3 multipv 1 score cp 101 nodes 142 nps 23666 hashfull 0 tbhits 0 time 6 pv d6e5
info depth 5 seldepth 5 multipv 1 score cp 101 nodes 173 nps 28833 hashfull 0 tbhits 0 time 6 pv d6e5 f3e5
info depth 6 seldepth 5 multipv 1 score cp 110 nodes 229 nps 32714 hashfull 0 tbhits 0 time 7 pv d6e5 f3e5 f8e8
info depth 7 seldepth 5 multipv 1 score cp 137 nodes 309 nps 38625 hashfull 0 tbhits 0 time 8 pv d6e5 f3e5 f8e8
info depth 8 seldepth 7 multipv 1 score cp 181 nodes 563 nps 46916 hashfull 0 tbhits 0 time 12 pv d6e5 f3e5 f6e4
info depth 9 seldepth 10 multipv 1 score cp 164 nodes 1659 nps 63807 hashfull 0 tbhits 0 time 26 pv d6e5 d2d3 a7a5 e3e4 a5b4 a3b4 a8a1 b2a1
info depth 10 seldepth 10 multipv 1 score cp 174 nodes 2246 nps 66058 hashfull 1 tbhits 0 time 34 pv d6e5 b1c3 e5e4 f3g5 f8e8 g5h3
info depth 11 seldepth 10 multipv 1 score cp 173 nodes 3139 nps 78475 hashfull 1 tbhits 0 time 40 pv d6e5 b1c3 e5e4 f3d4 d7e5
info depth 12 seldepth 16 multipv 1 score cp 125 nodes 20699 nps 72374 hashfull 8 tbhits 0 time 286 pv d6e5 f1e2 c7c5 e1g1 e5e4 f3e1
info depth 13 seldepth 21 multipv 1 score cp 106 nodes 75847 nps 75245 hashfull 31 tbhits 0 time 1008 pv d7e5 b1c3 f6g4 f1d3 d6d5 e1g1
info depth 14 seldepth 20 multipv 1 score cp 95 upperbound nodes 110018 nps 75926 hashfull 44 tbhits 0 time 1449 pv d7e5 b1c3

everEvalIsSkipped: 0
positionEverSaved: 0
globalSearch: 0
globalQsearch: 0


bestmove d7e5 ponder b1c3
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue
info depth 1 seldepth 3 multipv 1 score cp 45 nodes 54 nps 37 hashfull 0 tbhits 0 time 1452 pv a8d8
info depth 2 seldepth 3 multipv 1 score cp 53 nodes 117 nps 80 hashfull 0 tbhits 0 time 1453 pv c7c5
info depth 3 seldepth 3 multipv 1 score cp 66 nodes 173 nps 118 hashfull 0 tbhits 0 time 1454 pv e5e4
info depth 4 seldepth 2 multipv 1 score cp 66 nodes 222 nps 152 hashfull 0 tbhits 0 time 1455 pv e5e4
info depth 5 seldepth 3 multipv 1 score cp 66 nodes 275 nps 189 hashfull 0 tbhits 0 time 1455 pv e5e4 g1h1
info depth 6 seldepth 6 multipv 1 score cp 75 nodes 476 nps 326 hashfull 0 tbhits 0 time 1458 pv d6d8
info depth 7 seldepth 6 multipv 1 score cp 87 nodes 550 nps 376 hashfull 0 tbhits 0 time 1459 pv d6d8
info depth 8 seldepth 6 multipv 1 score cp 88 nodes 706 nps 483 hashfull 0 tbhits 0 time 1460 pv c7c6
info depth 9 seldepth 6 multipv 1 score cp 136 nodes 1065 nps 726 hashfull 1 tbhits 0 time 1466 pv a8d8
info depth 10 seldepth 8 multipv 1 score cp 157 nodes 1285 nps 874 hashfull 1 tbhits 0 time 1469 pv a8d8 e2f3 d6d2 d1d2 d8d2
info depth 11 seldepth 10 multipv 1 score cp 154 nodes 2145 nps 1453 hashfull 1 tbhits 0 time 1476 pv a8d8 d2d3 e5e4 c3e4 f5e4 d3e4
info depth 12 seldepth 14 multipv 1 score cp 90 nodes 9614 nps 6042 hashfull 5 tbhits 0 time 1591 pv h7h5 d2d3 a8d8
info depth 13 seldepth 15 multipv 1 score cp 83 nodes 25701 nps 13998 hashfull 13 tbhits 0 time 1836 pv a8d8 g2g4 f5c8 g4g5 f6d5 c3e4 d6e7 e2d3 h7h6 h2h4 h6g5 h4g5
info depth 14 seldepth 19 multipv 1 score cp 86 nodes 41367 nps 20258 hashfull 18 tbhits 0 time 2042 pv h7h5 d1e1 a8d8 d2d3 e5e4 c3e4 f5e4 d3e4 f6g4 e2g4 g7b2 a1d1 d6c6 d1d8 f8d8

5rk1/1pp2pb1/5np1/4pb1p/1q2P3/2NP4/1BP1B1PP/Q4R1K w - - 0 17
info depth 15 seldepth 20 multipv 1 score cp 89 lowerbound nodes 66052 nps 28023 hashfull 27 tbhits 0 time 2357 pv a8d8

everEvalIsSkipped: 0
positionEverSaved: 0
globalSearch: 0
globalQsearch: 0


bestmove a8d8 ponder g2g4
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue
info depth 1 seldepth 5 multipv 1 score cp 89 nodes 53 nps 58 hashfull 0 tbhits 0 time 912 pv a7a5
info depth 2 seldepth 3 multipv 1 score cp 89 nodes 111 nps 121 hashfull 0 tbhits 0 time 913 pv a7a5
info depth 3 seldepth 3 multipv 1 score cp 89 nodes 158 nps 173 hashfull 0 tbhits 0 time 913 pv a7a5
info depth 4 seldepth 3 multipv 1 score cp 89 nodes 205 nps 224 hashfull 0 tbhits 0 time 914 pv a7a5
info depth 5 seldepth 5 multipv 1 score cp 89 nodes 261 nps 285 hashfull 0 tbhits 0 time 915 pv a7a5 e3e4
info depth 6 seldepth 9 multipv 1 score cp 89 nodes 452 nps 492 hashfull 0 tbhits 0 time 917 pv a7a5 e3e4 a5b4
info depth 7 seldepth 9 multipv 1 score cp 98 nodes 823 nps 894 hashfull 0 tbhits 0 time 920 pv a7a5 b4a5 d6c5 d1d2
info depth 8 seldepth 10 multipv 1 score cp 101 nodes 2616 nps 2756 hashfull 0 tbhits 0 time 949 pv a7a5 b4a5 a8a5 b2c1 d6b6 g1h1
info depth 9 seldepth 12 multipv 1 score cp 92 nodes 5004 nps 4901 hashfull 1 tbhits 0 time 1021 pv a7a5 b4a5 a8a5 d1e1 d6b6 c3d1 a5b5 a1b1 f8a8 h2h3
info depth 10 seldepth 12 multipv 1 score cp 93 nodes 5476 nps 5306 hashfull 2 tbhits 0 time 1032 pv a7a5 b4a5 a8a5 d1e1 d6b6 c3d1 a5b5 b2c3 b5c5 g1h1
info depth 11 seldepth 14 multipv 1 score cp 92 nodes 6981 nps 6294 hashfull 2 tbhits 0 time 1109 pv a7a5 b4a5 a8a5 d1e1 d6b6 c3d1 f8d8 h2h3 a5b5
info depth 12 seldepth 16 multipv 1 score cp 88 nodes 11793 nps 8614 hashfull 4 tbhits 0 time 1369 pv a7a5 b4a5 d6c5 d1d2 f8d8 c3d1 a8a5 g1h1 a5a4 a1c1

5rk1/1pp2pb1/5np1/4pb1p/1q2P3/2NP4/1BP1B1PP/Q4R1K w - - 0 18
info depth 13 seldepth 16 multipv 1 score cp 94 nodes 20281 nps 13611 hashfull 8 tbhits 0 time 1490 pv a7a5 b4a5 d6c5 d1d2 g7h6 b2c1 a8a5 g1h1

4r1k1/1pp2p2/5np1/2r2b1p/4p3/P1N1b3/2PQB1PP/1RB2R1K w - - 0 21
info depth 14 seldepth 16 multipv 1 score cp 94 nodes 21639 nps 14143 hashfull 8 tbhits 0 time 1530 pv a7a5 b4a5 d6c5 d1d2 g7h6 b2c1 a8a5 g1h1

everEvalIsSkipped: 0
positionEverSaved: 0
globalSearch: 0
globalQsearch: 0


bestmove a7a5 ponder b4a5

r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b - - 0 21

bestValueOfQueenBlunder: 1516


r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b - - 0 21

bestValueOfQueenBlunder: 1516

info depth 1 seldepth 6 multipv 1 score cp 27 nodes 73 nps 116 hashfull 0 tbhits 0 time 629 pv f6e8

r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b - - 0 21

bestValueOfQueenBlunder: 1516

info depth 2 seldepth 3 multipv 1 score cp 27 nodes 105 nps 166 hashfull 0 tbhits 0 time 631 pv f6e8

everEvalIsSkipped: 0
positionEverSaved: 0
globalSearch: 0
globalQsearch: 3


bestmove f6e8

[robertnurnberg]

From @peregrineshahin's output, these are the positions that collide in TT:

5rk1/1pp2pb1/5np1/4pb1p/1q2P3/2NP4/1BP1B1PP/Q4R1K w - - 0 17
5rk1/1pp2pb1/5np1/4pb1p/1q2P3/2NP4/1BP1B1PP/Q4R1K w - - 0 18
4r1k1/1pp2p2/5np1/2r2b1p/4p3/P1N1b3/2PQB1PP/1RB2R1K w - - 0 21
r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b - - 0 21

The first two are the same position with different full move counter, and the final one is the position after the blunder was punished. Both positions for white to move are clearly winning, the second one more so. I believe the latter one leads the incorrect (winning) eval for the blundered position.

Corrected: Only true collisions as per #5174 (comment) are

4r1k1/1pp2p2/5np1/2r2b1p/4p3/P1N1b3/2PQB1PP/1RB2R1K w - - 0 21
r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b - - 0 21

[peregrineshahin]

Yes worth noting that the value we cut off with is 1516 which is a winning eval for the side to move.
this happened in qsearch stand. pat. (immediately cut off with the wrong tt value) i.e. the hash collision is in effect:

diff --git a/src/search.cpp b/src/search.cpp
index 24805aa7..5a1e993b 100644
--- a/src/search.cpp
+++ b/src/search.cpp
@@ -1445,6 +1445,8 @@ Value Search::Worker::qsearch(Position& pos, Stack* ss, Value alpha, Value beta,
     {
         if (ss->ttHit)
         {
+            // Wrong ttHit as we never saved this position before
+
             // Never assume anything about values stored in TT
             unadjustedStaticEval = tte->eval();
             if (unadjustedStaticEval == VALUE_NONE)
@@ -1456,6 +1458,9 @@ Value Search::Worker::qsearch(Position& pos, Stack* ss, Value alpha, Value beta,
             if (ttValue != VALUE_NONE
                 && (tte->bound() & (ttValue > bestValue ? BOUND_LOWER : BOUND_UPPER)))
                 bestValue = ttValue;
+
+            // best value is set to a wrong value of 1516
+            // for r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b - - 0 21
         }
         else
         {
@@ -1470,10 +1475,12 @@ Value Search::Worker::qsearch(Position& pos, Stack* ss, Value alpha, Value beta,
         // Stand pat. Return immediately if static value is at least beta
         if (bestValue >= beta)
         {
+            // ohuh we are about to Stand pat.
             if (!ss->ttHit)
                 tte->save(posKey, value_to_tt(bestValue, ss->ply), false, BOUND_LOWER, DEPTH_NONE,
                           Move::none(), unadjustedStaticEval, tt.generation());

+            // wrong bestValue is immediatly used and cutoff happened
             return bestValue;
         }

[peregrineshahin]

For completeness, it looks like there was a missing check in my analysis that led to identifying the only collision ( culprit )
which is with 4r1k1/1pp2p2/5np1/2r2b1p/4p3/P1N1b3/2PQB1PP/1RB2R1K w - - 0 21

Analysis Code

diff --git a/src/evaluate.cpp b/src/evaluate.cpp
index bc705b85..d76e0e4d 100644
--- a/src/evaluate.cpp
+++ b/src/evaluate.cpp
@@ -85,6 +85,14 @@ Value Eval::evaluate(const Eval::NNUE::Networks& networks, const Position& pos,
     // Guarantee evaluation does not hit the tablebase range
     v = std::clamp(v, VALUE_TB_LOSS_IN_MAX_PLY + 1, VALUE_TB_WIN_IN_MAX_PLY - 1);

+
+    /// ever evaluated
+    if (pos.fen().find("r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b")
+        != std::string::npos)
+    {
+        assert(false);
+    }
+
     return v;
 }

diff --git a/src/search.cpp b/src/search.cpp
index 3f882aab..c5988c1b 100644
--- a/src/search.cpp
+++ b/src/search.cpp
@@ -46,6 +46,13 @@

 namespace Stockfish {

+int         searchHit          = 0;
+int         qsearchHit         = 0;
+int         positionEverSaved  = 0;
+int         everEvalIsSkipped  = 0;
+std::string blunderPosition    = "r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b";
+auto        blunderPositionKey = 2069647028597739265uLL;
+
 namespace TB = Tablebases;

 using Eval::evaluate;
@@ -204,6 +211,13 @@ void Search::Worker::start_searching() {
         sync_cout << main_manager()->pv(*bestThread, threads, tt, bestThread->completedDepth)
                   << sync_endl;

+    std::cerr << std::endl;
+    std::cerr << "everEvalIsSkipped: " << everEvalIsSkipped << std::endl;
+    std::cerr << "positionEverSaved: " << positionEverSaved << std::endl;
+    std::cerr << "searchHit: " << searchHit << std::endl;
+    std::cerr << "qsearchHit: " << qsearchHit << std::endl;
+    std::cerr << std::endl << std::endl;
+
     sync_cout << "bestmove " << UCI::move(bestThread->rootMoves[0].pv[0], rootPos.is_chess960());

     if (bestThread->rootMoves[0].pv.size() > 1
@@ -510,6 +524,9 @@ template<NodeType nodeType>
 Value Search::Worker::search(
   Position& pos, Stack* ss, Value alpha, Value beta, Depth depth, bool cutNode) {

+    if (pos.fen().find(blunderPosition) != std::string::npos)
+        searchHit++;
+
     constexpr bool PvNode   = nodeType != NonPV;
     constexpr bool rootNode = nodeType == Root;

@@ -597,7 +614,8 @@ Value Search::Worker::search(
     // Step 4. Transposition table lookup.
     excludedMove = ss->excludedMove;
     posKey       = pos.key();
-    tte          = tt.probe(posKey, ss->ttHit);
+
+    tte       = tt.probe(posKey, ss->ttHit);
     ttValue   = ss->ttHit ? value_from_tt(tte->value(), ss->ply, pos.rule50_count()) : VALUE_NONE;
     ttMove    = rootNode  ? thisThread->rootMoves[thisThread->pvIdx].pv[0]
               : ss->ttHit ? tte->move()
@@ -671,6 +689,17 @@ Value Search::Worker::search(

                 if (b == BOUND_EXACT || (b == BOUND_LOWER ? value >= beta : value <= alpha))
                 {
+                    if (pos.fen().find(blunderPosition) != std::string::npos)
+                    {
+                        positionEverSaved++;
+                    }
+
+                    if (uint16_t(posKey) == uint16_t(blunderPositionKey)
+                        && mul_hi64(posKey, 524288) == mul_hi64(blunderPositionKey, 524288))
+                    {
+                        sync_cout << std::endl << pos.fen() << sync_endl;
+                    }
+
                     tte->save(posKey, value_to_tt(value, ss->ply), ss->ttPv, b,
                               std::min(MAX_PLY - 1, depth + 6), Move::none(), VALUE_NONE,
                               tt.generation());
@@ -716,6 +745,11 @@ Value Search::Worker::search(

         ss->staticEval = eval = to_corrected_static_eval(unadjustedStaticEval, *thisThread, pos);

+        if (pos.fen().find(blunderPosition) != std::string::npos)
+        {
+            positionEverSaved++;
+        }
+
         // ttValue can be used as a better position evaluation (~7 Elo)
         if (ttValue != VALUE_NONE && (tte->bound() & (ttValue > eval ? BOUND_LOWER : BOUND_UPPER)))
             eval = ttValue;
@@ -726,6 +760,18 @@ Value Search::Worker::search(
         ss->staticEval = eval = to_corrected_static_eval(unadjustedStaticEval, *thisThread, pos);

         // Static evaluation is saved as it was before adjustment by correction history
+
+        if (pos.fen().find(blunderPosition) != std::string::npos)
+        {
+            positionEverSaved++;
+        }
+
+        if (uint16_t(posKey) == uint16_t(blunderPositionKey)
+            && mul_hi64(posKey, 524288) == mul_hi64(blunderPositionKey, 524288))
+        {
+            sync_cout << std::endl << pos.fen() << sync_endl;
+        }
+
         tte->save(posKey, VALUE_NONE, ss->ttPv, BOUND_NONE, DEPTH_NONE, Move::none(),
                   unadjustedStaticEval, tt.generation());
     }
@@ -871,6 +917,17 @@ Value Search::Worker::search(

                 if (value >= probCutBeta)
                 {
+                    if (pos.fen().find(blunderPosition) != std::string::npos)
+                    {
+                        positionEverSaved++;
+                    }
+
+                    if (uint16_t(posKey) == uint16_t(blunderPositionKey)
+                        && mul_hi64(posKey, 524288) == mul_hi64(blunderPositionKey, 524288))
+                    {
+                        sync_cout << std::endl << pos.fen() << sync_endl;
+                    }
+
                     // Save ProbCut data into transposition table
                     tte->save(posKey, value_to_tt(value, ss->ply), ss->ttPv, BOUND_LOWER, depth - 3,
                               move, unadjustedStaticEval, tt.generation());
@@ -1341,11 +1398,25 @@ moves_loop:  // When in check, search starts here
     // Write gathered information in transposition table
     // Static evaluation is saved as it was before correction history
     if (!excludedMove && !(rootNode && thisThread->pvIdx))
+    {
+        if (pos.fen().find(blunderPosition) != std::string::npos)
+        {
+            positionEverSaved++;
+        }
+
+        if (uint16_t(posKey) == uint16_t(blunderPositionKey)
+            && mul_hi64(posKey, 524288) == mul_hi64(blunderPositionKey, 524288))
+        {
+            sync_cout << std::endl << pos.fen() << sync_endl;
+        }
+
         tte->save(posKey, value_to_tt(bestValue, ss->ply), ss->ttPv,
                   bestValue >= beta    ? BOUND_LOWER
                   : PvNode && bestMove ? BOUND_EXACT
                                        : BOUND_UPPER,
                   depth, bestMove, unadjustedStaticEval, tt.generation());
+    }
+

     // Adjust correction history
     if (!ss->inCheck && (!bestMove || !pos.capture(bestMove))
@@ -1372,6 +1443,9 @@ Value Search::Worker::qsearch(Position& pos, Stack* ss, Value alpha, Value beta,
     static_assert(nodeType != Root);
     constexpr bool PvNode = nodeType == PV;

+    if (pos.fen().find(blunderPosition) != std::string::npos)
+        qsearchHit++;
+
     assert(alpha >= -VALUE_INFINITE && alpha < beta && beta <= VALUE_INFINITE);
     assert(PvNode || (alpha == beta - 1));
     assert(depth <= 0);
@@ -1457,6 +1531,13 @@ Value Search::Worker::qsearch(Position& pos, Stack* ss, Value alpha, Value beta,
             if (ttValue != VALUE_NONE
                 && (tte->bound() & (ttValue > bestValue ? BOUND_LOWER : BOUND_UPPER)))
                 bestValue = ttValue;
+
+            if (pos.fen().find(blunderPosition) != std::string::npos)
+            {
+                std::cerr << std::endl
+                          << "bestValueOfQueenBlunder: " << bestValue << std::endl
+                          << std::endl;
+            }
         }
         else
         {
@@ -1466,14 +1547,31 @@ Value Search::Worker::qsearch(Position& pos, Stack* ss, Value alpha, Value beta,
                                    : -(ss - 1)->staticEval;
             ss->staticEval       = bestValue =
               to_corrected_static_eval(unadjustedStaticEval, *thisThread, pos);
+
+            if (pos.fen().find(blunderPosition) != std::string::npos)
+            {
+                everEvalIsSkipped++;
+            }
         }

         // Stand pat. Return immediately if static value is at least beta
         if (bestValue >= beta)
         {
+
             if (!ss->ttHit)
+            {
+                // no check for saving here as the position is never evaluated
+                if (pos.fen().find("r3n1k1/1p2Qpb1/2p1b1p1/r3p2p/8/P1NPPB1P/1BP3P1/1R3R1K b")
+                      == std::string::npos
+                    && uint16_t(posKey) == uint16_t(blunderPositionKey)
+                    && mul_hi64(posKey, 524288) == mul_hi64(blunderPositionKey, 524288))
+                {
+                    sync_cout << std::endl << pos.fen() << sync_endl;
+                }
+
                 tte->save(posKey, value_to_tt(bestValue, ss->ply), false, BOUND_LOWER, DEPTH_NONE,
                           Move::none(), unadjustedStaticEval, tt.generation());
+            }

             return bestValue;
         }
@@ -1616,6 +1714,12 @@ Value Search::Worker::qsearch(Position& pos, Stack* ss, Value alpha, Value beta,
     if (std::abs(bestValue) < VALUE_TB_WIN_IN_MAX_PLY && bestValue >= beta)
         bestValue = (3 * bestValue + beta) / 4;

+    if (uint16_t(posKey) == uint16_t(blunderPositionKey)
+        && mul_hi64(posKey, 524288) == mul_hi64(blunderPositionKey, 524288))
+    {
+        sync_cout << std::endl << pos.fen() << sync_endl;
+    }
+
     // Save gathered info in transposition table
     // Static evaluation is saved as it was before adjustment by correction history
     tte->save(posKey, value_to_tt(bestValue, ss->ply), pvHit,

Analysis Result

$ more input.txt | ./stockfish.exe
Stockfish dev-20240402-0716b845 by the Stockfish developers (see AUTHORS file)
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue
info depth 1 seldepth 5 multipv 1 score cp 56 nodes 44 nps 8800 hashfull 0 tbhits 0 time 5 pv d7e5
info depth 2 seldepth 3 multipv 1 score cp 101 nodes 81 nps 13500 hashfull 0 tbhits 0 time 6 pv d6e5
info depth 3 seldepth 3 multipv 1 score cp 101 nodes 113 nps 16142 hashfull 0 tbhits 0 time 7 pv d6e
5
info depth 4 seldepth 3 multipv 1 score cp 101 nodes 142 nps 20285 hashfull 0 tbhits 0 time 7 pv d6e
5
info depth 5 seldepth 5 multipv 1 score cp 101 nodes 173 nps 21625 hashfull 0 tbhits 0 time 8 pv d6e
5 f3e5
info depth 6 seldepth 5 multipv 1 score cp 110 nodes 229 nps 25444 hashfull 0 tbhits 0 time 9 pv d6e
5 f3e5 f8e8
info depth 7 seldepth 5 multipv 1 score cp 137 nodes 309 nps 34333 hashfull 0 tbhits 0 time 9 pv d6e
5 f3e5 f8e8
info depth 8 seldepth 7 multipv 1 score cp 181 nodes 563 nps 46916 hashfull 0 tbhits 0 time 12 pv d6
e5 f3e5 f6e4
info depth 9 seldepth 10 multipv 1 score cp 164 nodes 1659 nps 82950 hashfull 0 tbhits 0 time 20 pv
d6e5 d2d3 a7a5 e3e4 a5b4 a3b4 a8a1 b2a1
info depth 10 seldepth 10 multipv 1 score cp 174 nodes 2246 nps 89840 hashfull 1 tbhits 0 time 25 pv
 d6e5 b1c3 e5e4 f3g5 f8e8 g5h3
info depth 11 seldepth 10 multipv 1 score cp 173 nodes 3139 nps 80487 hashfull 1 tbhits 0 time 39 pv
 d6e5 b1c3 e5e4 f3d4 d7e5
info depth 12 seldepth 16 multipv 1 score cp 125 nodes 20699 nps 113109 hashfull 8 tbhits 0 time 183
 pv d6e5 f1e2 c7c5 e1g1 e5e4 f3e1
info depth 13 seldepth 21 multipv 1 score cp 106 nodes 75847 nps 130770 hashfull 31 tbhits 0 time 58
0 pv d7e5 b1c3 f6g4 f1d3 d6d5 e1g1
info depth 14 seldepth 20 multipv 1 score cp 95 upperbound nodes 110018 nps 133193 hashfull 44 tbhit
s 0 time 826 pv d7e5 b1c3

everEvalIsSkipped: 0
positionEverSaved: 0
searchHit: 0
qsearchHit: 0


bestmove d7e5 ponder b1c3
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue
info depth 1 seldepth 3 multipv 1 score cp 45 nodes 54 nps 65 hashfull 0 tbhits 0 time 828 pv a8d8
info depth 2 seldepth 3 multipv 1 score cp 53 nodes 117 nps 141 hashfull 0 tbhits 0 time 829 pv c7c5
info depth 3 seldepth 3 multipv 1 score cp 66 nodes 173 nps 208 hashfull 0 tbhits 0 time 829 pv e5e4
info depth 4 seldepth 2 multipv 1 score cp 66 nodes 222 nps 267 hashfull 0 tbhits 0 time 830 pv e5e4
info depth 5 seldepth 3 multipv 1 score cp 66 nodes 275 nps 331 hashfull 0 tbhits 0 time 830 pv e5e4
 g1h1
info depth 6 seldepth 6 multipv 1 score cp 75 nodes 476 nps 572 hashfull 0 tbhits 0 time 832 pv d6d8
info depth 7 seldepth 6 multipv 1 score cp 87 nodes 550 nps 661 hashfull 0 tbhits 0 time 832 pv d6d8
info depth 8 seldepth 6 multipv 1 score cp 88 nodes 706 nps 846 hashfull 0 tbhits 0 time 834 pv c7c6
info depth 9 seldepth 6 multipv 1 score cp 136 nodes 1065 nps 1272 hashfull 1 tbhits 0 time 837 pv a
8d8
info depth 10 seldepth 8 multipv 1 score cp 157 nodes 1285 nps 1531 hashfull 1 tbhits 0 time 839 pv
a8d8 e2f3 d6d2 d1d2 d8d2
info depth 11 seldepth 10 multipv 1 score cp 154 nodes 2145 nps 2535 hashfull 1 tbhits 0 time 846 pv
 a8d8 d2d3 e5e4 c3e4 f5e4 d3e4
info depth 12 seldepth 14 multipv 1 score cp 90 nodes 9614 nps 10541 hashfull 5 tbhits 0 time 912 pv
 h7h5 d2d3 a8d8
info depth 13 seldepth 15 multipv 1 score cp 83 nodes 25701 nps 24594 hashfull 13 tbhits 0 time 1045
 pv a8d8 g2g4 f5c8 g4g5 f6d5 c3e4 d6e7 e2d3 h7h6 h2h4 h6g5 h4g5
info depth 14 seldepth 19 multipv 1 score cp 86 nodes 41367 nps 34938 hashfull 18 tbhits 0 time 1184
 pv h7h5 d1e1 a8d8 d2d3 e5e4 c3e4 f5e4 d3e4 f6g4 e2g4 g7b2 a1d1 d6c6 d1d8 f8d8
info depth 15 seldepth 20 multipv 1 score cp 89 lowerbound nodes 66052 nps 47485 hashfull 27 tbhits
0 time 1391 pv a8d8

everEvalIsSkipped: 0
positionEverSaved: 0
searchHit: 0
qsearchHit: 0


bestmove a8d8 ponder g2g4
info string NNUE evaluation using nn-ae6a388e4a1a.nnue
info string NNUE evaluation using nn-baff1ede1f90.nnue
info depth 1 seldepth 5 multipv 1 score cp 89 nodes 53 nps 92 hashfull 0 tbhits 0 time 570 pv a7a5
info depth 2 seldepth 3 multipv 1 score cp 89 nodes 111 nps 194 hashfull 0 tbhits 0 time 570 pv a7a5
info depth 3 seldepth 3 multipv 1 score cp 89 nodes 158 nps 276 hashfull 0 tbhits 0 time 571 pv a7a5
info depth 4 seldepth 3 multipv 1 score cp 89 nodes 205 nps 359 hashfull 0 tbhits 0 time 571 pv a7a5
info depth 5 seldepth 5 multipv 1 score cp 89 nodes 261 nps 456 hashfull 0 tbhits 0 time 572 pv a7a5
 e3e4
info depth 6 seldepth 9 multipv 1 score cp 89 nodes 452 nps 788 hashfull 0 tbhits 0 time 573 pv a7a5
 e3e4 a5b4
info depth 7 seldepth 9 multipv 1 score cp 98 nodes 823 nps 1428 hashfull 0 tbhits 0 time 576 pv a7a
5 b4a5 d6c5 d1d2
info depth 8 seldepth 10 multipv 1 score cp 101 nodes 2616 nps 4441 hashfull 0 tbhits 0 time 589 pv
a7a5 b4a5 a8a5 b2c1 d6b6 g1h1
info depth 9 seldepth 12 multipv 1 score cp 92 nodes 5004 nps 8243 hashfull 1 tbhits 0 time 607 pv a
7a5 b4a5 a8a5 d1e1 d6b6 c3d1 a5b5 a1b1 f8a8 h2h3
info depth 10 seldepth 12 multipv 1 score cp 93 nodes 5476 nps 8962 hashfull 2 tbhits 0 time 611 pv
a7a5 b4a5 a8a5 d1e1 d6b6 c3d1 a5b5 b2c3 b5c5 g1h1
info depth 11 seldepth 14 multipv 1 score cp 92 nodes 6981 nps 11205 hashfull 2 tbhits 0 time 623 pv
 a7a5 b4a5 a8a5 d1e1 d6b6 c3d1 f8d8 h2h3 a5b5
info depth 12 seldepth 16 multipv 1 score cp 88 nodes 11793 nps 17841 hashfull 4 tbhits 0 time 661 p
v a7a5 b4a5 d6c5 d1d2 f8d8 c3d1 a8a5 g1h1 a5a4 a1c1
info depth 13 seldepth 16 multipv 1 score cp 94 nodes 20281 nps 27518 hashfull 8 tbhits 0 time 737 p
v a7a5 b4a5 d6c5 d1d2 g7h6 b2c1 a8a5 g1h1

4r1k1/1pp2p2/5np1/2r2b1p/4p3/P1N1b3/2PQB1PP/1RB2R1K w - - 0 21
info depth 14 seldepth 16 multipv 1 score cp 94 nodes 21639 nps 28813 hashfull 8 tbhits 0 time 751 p
v a7a5 b4a5 d6c5 d1d2 g7h6 b2c1 a8a5 g1h1

everEvalIsSkipped: 0
positionEverSaved: 0
searchHit: 0
qsearchHit: 0


bestmove a7a5 ponder b4a5

bestValueOfQueenBlunder: 1516


bestValueOfQueenBlunder: 1516

info depth 1 seldepth 6 multipv 1 score cp 27 nodes 73 nps 380 hashfull 0 tbhits 0 time 192 pv f6e8

bestValueOfQueenBlunder: 1516

info depth 2 seldepth 3 multipv 1 score cp 27 nodes 105 nps 544 hashfull 0 tbhits 0 time 193 pv f6e8

everEvalIsSkipped: 0
positionEverSaved: 0
searchHit: 0
qsearchHit: 3


bestmove f6e8