MDEV-36684 - main.mdl_sync fails under valgrind (test for Bug#42643)

Valgrind is single threaded and only changes threads as part of
system calls or waits.

Some busy loops were identified and fixed where the server assumes
that some other thread will change the state, which will not happen
with valgrind.

Based on patch by Monty. Original patch introduced VALGRIND_YIELD,
which emits pthread_yield() only in valgrind builds. However it was
agreed that it is a good idea to emit yield() unconditionally, such
that other affected schedulers (like SCHED_FIFO) benefit from this
change. Also avoid pthread_yield() in favour of standard
std::this_thread::yield().

Author: svoj

sql/sql_base.cc

@@ -4460,6 +4460,7 @@ bool open_tables(THD *thd, const DDL_options_st &options,
             goto error;
 
           error= FALSE;
+          std::this_thread::yield();
           goto restart;
         }
         goto error;
@@ -4522,6 +4523,7 @@ bool open_tables(THD *thd, const DDL_options_st &options,
               goto error;
 
             error= FALSE;
+            std::this_thread::yield();
             goto restart;
           }
           /*

sql/table_cache.cc

@@ -901,6 +901,7 @@ TABLE_SHARE *tdc_acquire_share(THD *thd, TABLE_LIST *tl, uint flags,
   {
     mysql_mutex_unlock(&element->LOCK_table_share);
     lf_hash_search_unpin(thd->tdc_hash_pins);
+    std::this_thread::yield();
     goto retry;
   }
   lf_hash_search_unpin(thd->tdc_hash_pins);

storage/innobase/trx/trx0purge.cc

@@ -1176,6 +1176,7 @@ dict_table_t *purge_sys_t::close_and_reopen(table_id_t id, THD *thd,
   MDL_context *mdl_context= static_cast<MDL_context*>(thd_mdl_context(thd));
   ut_ad(mdl_context);
  retry:
+  std::this_thread::yield();
   ut_ad(m_active);
 
   for (que_thr_t *thr= UT_LIST_GET_FIRST(purge_sys.query->thrs); thr;