Bug#37882398 [InnoDB] Assertion failure: ddl0file-reader.cc:193:

             m_ptr + data_size < m_bounds.second

PROBLEM:

1. After the fix for the "Bug #37233273 - DDL buffer increase has
   out of control memory usage", this assertion was seen.
2. The fix for Bug #37233273 basically accounts for meta data as
   part of innodb_ddl_buffer_size buffer  which reduces the
   number of rows which can fit in a key buffer block while
   adding an index.
3. When the key buffer is full, the records in the key buffer
   are added to the temporary file, which constitutes a chunk.
   This chunk is smaller than actual key buffer size.
4. The offsets of each chunk is recorded in a queue. The chunks
   sizes are multiples of the sector size (4KB).
5. When merging the two chunks into one sorted chunk, we read
   from this queue.
6. Another bug, Bug#36444172 "Regression in 8.0.27+ related to
   Parallel DDL IO Amplification" read the sizes of the first
   two chunks from the queue and set a limit on the input buffer
   based on the size of the first two chunks.
7. In our case, the sizes of the first two chunks were smaller
   compared to the subsequent chunk sizes.
8. Therefore, while reading the rows from subsequent chunks we
   hit this imit on the input buffer.

FIX:

1. Each ddl thread is allocated a certain amount of input buffer
   based on the number of scan threads and the DDL buffer size.
   We should use this allocation as the limit for the input
   buffer while reading the rows from a chunk.

Change-Id: Iba4a224c9973bea2b5a6a2288a23473dd2c21b06

Author: Aditya A

storage/innobase/ddl/ddl0builder.cc

@@ -363,10 +363,6 @@ bool Merge_cursor::Compare::operator()(const File_cursor *lhs,
 dberr_t Merge_cursor::add_file(const ddl::file_t &file, size_t buffer_size,
                                const Range &range) noexcept {
   ut_a(file.m_file.is_open());
-  /* Keep the buffer size as much required to avoid the overlapping reads from
-  the subsequent ranges. In this iteration, buffer size would remain same for
-  subsequent reads */
-  buffer_size = std::min(size_t(range.second - range.first), buffer_size);
   auto cursor = ut::new_withkey<File_cursor>(
       ut::make_psi_memory_key(mem_key_ddl), m_builder, file.m_file, buffer_size,
       range, m_stage);

storage/innobase/ddl/ddl0file-reader.cc

@@ -79,11 +79,12 @@ dberr_t File_reader::prepare() noexcept {
     return DB_OUT_OF_MEMORY;
   }
 
+  /* Position the m_ptr to the first record. */
   if (const auto err = seek(); err != DB_SUCCESS) {
     return err;
   }
 
-  /* Position m_mrec on the first record. */
+  /* Position m_mrec on the first record and advance to next record. */
   return next();
 }
 
@@ -95,7 +96,7 @@ dberr_t File_reader::seek() noexcept {
       ddl::pread(m_file.get(), m_io_buffer.first, len, m_range.first);
 
   if (err == DB_SUCCESS) {
-    /* Fetch and advance to the next record. */
+    /* Fetch the record. */
     m_ptr = m_io_buffer.first;
   }