Periodically log queue size #18
Description
We've confirmed in #16 that the maximum queue size has the potential of affecting overall performance. In fact, the number of items in the queue can eventually be used by the system to dynamically adjust step placement for handling an unstable input stream (a high number of items in the queue means that the system is having a hard time following the input stream). We should collect data on the number of items in the queue for our experiments to get more insights on how to use the information for dynamic step placement.
We could either create a separate log file for this information, or somehow embed it within the inference timing logs.