Problem: Packed Loads Format Could (Potentially) be even faster #84
Labels
benchmarking
Related to benchmarking performance
enhancement
New feature or request
low-priority
"Nice to have" but not necessary; prioritize lower
performance
A task related to assessing/enhancing performance
Milestone
Problem Description
The current packed loads format is something like this:
128 characters
However, we can incorporate a few simple ideas that should reduce the number of bytes required to write:
h
instead ofhours
(note: this may already work)Note that item 2 above only really helps when we write the "blank cells" using commas which is fairly cheap. In the above example, it is unnecessary since the longer loads are on the left. However, if the load-tag2 had been written first, the "blank cell" commas would have been necessary to write. If we had quite a few loads to write with differing sizes, the number of "blank cell commas" could become significant.
Here is the above format using these 3 principles:
105 characters
Strings can be read one line at a time. The last read time vector is assumed to be "active" until redefined. For reserving space for
std::vector
, we can read a single line and count the number of commas. The length of the last active time vector will be the length of all subsequent load vectors.NOTE: I did check and Microsoft Excel is able to display over 8,760 columns so a transposed packed load file could still be loaded to view in MS Excel.
Second Note: this is extremely unlikely to make much of a difference in terms of time or space (especially if the file is zipped). Also, two of the suggestions could be implemented without transposing the file. However, just want to jot down the idea while thinking of it.
The text was updated successfully, but these errors were encountered: