Add a dataloader class for shuffled multiple sequences #134
Conversation
exchange scipy.stats.pearsonr by np.corrcoef and speed up with numba.
changed field.name to f.name
BUGFIX _create_config
Version bump v0.3.0
for multiple patterns. Feels a bit like an overkill. :P
and teh respective tests for the ShuffleDataloader
timomit
force-pushed
the
FeatureMultiPattern
branch
from
c5c464a to
40e9b5f
Compare
Benano
left a comment
Benano
left a comment
There was a problem hiding this comment.
Very nicely written code of an elegant solution!
| # self.pattern[0].transform(pre_transforms[0]) | ||
| # print(self.pattern[0]) | ||
|
|
||
| self._rng = np.random.default_rng(seed) |
There was a problem hiding this comment.
I remember J telling me that its good practice to define the randomness generator outside of the class and to pass the generator rather than passing a seed.
|
|
||
| # now, fill the arrays until t_max | ||
| # we do this, because the pattern can have arbitrary lengths: | ||
| while self._starting_times[-1] < t_max: |
There was a problem hiding this comment.
I like this approach a lot. Having all of the starting times is super handy!
| current_pat_idx = self.choice() | ||
| self._pat_ids.append(current_pat_idx) | ||
| self._starting_times.append( | ||
| self._starting_times[-1] + self.durations[self._pat_ids[-2]] |
There was a problem hiding this comment.
I don't quite understand what is happening here. Why are you adding only the duration of the second to last pattern? Lets quickly discuss.
| Apply online transformations to a pattern. | ||
| This method applies all specified online transformations in sequence | ||
| to the given 1-dimensional pattern. |
There was a problem hiding this comment.
Here I was confused for a moment why it has to be a 1D pattern but I understand now thats it is a 1D slice of the pattern. Consider renaming :)
| raise ValueError("t must be small smaller than t_max.") | ||
| # find index of the pattern that is at t: | ||
| t += offset | ||
| idx_in_sequence = np.searchsorted(self._starting_times, t) - 1 |
There was a problem hiding this comment.
What a handy function! I did not know about it.
2 participants
the dataloader takes a list of pattern. These pattern are internally randomly shuffled and loaded in a lazy way.