configure mem pool buckets rotation

[HagarMeir]

#269

[tock-ibm]

Now the mean of the epoch is longer than FirstStrikeThreshold / 10; in fact it is FirstStrikeThreshold / 10 +25ms. I don't think this is the right approach.

Draw a random number d whose range is [-D,D]. Then add the "dither" d when you compute if a first strike had happened, e.g. in here:

func (ps *PendingStore) checkFirstStrike(now time.Time)

if now.Sub(bucket.lastTimestamp) <= ps.FirstStrikeThreshold {    <<<
			continue
		}

by adding d to the ps.FirstStrikeThreshold.

In any event, this is not expected to help much if d is much smaller than the batching interval. If the first TXs forwarded to the primary (by the secondary who strikes first) don't return in a batch by the time the next secondary batcher strikes, they will be forwarded anyway....

It seems to me a good choice for D would be around 2*BatchInterval or 3*BatchInterval.... what do you think?

[HagarMeir]

ok

[tock-ibm]

This is adding randomness only once, when the pool is created. Wouldn't it make more sense to do it per bucket, or would that be too much? what do you think?

[HagarMeir]

I think it will be too much

[tock-ibm]

Lets make sure that in the worse case the first strike is still > BatchTimeout. IE compare the BatchTimeout with the FirstStrikeThreshold. It seems to me the FirstStrikeThreshold should be at least 5*BatchTimeout or so...

Even without randomization, if FirstStrikeThreshold < BatchTimeout + network-delay we'll get constant forwarding. So some sanity checks are required.

[HagarMeir]

Will add checks
Currently the batch timeout is 500ms and we are running tests with first strike of 2s

[tock-ibm]

We park this one until we return and implement a robust comprehensive method of dealing with Router failure. We can then evaluate this technique here relative or in combination with the comprehensive approach.