exponential backoff

[straeter]

Proposed change description

Hey there, great work!

As far as I can tell, there is a fixed waiting time for re-querying of 30 seconds -> "wait_fixed(30)" and this cannot be changed in the client.

Semanticscholar requires exponential back-off for the api usage (you have to agree to this if you apply for a api key) and this might also be a benefitial feature and could save a lot of time (like first retry after 5 seconds, then 10, 20 etc.). An argument like wait_type = fixed or exponential and a wait_time argument (which could be the base time for the exponential back-off) might be a nice addition and could be easily implemented with wait_exponential form tenecity.

I could also work on this

[danielnsilva]

As far as I can tell, there is a fixed waiting time for re-querying of 30 seconds -> "wait_fixed(30)" and this cannot be changed in the client.

Right now, the only option is to disable retry mode, so it just makes a single attempt.

Semanticscholar requires exponential back-off for the api usage (you have to agree to this if you apply for a api key) and this might also be a benefitial feature and could save a lot of time (like first retry after 5 seconds, then 10, 20 etc.). An argument like wait_type = fixed or exponential and a wait_time argument (which could be the base time for the exponential back-off) might be a nice addition and could be easily implemented with wait_exponential form tenecity.

Great suggestion! Since it's a good practice, exponential back-off should be the default.

I could also work on this

Contributions are welcome. Thanks!

[dbosk]

From the tenacity documentation:

@retry(wait=wait_exponential(multiplier=1, min=4, max=10))
def wait_exponential_1():
    print("Wait 2^x * 1 second between each retry starting with 4 seconds, then up to 10 seconds, then 10 seconds afterwards")
    raise Exception

[dbosk]

Please test #108.