Distributed Timer Use Case

[mksource]

Hi i am planning to use db-scheduler as a Distributed Timer. I will explain my use case. Please let me know if i can use db-scheduler for our use case.

1. There might 5K events per second. So each of the event has START and END Time. When the event starts a START message is thrown and when event ends the END messages. But each event might have a SLA. For example it should finish in say 5 minutes. If does not finish within a stipulated time then we need to throw an alert.

2. I am planning to use db-scheduler like a distributed timer. whenever a start of event message arrives we schedule a task execution. If the event end message arrives we plan to cancel the task execution. Otherwise the if it is late or does not arrive scheduler will pick up the task will execute and will generate alert message,

3. We are thinking this might go to 10K events per second and out of that say 1% might be delayed.

4. Can we db-scheduler handle this load. Any suggestions on this?

[mksource]

@kagkarlsson Can you please let me know

[kagkarlsson]

Hard to say. It might handle those volumes ok, but you might also hit a limit somewhere 5-10k events/s. Depends on how much load the db has, the specs etc. It might be possible to workaround by sharding over multiple databases should it be a problem.

I have reached 10k events/s using GCP Postgres instance (when I did some benchmarking)