Skip to content

[WIP] CRON like scheduling / scheduler #100

Open
@treeder

Description

@treeder

Triggers functions on a repeating schedule.

Use new input format discussed recently and probably CRON format as well.


Proposal for cron support

Basic architecture sketch for each IronFunctions node:

                 ┌────────┐
     ┌──────────▶│D. Lock │
     │           └────────┘
     │
┌────────┐       ┌────────┐
│  Cron  │──────▶│   DB   │◀─────────────┐
└────────┘       └────────┘              │
     │                                   │
     │           ┌────────┐         ┌────────┐
     └──────────▶│   MQ   │◀────────│  HTTP  │
                 └────────┘         └────────┘
                      ▲                  │
                      │                  │
                      │                  ▼
                 ┌────────┐         ┌────────┐
                 │ Async  │         │  Sync  │
                 └────────┘         └────────┘

The two additional components are:

D. Lock is a some sort of distributed lock offered by latest versions of etcd
and Consul. In theory, any distributed lock would be enough in order to achieve
the necessary role in this design.

Cron is a gear similar to async. It enqueues a cron job while holding a
distributed mutex lock. Its implementation would be similar to the following:

lock()
jobs := cronjobs()
for _, job := range jobs {
	ls := laststart(j)
	if job.shouldRunAfter(ls) {
		ls.touch()
		enqueue(j)
	}
}
unlock()

In the DB, the new datastructure would be:

[{
	"id": "...",
	"function": {
		"app": "app",
		"path": "/fn",
		"headers": ["Context-Type: application/json"],
		"body": "....",
	}
	"frequency": "*/10 * * * *"
}, ...]

id is a unique identifier for the cronjob. It aims to allow repeated entries
for a same given function.

function is the description of the function to be ran. It will mimic a HTTP
POST call with headers and body. It also adds a User-Agent with the
specific information that this call is made by the cron worker.

frequency is a crond-like string that specifies when should this cronjob be
ran.

Restricting the distributed lock to etcd/consul and alikes, it means they can
also be used as an authoritative key-value source of truth regarding the last
execution.

laststart in the example code above, is the last time this cronjob was started,
regardless of success or failure.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions