|
| 1 | +*For an overview of how to __use__ php-resque, see `README.md`.* |
| 2 | + |
| 3 | +The following is a step-by-step breakdown of how php-resque operates. |
| 4 | + |
| 5 | +## Enqueue Job ## |
| 6 | + |
| 7 | +What happens when you call `Resque::enqueue()`? |
| 8 | + |
| 9 | +1. `Resque::enqueue()` calls `Resque_Job::create()` with the same arguments it |
| 10 | + received. |
| 11 | +2. `Resque_Job::create()` checks that your `$args` (the third argument) are |
| 12 | + either `null` or in an array |
| 13 | +3. `Resque_Job::create()` generates a job ID (a "token" in most of the docs) |
| 14 | +4. `Resque_Job::create()` pushes the job to the requested queue (first |
| 15 | + argument) |
| 16 | +5. `Resque_Job::create()`, if status monitoring is enabled for the job (fourth |
| 17 | + argument), calls `Resque_Job_Status::create()` with the job ID as its only |
| 18 | + argument |
| 19 | +6. `Resque_Job_Status::create()` creates a key in Redis with the job ID in its |
| 20 | + name, and the current status (as well as a couple of timestamps) as its |
| 21 | + value, then returns control to `Resque_Job::create()` |
| 22 | +7. `Resque_Job::create()` returns control to `Resque::enqueue()`, with the job |
| 23 | + ID as a return value |
| 24 | +8. `Resque::enqueue()` triggers the `afterEnqueue` event, then returns control |
| 25 | + to your application, again with the job ID as its return value |
| 26 | + |
| 27 | +## Workers At Work ## |
| 28 | + |
| 29 | +How do the workers process the queues? |
| 30 | + |
| 31 | +1. `Resque_Worker::work()`, the main loop of the worker process, calls |
| 32 | + `Resque_Worker->reserve()` to check for a job |
| 33 | +2. `Resque_Worker->reserve()` checks whether to use blocking pops or not (from |
| 34 | + `BLOCKING`), then acts accordingly: |
| 35 | + * Blocking Pop |
| 36 | + 1. `Resque_Worker->reserve()` calls `Resque_Job::reserveBlocking()` with |
| 37 | + the entire queue list and the timeout (from `INTERVAL`) as arguments |
| 38 | + 2. `Resque_Job::reserveBlocking()` calls `Resque::blpop()` (which in turn |
| 39 | + calls Redis' `blpop`, after prepping the queue list for the call, then |
| 40 | + processes the response for consistency with other aspects of the |
| 41 | + library, before finally returning control [and the queue/content of the |
| 42 | + retrieved job, if any] to `Resque_Job::reserveBlocking()`) |
| 43 | + 3. `Resque_Job::reserveBlocking()` checks whether the job content is an |
| 44 | + array (it should contain the job's type [class], payload [args], and |
| 45 | + ID), and aborts processing if not |
| 46 | + 4. `Resque_Job::reserveBlocking()` creates a new `Resque_Job` object with |
| 47 | + the queue and content as constructor arguments to initialize the job |
| 48 | + itself, and returns it, along with control of the process, to |
| 49 | + `Resque_Worker->reserve()` |
| 50 | + * Queue Polling |
| 51 | + 1. `Resque_Worker->reserve()` iterates through the queue list, calling |
| 52 | + `Resque_Job::reserve()` with the current queue's name as the sole |
| 53 | + argument on each pass |
| 54 | + 2. `Resque_Job::reserve()` passes the queue name on to `Resque::pop()`, |
| 55 | + which in turn calls Redis' `lpop` with the same argument, then returns |
| 56 | + control (and the job content, if any) to `Resque_Job::reserve()` |
| 57 | + 3. `Resque_Job::reserve()` checks whether the job content is an array (as |
| 58 | + before, it should contain the job's type [class], payload [args], and |
| 59 | + ID), and aborts processing if not |
| 60 | + 4. `Resque_Job::reserve()` creates a new `Resque_Job` object in the same |
| 61 | + manner as above, and also returns this object (along with control of |
| 62 | + the process) to `Resque_Worker->reserve()` |
| 63 | +3. In either case, `Resque_Worker->reserve()` returns the new `Resque_Job` |
| 64 | + object, along with control, up to `Resque_Worker::work()`; if no job is |
| 65 | + found, it simply returns `FALSE` |
| 66 | + * No Jobs |
| 67 | + 1. If blocking mode is not enabled, `Resque_Worker::work()` sleeps for |
| 68 | + `INTERVAL` seconds; it calls `usleep()` for this, so fractional seconds |
| 69 | + *are* supported |
| 70 | + * Job Reserved |
| 71 | + 1. `Resque_Worker::work()` triggers a `beforeFork` event |
| 72 | + 2. `Resque_Worker::work()` calls `Resque_Worker->workingOn()` with the new |
| 73 | + `Resque_Job` object as its argument |
| 74 | + 3. `Resque_Worker->workingOn()` does some reference assignments to help keep |
| 75 | + track of the worker/job relationship, then updates the job status from |
| 76 | + `WAITING` to `RUNNING` |
| 77 | + 4. `Resque_Worker->workingOn()` stores the new `Resque_Job` object's payload |
| 78 | + in a Redis key associated to the worker itself (this is to prevent the job |
| 79 | + from being lost indefinitely, but does rely on that PID never being |
| 80 | + allocated on that host to a different worker process), then returns control |
| 81 | + to `Resque_Worker::work()` |
| 82 | + 5. `Resque_Worker::work()` forks a child process to run the actual `perform()` |
| 83 | + 6. The next steps differ between the worker and the child, now running in |
| 84 | + separate processes: |
| 85 | + * Worker |
| 86 | + 1. The worker waits for the job process to complete |
| 87 | + 2. If the exit status is not 0, the worker calls `Resque_Job->fail()` with |
| 88 | + a `Resque_Job_DirtyExitException` as its only argument. |
| 89 | + 3. `Resque_Job->fail()` triggers an `onFailure` event |
| 90 | + 4. `Resque_Job->fail()` updates the job status from `RUNNING` to `FAILED` |
| 91 | + 5. `Resque_Job->fail()` calls `Resque_Failure::create()` with the job |
| 92 | + payload, the `Resque_Job_DirtyExitException`, the internal ID of the |
| 93 | + worker, and the queue name as arguments |
| 94 | + 6. `Resque_Failure::create()` creates a new object of whatever type has |
| 95 | + been set as the `Resque_Failure` "backend" handler; by default, this is |
| 96 | + a `Resque_Failure_Redis` object, whose constructor simply collects the |
| 97 | + data passed into `Resque_Failure::create()` and pushes it into Redis |
| 98 | + in the `failed` queue |
| 99 | + 7. `Resque_Job->fail()` increments two failure counters in Redis: one for |
| 100 | + a total count, and one for the worker |
| 101 | + 8. `Resque_Job->fail()` returns control to the worker (still in |
| 102 | + `Resque_Worker::work()`) without a value |
| 103 | + * Job |
| 104 | + 1. The job calls `Resque_Worker->perform()` with the `Resque_Job` as its |
| 105 | + only argument. |
| 106 | + 2. `Resque_Worker->perform()` sets up a `try...catch` block so it can |
| 107 | + properly handle exceptions by marking jobs as failed (by calling |
| 108 | + `Resque_Job->fail()`, as above) |
| 109 | + 3. Inside the `try...catch`, `Resque_Worker->perform()` triggers an |
| 110 | + `afterFork` event |
| 111 | + 4. Still inside the `try...catch`, `Resque_Worker->perform()` calls |
| 112 | + `Resque_Job->perform()` with no arguments |
| 113 | + 5. `Resque_Job->perform()` calls `Resque_Job->getInstance()` with no |
| 114 | + arguments |
| 115 | + 6. If `Resque_Job->getInstance()` has already been called, it returns the |
| 116 | + existing instance; otherwise: |
| 117 | + 7. `Resque_Job->getInstance()` checks that the job's class (type) exists |
| 118 | + and has a `perform()` method; if not, in either case, it throws an |
| 119 | + exception which will be caught by `Resque_Worker->perform()` |
| 120 | + 8. `Resque_Job->getInstance()` creates an instance of the job's class, and |
| 121 | + initializes it with a reference to the `Resque_Job` itself, the job's |
| 122 | + arguments (which it gets by calling `Resque_Job->getArguments()`, which |
| 123 | + in turn simply returns the value of `args[0]`, or an empty array if no |
| 124 | + arguments were passed), and the queue name |
| 125 | + 9. `Resque_Job->getInstance()` returns control, along with the job class |
| 126 | + instance, to `Resque_Job->perform()` |
| 127 | + 10. `Resque_Job->perform()` sets up its own `try...catch` block to handle |
| 128 | + `Resque_Job_DontPerform` exceptions; any other exceptions are passed |
| 129 | + up to `Resque_Worker->perform()` |
| 130 | + 11. `Resque_Job->perform()` triggers a `beforePerform` event |
| 131 | + 12. `Resque_Job->perform()` calls `setUp()` on the instance, if it exists |
| 132 | + 13. `Resque_Job->perform()` calls `perform()` on the instance |
| 133 | + 14. `Resque_Job->perform()` calls `tearDown()` on the instance, if it |
| 134 | + exists |
| 135 | + 15. `Resque_Job->perform()` triggers an `afterPerform` event |
| 136 | + 16. The `try...catch` block ends, suppressing `Resque_Job_DontPerform` |
| 137 | + exceptions by returning control, and the value `FALSE`, to |
| 138 | + `Resque_Worker->perform()`; any other situation returns the value |
| 139 | + `TRUE` along with control, instead |
| 140 | + 17. The `try...catch` block in `Resque_Worker->perform()` ends |
| 141 | + 18. `Resque_Worker->perform()` updates the job status from `RUNNING` to |
| 142 | + `COMPLETE`, then returns control, with no value, to the worker (again |
| 143 | + still in `Resque_Worker::work()`) |
| 144 | + 19. `Resque_Worker::work()` calls `exit(0)` to terminate the job process |
| 145 | + cleanly |
| 146 | + * SPECIAL CASE: Non-forking OS (Windows) |
| 147 | + 1. Same as the job above, except it doesn't call `exit(0)` when done |
| 148 | + 7. `Resque_Worker::work()` calls `Resque_Worker->doneWorking()` with no |
| 149 | + arguments |
| 150 | + 8. `Resque_Worker->doneWorking()` increments two processed counters in Redis: |
| 151 | + one for a total count, and one for the worker |
| 152 | + 9. `Resque_Worker->doneWorking()` deletes the Redis key set in |
| 153 | + `Resque_Worker->workingOn()`, then returns control, with no value, to |
| 154 | + `Resque_Worker::work()` |
| 155 | +4. `Resque_Worker::work()` returns control to the beginning of the main loop, |
| 156 | + where it will wait for the next job to become available, and start this |
| 157 | + process all over again |
0 commit comments