1
- RabbitMQ cli consumer
2
- ---------------------
1
+ # RabbitMQ cli consumer
3
2
4
- If you are a fellow PHP developer just like me you're probably aware of the following fact:
5
- PHP really SUCKS in long running tasks.
3
+ [ ![ Build Status] ( https://travis-ci.org/corvus-ch/rabbitmq-cli-consumer.svg?branch=master )] ( https://travis-ci.org/corvus-ch/rabbitmq-cli-consumer )
4
+ [ ![ Maintainability] ( https://api.codeclimate.com/v1/badges/392b42c2fe09633dfd30/maintainability )] ( https://codeclimate.com/github/corvus-ch/rabbitmq-cli-consumer/maintainability )
5
+ [ ![ Test Coverage] ( https://api.codeclimate.com/v1/badges/392b42c2fe09633dfd30/test_coverage )] ( https://codeclimate.com/github/corvus-ch/rabbitmq-cli-consumer/test_coverage )
6
6
7
- When using RabbitMQ with pure PHP consumers you have to deal with stability issues. Probably you are killing your
8
- consumers regularly. And try to solve the problem with supervisord. Which also means on every deploy you
9
- have to restart your consumers. A little bit dramatic if you ask me.
7
+ If you are a fellow PHP developer just like me you're probably aware of the
8
+ following fact: [ PHP is meant to die] [ die ] .
10
9
11
- This is a fork of the work done by Richard van den Brand and provides a command that aims to solve the above described
12
- problem for RabbitMQ workers by delegate the long running part to a tool written in go which is much better suited for
13
- this task. The PHP application then is only executed when there is an AMQP message to process.
10
+ When using RabbitMQ with pure PHP consumers you have to deal with stability
11
+ issues. Probably you are killing your consumers regularly. And try to solve the
12
+ problem with supervisord. Which also means on every deploy you have to restart
13
+ your consumers. A little bit dramatic if you ask me.
14
14
15
- This fork came to be, when Richard van den Brand did no longer had the time to maintain his version. Simplification is
16
- one of the main goal of this fork and no effort will be made to keep the changes backwards compatible.
15
+ This is a fork of the work done by Richard van den Brand and provides a command
16
+ that aims to solve the above described problem for RabbitMQ workers by delegate
17
+ the long running part to a tool written in go which is much better suited for
18
+ this task. The PHP application then is only executed when there is an AMQP
19
+ message to process. This is comparable to how HTTP requests usually are handled
20
+ where the webs server waits for new incoming requests and calls your script once
21
+ for each request.
17
22
18
- - Environment dependent settings should be configurable by environment variables.
19
- - All logs, including the output of the called scripts, will be written to STDOUT/STDERR.
20
- - The AMQP message will be passed via STDIN (not as argument limitation in size)
23
+ This fork came to be, when [ Richard van den Brand] [ ricbra ] did no longer had
24
+ the time to maintain his version. The main goals of the fork are:
21
25
22
- WARNING: This is work in progress and not yet intended to be used in production.
26
+ - Following the principle of [ The Twelve-Factor App] [ 12factor ] environment
27
+ dependent settings should be configurable by environment variables.
28
+ - All logs, including the output of the called executable, will be available in
29
+ STDOUT/STDERR.
30
+ - The AMQP message will be passed via STDIN (not as argument with its
31
+ limitation in size)
32
+ - Have tests with a decent level of code coverage.
23
33
24
- # Installation
34
+ NOTE: If you previously used the consumer of [ Richard van den Brand] [ ricbra ] ,
35
+ this version should work as a drop in replacement. Do not migrate blindly but
36
+ do some testing before. Effort was made to remain backwards compatible, no
37
+ guarantees are made.
25
38
26
- You have the choice to either compile yourself or by installing via package or binary.
39
+ ## Installation
27
40
28
- ## APT Package
29
-
30
- As I'm a Debian user myself Debian-based peeps are lucky and can use my APT repository.
31
-
32
- Add this line to your <code >/etc/apt/sources.list</code > file:
33
-
34
- deb http://apt.vandenbrand.org/debian testing main
35
-
36
- Fetch and install GPG key:
37
-
38
- $ wget http://apt.vandenbrand.org/apt.vandenbrand.org.gpg.key
39
- $ sudo apt-key add apt.vandenbrand.org.gpg.key
40
-
41
- Update and install:
42
-
43
- $ sudo apt-get update
44
- $ sudo apt-get install rabbitmq-cli-consumer
45
-
46
- ## Create .deb package for service install
47
-
48
- sudo apt-get install golang gccgo-go ruby -y
49
- # Ubuntu
50
- sudo apt-get install gccgo-go -y
51
- # Debian
52
- sudo apt-get install gccgo -y
53
- sudo gem install fpm
54
- ./build_service_deb.sh
41
+ You have the choice to either compile yourself or by installing via package or
42
+ binary.
55
43
56
44
## Binary
57
45
58
- Binaries can be found at: https://github.com/ricbra /rabbitmq-cli-consumer/releases
46
+ Binaries can be found at: https://github.com/corvus-ch /rabbitmq-cli-consumer/releases
59
47
60
- ## Compiling
48
+ ### Compiling
61
49
62
50
This section assumes you're familiar with the Go language.
63
51
@@ -86,99 +74,32 @@ $ go build
86
74
$ go install
87
75
```
88
76
89
- # Usage
90
-
91
- Run without arguments or with <code >--help</code > switch to show the helptext:
77
+ ## Usage
92
78
93
- $ rabbitmq-cli-consumer
94
- NAME:
95
- rabbitmq-cli-consumer - Consume RabbitMQ easily to any cli program
79
+ rabbitmq-cli-consumer --verbose --url amqp://guest:guest@localhost --queue myqueue --executable '/path/to/your/app argument --flag'
96
80
97
- USAGE:
98
- rabbitmq-cli-consumer [global options] command [command options] [arguments...]
81
+ Run without arguments or with ` -help ` switch to show the helptext:
99
82
100
- VERSION:
101
- 0.0.1
102
-
103
- AUTHOR:
104
- Richard van den Brand - <[email protected] >
105
-
106
- COMMANDS:
107
- help, h Shows a list of commands or help for one command
108
-
109
- GLOBAL OPTIONS:
110
- --executable, -e Location of executable
111
- --configuration, -c Location of configuration file
112
- --verbose, -V Enable verbose mode (logs to stdout and stderr)
113
- --include, -i Include metadata. Passes message as JSON data including headers, properties and message body.
114
- --help, -h show help
115
- --version, -v print the version
116
-
117
- ## Configuration
118
-
119
- A configuration file is required. Example:
120
-
121
- ``` ini
122
- [rabbitmq]
123
- host = localhost
124
- username = username-of-rabbitmq-user
125
- password = secret
126
- vhost =/your-vhost
127
- port =5672
128
- queue =name-of-queue
129
- compression =Off
130
-
131
- [logs]
132
- error = /location/to/error.log
133
- info = /location/to/info.log
134
- ```
83
+ ### Configuration
135
84
136
- When you've created the configuration you can start the consumer like this:
85
+ The file ` example.com ` contains all available configuration options together
86
+ with its explanation.
137
87
138
- $ rabbitmq-cli-consumer -e "/path/to/your/app argument --flag" -c /path/to/your/configuration.conf -V
88
+ In Go the zero value for a string is ` "" ` . So, any values not configured in the
89
+ config file will result in a empty string. Now imagine you want to define an
90
+ empty name for one of the configuration settings. Yes, we now cannot determine
91
+ whether this value was empty on purpose or just left out. If you want to
92
+ configure an empty string you have to be explicit by using the value ` <empty> ` .
139
93
140
- Run without <code >-V</code > to get rid of the output:
141
-
142
- $ rabbitmq-cli-consumer -e "/path/to/your/app argument --flag" -c /path/to/your/configuration.conf
143
-
144
- ### Prefetch count
145
-
146
- It's possible to configure the prefetch count and if you want set it as global. Add the following section to your
147
- configuration to confol these values:
148
-
149
- ``` ini
150
- [prefetch]
151
- count =3
152
- global =Off
153
- ```
154
-
155
- ### Configuring the exchange
156
-
157
- It's also possible to configure the exchange and its options. When left out in the configuration file, the default
158
- exchange will be used. To configure the exchange add the following to your configuration file:
159
-
160
- ``` ini
161
- [exchange]
162
- name =mail
163
- autodelete =Off
164
- type =direct
165
- durable =On
166
- ```
167
-
168
- ### How to configure an empty string value
169
-
170
- In Go the zero value for a string is ` "" ` . So, any values not configured in the config file will result in a
171
- empty string. Now imagine you want to define an empty name for one of the configuration settings. Yes, we now
172
- cannot determine whether this value was empty on purpose or just left out. If you want to configure an empty string
173
- you have to be explicit by using the value ` <empty> ` .
94
+ rabbitmq-cli-consumer --verbose --url amqp://guest: guest @localhost --queue myqueue --executable command.php --configuration example.conf
174
95
175
96
## The executable
176
97
177
98
Your executable receives the message as the last argument. So consider the following:
178
99
179
- $ rabbitmq-cli-consumer -e "/home/vagrant/current/app/command.php" -c example.conf -V
100
+ rabbitmq-cli-consumer --verbose --url amqp://guest : guest @localhost --queue myqueue --executable command.php
180
101
181
- The < code > command.php</ code > file should look like this:
102
+ The ` command.php ` file should look like this:
182
103
183
104
``` php
184
105
#!/usr/bin/env php
@@ -202,7 +123,7 @@ exit(1);
202
123
203
124
Or a Symfony2 example:
204
125
205
- $ rabbitmq-cli-consumer -e "/path/to/symfony/ app/console event:processing -e =prod" -c example.conf -V
126
+ rabbitmq-cli-consumer --verbose --url amqp://guest:guest@localhost --queue myqueue --executable ' app/console event:processing --env =prod'
206
127
207
128
Command looks like this:
208
129
@@ -238,30 +159,23 @@ class TestCommand extends ContainerAwareCommand
238
159
}
239
160
```
240
161
241
- ## Compression
162
+ ### Compression
242
163
243
- Depending on what you're passing around on the queue, it may be wise to enable compression support. If you don't you may
244
- encouter the infamous "Argument list too long" error.
164
+ Depending on what you're passing around on the queue, it may be wise to enable
165
+ compression support. If you don't you may encouter the infamous "Argument list
166
+ too long" error.
245
167
246
- When compression is enabled, the message gets compressed with zlib maximum compression before it's base64 encoded. We
247
- have to pay a performance penalty for this. If you are serializing large php objects I suggest to turn it on. Better
248
- safe then sorry.
168
+ When compression is enabled, the message gets compressed with zlib maximum
169
+ compression before it's base64 encoded. We have to pay a performance penalty
170
+ for this. If you are serializing large php objects I suggest to turn it on.
171
+ Better safe then sorry.
249
172
250
173
In your config:
251
174
252
175
``` ini
253
176
[rabbitmq]
254
- host = localhost
255
- username = username-of-rabbitmq-user
256
- password = secret
257
- vhost =/your-vhost
258
- port =5672
259
- queue =name-of-queue
260
- compression =On
261
-
262
- [logs]
263
- error = /location/to/error.log
264
- info = /location/to/info.log
177
+ compression = On
178
+
265
179
```
266
180
267
181
And in your php app:
@@ -292,13 +206,13 @@ exit(1);
292
206
293
207
```
294
208
295
- ## Including properties and message headers
209
+ ### Including properties and message headers
296
210
297
211
298
- If you need to access message headers or properties, call the command with the
299
- ` --include, -i ` option set.
212
+ If you need to access message headers and or properties, call the command with
213
+ the ` --include ` option set.
300
214
301
- $ rabbitmq-cli-consumer -e "/home/vagrant/current/app/command.php" -c example.conf -i
215
+ rabbitmq-cli-consumer --verbose --url amqp://guest:guest@localhost --queue myqueue --executable command.php --include
302
216
303
217
The script then will receive a json encoded data structure which looks like
304
218
the following.
@@ -356,8 +270,8 @@ if (do_heavy_lifting($data->body, $data->properties)) {
356
270
exit(1);
357
271
```
358
272
359
- If you are using symfonies RabbitMQ bundle (` oldsound /rabbitmq-bundle` ) you can
360
- wrap the consumer with the following symfony command.
273
+ If you are using symfonies RabbitMQ bundle (` php-amqplib /rabbitmq-bundle` ) you
274
+ can wrap the consumer with the following symfony command.
361
275
362
276
``` php
363
277
<?php
@@ -396,12 +310,15 @@ class TestCommand extends ContainerAwareCommand
396
310
}
397
311
```
398
312
399
- ## Use pipes instead of arguments
313
+ ### Use pipes instead of arguments
400
314
401
- When starting the consymer with the ` -pipes ` option, the AMQP message will be
315
+ When starting the consumer with the ` - -pipes` option, the AMQP message will be
402
316
passed on to the executable using STDIN for the message body and fd3 for the
403
317
metadata containing the properties and the delivery info encoded as JSON.
404
318
319
+ rabbitmq-cli-consumer --verbose --url amqp://guest:guest@localhost --queue myqueue --executable command.php --pipe
320
+
321
+
405
322
``` php
406
323
#!/usr/bin/env php
407
324
<?php
@@ -430,11 +347,16 @@ if (false === $body) {
430
347
431
348
```
432
349
433
- # Strict exit code processing
350
+ ### Strict exit code processing
434
351
435
- By default, any non-zero exit code will make consumer send a negative acknowledgement and re-queue message back to the queue, in some cases it may cause your consumer to fall into an infinite loop as re-queued message will be getting back to consumer and it probably will fail again.
352
+ By default, any non-zero exit code will make consumer send a negative
353
+ acknowledgement and re-queue message back to the queue, in some cases it may
354
+ cause your consumer to fall into an infinite loop as re-queued message will be
355
+ getting back to consumer and it probably will fail again.
436
356
437
- It's possible to get better control over message acknowledgement by setting up strict exit code processing. In this mode consumer will acknowledge messages only if executable process return an allowed exit code.
357
+ It's possible to get better control over message acknowledgement by setting up
358
+ strict exit code processing. In this mode consumer will acknowledge messages
359
+ only if executable process return an allowed exit code.
438
360
439
361
** Allowed exit codes**
440
362
@@ -450,7 +372,7 @@ All other exit codes will cause consumer to fail.
450
372
451
373
Run consumer with ` --strict-exit-code ` option to enable strict exit code processing:
452
374
453
- $ rabbitmq-cli-consumer -e "/path/to/your/app argument --flag" -c /path/to/your/configuration.conf --strict-exit-code
375
+ rabbitmq-cli-consumer --verbose --url amqp://guest:guest@localhost --queue myqueue --executable command.php --strict-exit-code
454
376
455
377
Make sure your executable returns correct exit code
456
378
@@ -470,8 +392,15 @@ try {
470
392
} catch(Exception $e) {
471
393
exit(1); // Unexpected exception will cause consumer to stop consuming
472
394
}
395
+
473
396
```
474
397
475
- # Developing
398
+ ## Contributing and license
399
+
400
+ This library is licenced under [ MIT] ( LICENSE.md ) . For information about how to
401
+ contribute to this project, see [ CONTRIBUTING.md] .
476
402
477
- Missing anything? Found a bug? I love to see your PR.
403
+ [ 12factor ] : https://12factor.net
404
+ [ CONTRIBUTING.md ] : https://github.com/corvus-ch/rabbitmq-cli-consumer/blob/master/CONTRIBUTING.md
405
+ [ die ] : https://software-gunslinger.tumblr.com/post/47131406821/php-is-meant-to-die
406
+ [ ricbra ] : https://github.com/ricbra
0 commit comments