Skip to content
This repository was archived by the owner on Feb 21, 2025. It is now read-only.

[WIP] Version 2.0.x #13

Open
wants to merge 37 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
2aeb815
Added instructions on how to start RabbitMQ in docker container
ricbra May 22, 2015
b3f3b4a
Added some more instructions about docker
ricbra May 23, 2015
44aacd8
Changed version to 2.0.0-dev
ricbra Aug 16, 2015
3dcc503
Fixed identation in README.md
ricbra Aug 16, 2015
0c49850
Added FileLocator and tests
ricbra Aug 17, 2015
09cd3ab
First take on config merger
ricbra Aug 18, 2015
ad00379
Added config merger and test
ricbra Aug 24, 2015
868e5d6
Rewrote config
ricbra Aug 31, 2015
97c5b75
Added default config
ricbra Sep 2, 2015
ed14f0a
Added Travis CI config
ricbra Sep 2, 2015
38c742e
Added Travis CI button
ricbra Sep 2, 2015
66ce75f
Fixed app after config refactoring
ricbra Sep 2, 2015
770a1e4
Changed verbose to quiet
ricbra Sep 2, 2015
efb33c2
Started rewrite of consumer
ricbra Sep 3, 2015
fdbc2de
Added QoS tests
ricbra Sep 23, 2015
c97af6e
Added TestDeclareQueueFails
ricbra Sep 23, 2015
05e4414
Added TestDeclareQueueSucceeds
ricbra Sep 23, 2015
bb7c464
Removed debug info
ricbra Sep 23, 2015
13f6b1b
Added TestBindQueueFails
ricbra Sep 23, 2015
11c4f0b
Added TestBindQueueSucceeds
ricbra Sep 23, 2015
637d73a
Cleanup of tests
ricbra Sep 23, 2015
30c8d3a
Added tests for CommandExecuter
ricbra Oct 13, 2015
5511565
Added test for processing of messages
ricbra Oct 18, 2015
7ff6b59
Refactored queue to its own config section
ricbra Oct 26, 2015
7a5876e
Made all queue params configurable while we're at it
ricbra Oct 26, 2015
7fa1fa9
Added RPC support
ricbra Nov 23, 2015
a5af4d2
Added RPC example and more stuff in docs
ricbra Nov 25, 2015
c1eaada
Routing key is now configurable
ricbra Nov 25, 2015
97b908f
Allow all options to be configured from cli
ricbra Nov 30, 2015
afae7c5
Updated README.md with new config options
ricbra Nov 30, 2015
bda2b19
Queue TTL is now configurable
ricbra Dec 2, 2015
f070bd4
Added some more docs
ricbra Dec 2, 2015
09f2fbf
Implemented logrotate on USR1 signal
ricbra Dec 4, 2015
aa94c90
Added small section in README.md for log rotation
ricbra Dec 4, 2015
f6aad0f
Fixed the import statements on the main.go and config.go files since …
Aug 23, 2016
76d75e5
Added the .gitignore file to the repository to ensure we do not commi…
Sep 13, 2016
eefaeeb
Merge pull request #41 from miguelsimoes/fixes-address-for-google-pac…
ricbra Sep 13, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/rabbitmq-cli-consumer
10 changes: 10 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
language: go

go:
- 1.4
- 1.5
- tip

matrix:
allow_failures:
- go: tip
151 changes: 148 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
RabbitMQ cli consumer
---------------------

[![Build Status](https://travis-ci.org/ricbra/rabbitmq-cli-consumer.svg)](https://travis-ci.org/ricbra/rabbitmq-cli-consumer)

If you are a fellow PHP developer just like me you're probably aware of the following fact:
PHP really SUCKS in long running tasks.

Expand Down Expand Up @@ -103,7 +105,60 @@ Run without arguments or with <code>--help</code> switch to show the helptext:
--help, -h show help
--version, -v print the version

## Configuration
## Fanout

Todo.

## Remote Procedure Call

No special configuration is required for enabling RPC mode. You should be aware
that any output on STDOUT will be returned as reply to the requesting client. To
demonstrate how RPC works, we'll implement [the example](https://www.rabbitmq.com/tutorials/tutorial-six-php.html) on the RabbitMQ site
using rabbitmq-cli-consumer.

We don't change the <code>rpc_client.php</code>, only the <code>rpc_server.php</code>:

```php
<?php

function fib($n) {
if ($n == 0)
return 0;
if ($n == 1)
return 1;
return fib($n-1) + fib($n-2);
}

echo fib(base64_decode($argv[1]));
// Return exit(1); if something goes wrong, msg will be requeued
exit(0);
```

Configuration for the consumer:

```ini
[rabbitmq]
...

[prefetch]
count=1
global=Off

[queue]
name=rpc_queue
durable=On
autodelete=Off
exclusive=Off
nowait=Off

[exchange]
name=rpc_queue
autodelete=Off
type=direct
durable=On
```

# Configuration

A configuration file is required. Example:

Expand All @@ -114,9 +169,11 @@ username = username-of-rabbitmq-user
password = secret
vhost=/your-vhost
port=5672
queue=name-of-queue
compression=Off

[queue]
nane=your-queue-name

[logs]
error = /location/to/error.log
info = /location/to/info.log
Expand All @@ -130,10 +187,27 @@ Run without <code>-V</code> to get rid of the output:

$ rabbitmq-cli-consumer -e "/path/to/your/app argument --flag" -c /path/to/your/configuration.conf

## Configuration inheritance

By default rabbitmq-cli-consumer looks for configuration files in the following
order:

1. /etc/rabbitmq-cli-consumer/rabbitmq-cli-consumer.conf
2. ~/.rabbitmq-cli-consumer.conf
3. location passed in via <code>-c</code> option

## Override options on the command line:

Every option in the configuration file can be overwritten by passing them on the
cli. Use <code>On</code> and <code>Off</code> for <code>true</code> and <code>false</code>
respectively. Example:

$ rabbitmq-cli-consumer -c /you/config.conf -e "/path/to/executble" --queue-key=custom-key

### Prefetch count

It's possible to configure the prefetch count and if you want set it as global. Add the following section to your
configuration to confol these values:
configuration to control these values:

```ini
[prefetch]
Expand All @@ -154,6 +228,43 @@ type=direct
durable=On
```

### Configuring the queue

All queue options are configurable. Full example:

```ini
[queue]
name=rpc_queue
durable=On
autodelete=Off
exclusive=Off
nowait=Off
```

If you want to define a TTL for your queue:

```ini
[queue]
name=rpc_queue
durable=On
autodelete=Off
exclusive=Off
nowait=Off
ttl=1200
```

When you want to configure the routing key:

```ini
[queue]
name=rpc_queue
durable=On
autodelete=Off
exclusive=Off
nowait=Off
key=your-routing-key
```

## The executable

Your executable receives the message as the last argument. So consider the following:
Expand Down Expand Up @@ -274,8 +385,42 @@ exit(1);

```

## Log rotation

To close and reopen the logs send the USR1 signal:

$ kill -s USR1 pid_of_process

# Developing

Missing anything? Found a bug? I love to see your PR.

## Setup development environment

It can be quite difficult to get an development environment up & running. I'll hope to
expand the instructions a bit in the future.

### Go and gvm

Todo.

### RabbitMQ

Start by installing docker.

Then:

$ docker run -d -P -e RABBITMQ_NODENAME=rabbitmq-cli-consumer --name rabbitmq-cli-consumer rabbitmq:3-management

To see which ports are available:

$ docker port rabbitmq-cli-consumer

You can login with <code>guest</code> / <code>guest</code>.
If you want stop the container:

$ docker stop rabbitmq-cli-consumer

And to restart the container:

# docker start rabbitmq-cli-consumer
21 changes: 16 additions & 5 deletions command/command_executer.go
Original file line number Diff line number Diff line change
@@ -1,34 +1,45 @@
package command

import (
"fmt"
"log"
"os/exec"
)

type Executer interface {
Execute(cmd Command) (result []byte, err error)
}

type CommandExecuter struct {
errLogger *log.Logger
infLogger *log.Logger
}

type Command interface {
CombinedOutput() (out []byte, err error)
Output() (out []byte, err error)
}

func New(errLogger, infLogger *log.Logger) *CommandExecuter {
return &CommandExecuter{
errLogger: errLogger,
infLogger: infLogger,
}
}

func (me CommandExecuter) Execute(cmd *exec.Cmd) bool {
func (me CommandExecuter) Execute(cmd Command) (result []byte, err error) {
me.infLogger.Println("Processing message...")
out, err := cmd.CombinedOutput()

out, err := cmd.Output()

if err != nil {
me.infLogger.Println("Failed. Check error log for details.")
me.errLogger.Printf("Failed: %s\n", string(out[:]))
me.errLogger.Printf("Error: %s\n", err)
return false

return out, fmt.Errorf("Error occured during execution of command: %s", err)
}

me.infLogger.Println("Processed!")

return true
return out, nil
}
41 changes: 41 additions & 0 deletions command/command_executer_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
package command

import (
"bytes"
"log"
"testing"

"github.com/stretchr/testify/mock"
)

func TestExecutesCommand(t *testing.T) {
var b bytes.Buffer
errLogger := log.New(&b, "", 0)
infLogger := log.New(&b, "", 0)

cmd := new(TestCommand)

cmd.On("Output").Return(make([]byte, 0), nil).Once()

executer := New(errLogger, infLogger)
executer.Execute(cmd)

cmd.AssertExpectations(t)

}

type TestCommand struct {
mock.Mock
}

func (t *TestCommand) CombinedOutput() (out []byte, err error) {
argsT := t.Called()

return argsT.Get(0).([]byte), argsT.Error(1)
}

func (t *TestCommand) Output() (out []byte, err error) {
argsT := t.Called()

return argsT.Get(0).([]byte), argsT.Error(1)
}
4 changes: 1 addition & 3 deletions command/command_factory.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
package command

import (
"os/exec"
)
import "os/exec"

type CommandFactory struct {
Cmd string
Expand Down
Loading