Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
  • Loading branch information
kysre committed Feb 10, 2024
1 parent 29e2cd9 commit 34d6250
Show file tree
Hide file tree
Showing 5 changed files with 71 additions and 8 deletions.
70 changes: 67 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,76 @@

This is a MVP of a message queueing system.

## How to test locally?
## Architecture

Test with docker compose:
Design graph of this system is like below:

![Design Graph](docs/design_graph.png)

## How to build locally

Datanode implementation is in Python and it doesn't require any build process.
Leader implementation is in Golang and in order to use it you can use it's `Makefile`.
First cd in to `./leader`:

```bash
cd leader
make leader
```

## How to run locally

This project is containerized.
Run the `test.docker-compose.yaml` with `latest` tag to run with docker compose:

```bash
docker compose -f test.docker-compose.yaml up -d
docker compose -f test.docker-compose.yaml logs -f
docker compose -f test.docker-compose.yaml down -d
docker compose -f test.docker-compose.yaml down
```

## Clients

Clients are implemented in Python and Golang. Import them accordingly and use them.

Python:

```
from client_py.client import QueueClient
if __name__ == '__main__':
c = QueueClient()
c.push('test_key', b'test_value')
k, v = c.pull()
print(f'key={k}, value={v}')
c.subscribe(lambda k, v: print(f'key={k}, value={v}'))
```

Golang:

```
import (
"fmt"
"github.com/kysre/TurtleMQ/client_go"
)
func testFunc(k string, v []byte) {
fmt.Printf("key=%s, value=%b", k, v)
}
func main() {
c := client_go.GetQueueClient()
c.Push("test_key", []byte("test_value"))
k, v := c.Pull()
fmt.Printf("key=%s, value=%b", k, v)
c.Subscribe(testFunc)
}
```
Binary file added docs/design_graph.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion leader/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ clean: ## to remove generated files
-rm -rf postviewd
-find . -type d -name mocks -exec rm -rf \{} +

leader: $(SRCS) $(PBS) | generate ## Compile leader daemon
leader:
go build -o $@ leader ./cmd/$@

docker: ## to build docker image
Expand Down
3 changes: 1 addition & 2 deletions leader/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ require (
github.com/spf13/viper v1.18.2
github.com/stretchr/testify v1.8.4
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2
google.golang.org/grpc v1.60.1
google.golang.org/grpc v1.61.0
google.golang.org/protobuf v1.32.0
)

Expand All @@ -21,7 +21,6 @@ require (
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/fsnotify/fsnotify v1.7.0 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/konsorten/go-windows-terminal-sequences v1.0.3 // indirect
Expand Down
4 changes: 2 additions & 2 deletions leader/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -196,8 +196,8 @@ golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2/go.mod h1:K8+ghG5WaK9qNq
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genproto/googleapis/rpc v0.0.0-20231120223509-83a465c0220f h1:ultW7fxlIvee4HYrtnaRPon9HpEgFk5zYpmfMgtKB5I=
google.golang.org/genproto/googleapis/rpc v0.0.0-20231120223509-83a465c0220f/go.mod h1:L9KNLi232K1/xB6f7AlSX692koaRnKaWSR0stBki0Yc=
google.golang.org/grpc v1.60.1 h1:26+wFr+cNqSGFcOXcabYC0lUVJVRa2Sb2ortSK7VrEU=
google.golang.org/grpc v1.60.1/go.mod h1:OlCHIeLYqSSsLi6i49B5QGdzaMZK9+M7LXN2FKz4eGM=
google.golang.org/grpc v1.61.0 h1:TOvOcuXn30kRao+gfcvsebNEa5iZIiLkisYEkf7R7o0=
google.golang.org/grpc v1.61.0/go.mod h1:VUbo7IFqmF1QtCAstipjG0GIoq49KvMe9+h1jFLBNJs=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
Expand Down

0 comments on commit 34d6250

Please sign in to comment.