Skip to content

Commit 0508587

Browse files
kkajla12kkajla12
authored
Feat/add postgres support (#28)
* Update datastore implementation to use 'mysql' struct tag to map between struct attrs and mysql db columns * Use microsecond precision timestamp when updating records in mysql * Add postgres datastore config support * Add datastore migrations for postgres * Remove support for basic page/limit pagination in favor of cursor-based pagination * Add postgres datastore support for all resources * Move the self-hosting instructions into separate READMEs for each database * Update link to SQLite issue in README * Fix typo in command
1 parent bc13ead commit 0508587

61 files changed

Lines changed: 3708 additions & 1132 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README.md‎

Lines changed: 4 additions & 97 deletions
Original file line numberDiff line numberDiff line change
@@ -60,106 +60,13 @@ Once you've created an account, refer to our [docs](https://docs.warrant.dev/) t
6060

6161
### Self-hosting
6262

63-
#### Configuring the database
63+
To self-host or run Warrant locally, follow one of the guides below (select the guide for your database of choice).
6464

65-
Warrant requires a database to persist access control data, authorization models, and access rules, so if you choose to self-host Warrant, you'll first need to setup a database, then configure Warrant to connect to the database on startup. The open source version of Warrant currently supports the following databases:
66-
67-
- MySQL (>= v8.0.31)
68-
- Postgres (coming soon)
69-
- SQLite (coming soon)
65+
- [MySQL](/migrations/datastore/mysql/README.md)
66+
- [Postgres](/migrations/datastore/postgres/README.md)
67+
- SQLite (coming soon) - Upvote [the issue](https://github.com/warrant-dev/warrant/issues/14)
7068
- To request support for another database, please [open an issue](https://github.com/warrant-dev/warrant/issues/new/choose)!
7169

72-
#### Configuring the database schema
73-
74-
Once you've setup your database, you can create the database schema using [golang-migrate](https://github.com/golang-migrate/migrate).
75-
76-
Update to the latest schema:
77-
78-
```bash
79-
migrate -source github://warrant-dev/warrant/migrations/datastore/mysql -database mysql://root@/warrant up
80-
```
81-
82-
#### Run the Warrant Docker image
83-
84-
Pull the latest Warrant Docker image
85-
86-
```bash
87-
docker pull warrantdev/warrant
88-
```
89-
90-
Create a configuration file `warrant.env` with the following
91-
92-
```bash
93-
WARRANT_PORT=8000
94-
WARRANT_LOGLEVEL=0
95-
WARRANT_ENABLEACCESSLOG=true
96-
WARRANT_DATASTORE=mysql
97-
WARRANT_DATASTORE_MYSQL_USERNAME=root
98-
WARRANT_DATASTORE_MYSQL_PASSWORD=
99-
WARRANT_DATASTORE_MYSQL_HOSTNAME=127.0.0.1
100-
WARRANT_DATASTORE_MYSQL_DATABASE=warrant
101-
```
102-
103-
Start the container, passing in the configuration file as environment variables to the container
104-
105-
```bash
106-
docker run --name warrant --env-file warrant.env warrantdev/warrant
107-
```
108-
109-
#### Docker Compose
110-
111-
To make it easier to run a database alongside Warrant, you can use [Docker Compose](https://docs.docker.com/compose/) to automatically setup and manage the database alongside Warrant. You can also accomplish this by running Warrant with [Kubernetes](https://kubernetes.io/).
112-
113-
Here's an example `docker-compose.yaml` that sets up a MySQL database, creates the database schema required by Warrant, and finally starts Warrant.
114-
115-
```yaml
116-
version: "3.9"
117-
services:
118-
datastore:
119-
image: mysql:8.0.32
120-
environment:
121-
MYSQL_ALLOW_EMPTY_PASSWORD: "true"
122-
MYSQL_PASSWORD:
123-
MYSQL_DATABASE: warrant
124-
ports:
125-
- 3306:3306
126-
healthcheck:
127-
test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
128-
timeout: 5s
129-
retries: 10
130-
131-
migrate-datastore:
132-
image: migrate/migrate
133-
command:
134-
[
135-
"-source",
136-
"github://warrant-dev/warrant/migrations/datastore/mysql",
137-
"-database",
138-
"mysql://root@tcp(datastore:3306)/warrant",
139-
"up",
140-
]
141-
depends_on:
142-
datastore:
143-
condition: service_healthy
144-
145-
web:
146-
image: warrantdev/warrant
147-
ports:
148-
- 8000:8000
149-
depends_on:
150-
migrate-datastore:
151-
condition: service_completed_successfully
152-
environment:
153-
WARRANT_PORT: 8000
154-
WARRANT_LOGLEVEL: 1
155-
WARRANT_ENABLEACCESSLOG: "true"
156-
WARRANT_DATASTORE: mysql
157-
WARRANT_DATASTORE_MYSQL_USERNAME: root
158-
WARRANT_DATASTORE_MYSQL_PASSWORD:
159-
WARRANT_DATASTORE_MYSQL_HOSTNAME: datastore
160-
WARRANT_DATASTORE_MYSQL_DATABASE: warrant
161-
```
162-
16370
## SDKs
16471

16572
Warrant's native SDKs are compatible with both the cloud and open-source versions of Warrant. We currently support SDKs for:

‎cmd/warrant/main.go‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,12 +41,23 @@ func (env *ServiceEnv) InitDB(config config.Config) error {
4141
return nil
4242
}
4343

44+
if config.Datastore.Postgres != nil {
45+
db := database.NewPostgres(*config.Datastore.Postgres)
46+
err := db.Connect(context.Background())
47+
if err != nil {
48+
return err
49+
}
50+
51+
env.Datastore = db
52+
return nil
53+
}
54+
4455
return fmt.Errorf("invalid database configuration provided")
4556
}
4657

4758
func NewServiceEnv() ServiceEnv {
4859
return ServiceEnv{
49-
Datastore: nil,
60+
Datastore: nil,
5061
}
5162
}
5263

‎go.mod‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ require (
88
github.com/google/uuid v1.3.0
99
github.com/gorilla/mux v1.8.0
1010
github.com/jmoiron/sqlx v1.3.5
11+
github.com/lib/pq v1.10.7
1112
github.com/pkg/errors v0.9.1
1213
github.com/rs/zerolog v1.29.0
1314
github.com/spf13/viper v1.15.0

‎go.sum‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,8 @@ github.com/leodido/go-urn v1.2.1 h1:BqpAaACuzVSgi/VLzGZIobT2z4v53pjosyNd9Yv6n/w=
153153
github.com/leodido/go-urn v1.2.1/go.mod h1:zt4jvISO2HfUBqxjfIshjdMTYS56ZS/qv49ictyFfxY=
154154
github.com/lib/pq v1.2.0 h1:LXpIM/LZ5xGFhOpXAQUIMM1HdyqzVYM13zNdjCEEcA0=
155155
github.com/lib/pq v1.2.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
156+
github.com/lib/pq v1.10.7 h1:p7ZhMD+KsSRozJr34udlUrhboJwWAgCg34+/ZZNvZZw=
157+
github.com/lib/pq v1.10.7/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
156158
github.com/magiconair/properties v1.8.7 h1:IeQXZAiQcpL9mgcAe1Nu6cX9LLw6ExEHKjN0VQdvPDY=
157159
github.com/magiconair/properties v1.8.7/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
158160
github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=

‎migrations/datastore/mysql/README.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Self-hosting Warrant with MySQL
2+
3+
This guide will cover how to self-host Warrant with MySQL as the datastore. Note that Warrant only supports versions of MySQL >= 8.0.32.
4+
5+
## Docker Compose (Recommended)
6+
7+
The following [Docker Compose](https://docs.docker.com/compose/) manifest will create a MySQL database, setup the database schema required by Warrant, and start Warrant. You can also accomplish this by running Warrant with [Kubernetes](https://kubernetes.io/).
8+
9+
```yaml
10+
version: "3.9"
11+
services:
12+
datastore:
13+
image: mysql:8.0.32
14+
environment:
15+
MYSQL_USER: replace_with_username
16+
MYSQL_PASSWORD: replace_with_password
17+
MYSQL_DATABASE: warrant
18+
ports:
19+
- 3306:3306
20+
healthcheck:
21+
test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
22+
timeout: 5s
23+
retries: 10
24+
25+
migrate-datastore:
26+
image: migrate/migrate
27+
command:
28+
[
29+
"-source",
30+
"github://warrant-dev/warrant/migrations/datastore/mysql",
31+
"-database",
32+
"mysql://replace_with_username:replace_with_password@tcp(datastore:3306)/warrant",
33+
"up",
34+
]
35+
depends_on:
36+
datastore:
37+
condition: service_healthy
38+
39+
web:
40+
image: warrantdev/warrant
41+
ports:
42+
- 8000:8000
43+
depends_on:
44+
migrate-datastore:
45+
condition: service_completed_successfully
46+
environment:
47+
WARRANT_PORT: 8000
48+
WARRANT_LOGLEVEL: 1
49+
WARRANT_ENABLEACCESSLOG: "true"
50+
WARRANT_DATASTORE: mysql
51+
WARRANT_DATASTORE_MYSQL_USERNAME: replace_with_username
52+
WARRANT_DATASTORE_MYSQL_PASSWORD: replace_with_password
53+
WARRANT_DATASTORE_MYSQL_HOSTNAME: datastore
54+
WARRANT_DATASTORE_MYSQL_DATABASE: warrant
55+
```
56+
57+
## Running the Binary
58+
59+
### Setup and Configure MySQL
60+
61+
Follow the [MySQL Installation Guide](https://dev.mysql.com/doc/mysql-installation-excerpt/8.0/en/) for your OS to install and start MySQL. For MacOS users, we recommend [installing MySQL using homebrew](https://formulae.brew.sh/formula/mysql).
62+
63+
### Migrate the Database Schema
64+
65+
Once you've setup and started your database, you can setup the database schema using [golang-migrate](https://github.com/golang-migrate/migrate).
66+
67+
Migrate to the latest schema:
68+
69+
```bash
70+
migrate -source github://warrant-dev/warrant/migrations/datastore/mysql -database mysql://replace_with_username@/warrant up
71+
```
72+
73+
### Download and Run the Binary
74+
75+
Download the latest warrant binary for your architecture from [here](https://github.com/warrant-dev/warrant/releases/latest). Then unzip the tarball and run the `warrant` executable.
76+
77+
```bash
78+
tar -xvf <name_of_tarball>
79+
cd <path_to_untarred_directory>
80+
./warrant
81+
```

‎migrations/datastore/postgres/000001_init.down.sql‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
BEGIN;
2+
3+
DROP TABLE IF EXISTS pricing_tier;
4+
DROP TABLE IF EXISTS feature;
5+
DROP TABLE IF EXISTS "user";
6+
DROP TABLE IF EXISTS tenant;
7+
DROP TABLE IF EXISTS role;
8+
DROP TABLE IF EXISTS permission;
9+
DROP TABLE IF EXISTS object;
10+
DROP TABLE IF EXISTS context;
11+
DROP TABLE IF EXISTS warrant;
12+
DROP TABLE IF EXISTS object_type;
13+
DROP FUNCTION IF EXISTS update_updated_at;
14+
15+
COMMIT;

0 commit comments

Comments
 (0)