Skip to content

Commit a3cf04a

Browse files
svix-gabrielsvix-gabriel
authored
Redshift docs (#309)
2 parents 0c35dab + 70f74ea commit a3cf04a

4 files changed

Lines changed: 146 additions & 0 deletions

File tree

‎content/stream/sinks/_meta.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ const meta = {
1414
sqs: 'Amazon SQS',
1515
bigquery: 'Google BigQuery',
1616
clickhouse: 'ClickHouse',
17+
redshift: 'Amazon Redshift',
1718
poller: 'Polling Endpoint',
1819
} satisfies Meta
1920

‎content/stream/sinks/introduction.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ The following sink types are supported by Svix Stream:
1919
* [Amazon SQS](./sqs)
2020
* [Google BigQuery](./bigquery)
2121
* [ClickHouse](./clickhouse)
22+
* [Amazon Redshift](./redshift)
2223
* [Polling Endpoint](./poller)
2324

2425
All Sinks can be configured with event type filtering, batching, and transformations, and can be created through the Stream Portal or the [API](https://api.svix.com/docs#tag/Sink/operation/v1.stream.sink.create).

‎content/stream/sinks/redshift.mdx‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
---
2+
title: Amazon Redshift
3+
---
4+
# Amazon Redshift
5+
6+
Events can be sent to an Amazon Redshift table using the `redshift` sink type. Svix writes to Redshift through the [Redshift Data API](https://docs.aws.amazon.com/redshift-data/latest/APIReference/Welcome.html), and supports both Redshift Serverless and provisioned clusters.
7+
8+
Like all Sinks, Redshift sinks can be created in the Stream Portal...
9+
10+
![redshift-create](/img/stream/redshift-create.png)
11+
12+
... or [in the API](https://api.svix.com/docs#tag/Sink/operation/v1.streaming.sink.create).
13+
14+
```shell
15+
curl -X 'POST' 'https://api.svix.com/api/v1/stream/strm_30XKA2tCdjHue2qLkTgc0/sink' \
16+
-H 'Authorization: Bearer AUTH_TOKEN' \
17+
-H 'Content-Type: application/json' \
18+
-d '{
19+
"type": "redshift",
20+
"config": {
21+
"region": "us-west-2",
22+
"accessKeyId": "AKIA3LIKMTLDNWBX2PPD",
23+
"secretAccessKey": "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",
24+
"workgroupName": "default",
25+
"dbName": "dev",
26+
"tableName": "events"
27+
},
28+
"uid": "unique-identifier",
29+
"status": "enabled",
30+
"batchSize": 1000,
31+
"maxWaitSecs": 300,
32+
"eventTypes": [],
33+
"metadata": {}
34+
}'
35+
```
36+
37+
Every event batch is inserted into the configured Redshift table.
38+
39+
- `region`, `accessKeyId`, `secretAccessKey` — the AWS region and credentials used to authenticate.
40+
- `dbName` — the database to write to.
41+
- `schemaName` — the schema that contains the table (optional).
42+
- `tableName` — the table that receives the rows.
43+
44+
## Connection
45+
46+
How you point Svix at your Redshift depends on the deployment type:
47+
48+
- **Redshift Serverless** — set `workgroupName` to the name of your workgroup.
49+
- **Provisioned clusters** — set `clusterIdentifier` to your cluster's identifier and `dbUser` to the database user to connect as.
50+
51+
```shell
52+
# Provisioned cluster variant of the config block
53+
"config": {
54+
"region": "us-west-2",
55+
"accessKeyId": "AKIA3LIKMTLDNWBX2PPD",
56+
"secretAccessKey": "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",
57+
"clusterIdentifier": "my-cluster",
58+
"dbUser": "awsuser",
59+
"dbName": "dev",
60+
"tableName": "events"
61+
}
62+
```
63+
64+
## Destination table
65+
66+
Without a transformation, Svix inserts each event into the table identified by `dbName`, `schemaName`, and `tableName` using two columns: `created_at` and `payload`. Svix sets `created_at` to the insert time and writes the raw event payload to `payload`.
67+
68+
The table must already exist before you enable the sink. For the default behavior, create it with:
69+
70+
```sql
71+
CREATE TABLE events (
72+
created_at TIMESTAMP,
73+
payload VARCHAR(65535)
74+
);
75+
```
76+
77+
At the time of writing, `VARCHAR(65535)` is the largest allowable `VARCHAR` size in Redshift. If events with larger payloads are written to the stream, the sink will be disabled since these events can't be written to Redshift.
78+
79+
The `dbName`, `schemaName`, and `tableName` fields are only required when you're not using a transformation. With a transformation, the target table is named directly in your statement.
80+
81+
# Transformations
82+
83+
Redshift transformations build a parameterized SQL statement. The transformation returns the `statement` to run and the `bindings` it references.
84+
85+
```JavaScript
86+
/**
87+
* @param input - The input object
88+
* @param input.events - The array of events in the batch. The number of events in the batch is capped by the Sink's batch size.
89+
* @param input.events[].payload - The message payload (string or JSON)
90+
* @param input.events[].eventType - The message event type (string)
91+
*
92+
* @returns Object describing the SQL to run against Redshift.
93+
* @returns returns.statement - The SQL statement to execute. Reference parameters by name (e.g. :payload0).
94+
* @returns returns.bindings - The parameters referenced by the statement. Each binding is an object with a `name` and a `value`.
95+
*/
96+
function handler(input) {
97+
let bindings = [];
98+
let values = [];
99+
100+
input.events.forEach((event, i) => {
101+
const name = `payload${i}`;
102+
bindings.push({ name: name, value: JSON.stringify(event.payload) });
103+
values.push(`(CURRENT_TIMESTAMP, :${name})`);
104+
});
105+
106+
return {
107+
bindings: bindings,
108+
statement: `INSERT INTO events (created_at, payload) VALUES ${values.join(", ")};`
109+
};
110+
}
111+
```
112+
113+
`input.events` matches the events sent in [`create_events`](https://api.svix.com/docs#tag/Event/operation/v1.streaming.events.create).
114+
115+
`bindings` is an array of `{ name, value }` parameters, and the `statement` references them by name (e.g. `:payload0`). The statement is run against your database through the Redshift Data API. To write different columns, adjust the `bindings`, the `statement`, and your table to match.
116+
117+
For example, if the following events are written to the stream:
118+
119+
```shell
120+
curl -X 'POST' \
121+
'https://api.svix.com/api/v1/stream/{stream_id}/events' \
122+
-H 'Authorization: Bearer AUTH_TOKEN' \
123+
-H 'Accept: application/json' \
124+
-H 'Content-Type: application/json' \
125+
-d '{
126+
"events": [
127+
{
128+
"eventType": "user.created",
129+
"payload": "{\"email\": \"joe@enterprise.io\"}"
130+
},
131+
{
132+
"eventType": "user.login",
133+
"payload": "{\"id\": 12, \"timestamp\": \"2025-07-21T14:23:17.861Z\"}"
134+
}
135+
]
136+
}'
137+
```
138+
139+
The transformation above inserts two rows into your table.
140+
141+
| `created_at` | `payload` |
142+
| --- | --- |
143+
| `2025-07-21 14:23:18` | `{"email":"joe@enterprise.io"}` |
144+
| `2025-07-21 14:23:18` | `{"id":12,"timestamp":"2025-07-21T14:23:17.861Z"}` |

‎public/img/stream/redshift-create.png‎

208 KB
Loading

0 commit comments

Comments
 (0)