Skip to content

Commit 87b5eee

Browse files
macobomacobo
authored
chore(ci): Set stream_poll_timeout_ms in dev (PostHog#12122)
This speeds up tests that rely on clickhouse ingestion (plugin-server) significantly. The previous default was 7500, setting this shaves off seconds off of every single test We don't set this in production due to the extra load it causes.
1 parent 357d489 commit 87b5eee

File tree

2 files changed

+130
-1
lines changed

2 files changed

+130
-1
lines changed

‎docker-compose.dev.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ services:
4545
- ./posthog/idl:/idl
4646
- ./docker/clickhouse/docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
4747
- ./docker/clickhouse/config.xml:/etc/clickhouse-server/config.xml
48-
- ./docker/clickhouse/users.xml:/etc/clickhouse-server/users.xml
48+
- ./docker/clickhouse/users-dev.xml:/etc/clickhouse-server/users.xml
4949
zookeeper:
5050
image: zookeeper:3.7.0
5151
restart: on-failure

‎docker/clickhouse/users-dev.xml

+129
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
<?xml version="1.0"?>
2+
<yandex>
3+
<!-- See also the files in users.d directory where the settings can be overridden. -->
4+
5+
<!-- Profiles of settings. -->
6+
<profiles>
7+
<!-- Default settings. -->
8+
<default>
9+
<!-- Maximum memory usage for processing single query, in bytes. -->
10+
<max_memory_usage>10000000000</max_memory_usage>
11+
12+
<!-- How to choose between replicas during distributed query processing.
13+
random - choose random replica from set of replicas with minimum number of errors
14+
nearest_hostname - from set of replicas with minimum number of errors, choose replica
15+
with minimum number of different symbols between replica's hostname and local hostname
16+
(Hamming distance).
17+
in_order - first live replica is chosen in specified order.
18+
first_or_random - if first replica one has higher number of errors, pick a random one from replicas with minimum number of errors.
19+
-->
20+
<load_balancing>random</load_balancing>
21+
22+
<allow_nondeterministic_mutations>1</allow_nondeterministic_mutations>
23+
24+
<stream_poll_timeout_ms>50</stream_poll_timeout_ms>
25+
26+
</default>
27+
28+
<!-- Profile that allows only read queries. -->
29+
<readonly>
30+
<readonly>1</readonly>
31+
</readonly>
32+
33+
</profiles>
34+
35+
<!-- Users and ACL. -->
36+
<users>
37+
<!-- If user name was not specified, 'default' user is used. -->
38+
<default>
39+
<!-- See also the files in users.d directory where the password can be overridden.
40+
41+
Password could be specified in plaintext or in SHA256 (in hex format).
42+
43+
If you want to specify password in plaintext (not recommended), place it in 'password' element.
44+
Example: <password>qwerty</password>.
45+
Password could be empty.
46+
47+
If you want to specify SHA256, place it in 'password_sha256_hex' element.
48+
Example: <password_sha256_hex>65e84be33532fb784c48129675f9eff3a682b27168c0ea744b2cf58ee02337c5</password_sha256_hex>
49+
Restrictions of SHA256: impossibility to connect to ClickHouse using MySQL JS client (as of July 2019).
50+
51+
If you want to specify double SHA1, place it in 'password_double_sha1_hex' element.
52+
Example: <password_double_sha1_hex>e395796d6546b1b65db9d665cd43f0e858dd4303</password_double_sha1_hex>
53+
54+
If you want to specify a previously defined LDAP server (see 'ldap_servers' in the main config) for authentication,
55+
place its name in 'server' element inside 'ldap' element.
56+
Example: <ldap><server>my_ldap_server</server></ldap>
57+
58+
If you want to authenticate the user via Kerberos (assuming Kerberos is enabled, see 'kerberos' in the main config),
59+
place 'kerberos' element instead of 'password' (and similar) elements.
60+
The name part of the canonical principal name of the initiator must match the user name for authentication to succeed.
61+
You can also place 'realm' element inside 'kerberos' element to further restrict authentication to only those requests
62+
whose initiator's realm matches it.
63+
Example: <kerberos />
64+
Example: <kerberos><realm>EXAMPLE.COM</realm></kerberos>
65+
66+
How to generate decent password:
67+
Execute: PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha256sum | tr -d '-'
68+
In first line will be password and in second - corresponding SHA256.
69+
70+
How to generate double SHA1:
71+
Execute: PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha1sum | tr -d '-' | xxd -r -p | sha1sum | tr -d '-'
72+
In first line will be password and in second - corresponding double SHA1.
73+
-->
74+
<password></password>
75+
76+
<!-- List of networks with open access.
77+
78+
To open access from everywhere, specify:
79+
<ip>::/0</ip>
80+
81+
To open access only from localhost, specify:
82+
<ip>::1</ip>
83+
<ip>127.0.0.1</ip>
84+
85+
Each element of list has one of the following forms:
86+
<ip> IP-address or network mask. Examples: 213.180.204.3 or 10.0.0.1/8 or 10.0.0.1/255.255.255.0
87+
2a02:6b8::3 or 2a02:6b8::3/64 or 2a02:6b8::3/ffff:ffff:ffff:ffff::.
88+
<host> Hostname. Example: server01.yandex.ru.
89+
To check access, DNS query is performed, and all received addresses compared to peer address.
90+
<host_regexp> Regular expression for host names. Example, ^server\d\d-\d\d-\d\.yandex\.ru$
91+
To check access, DNS PTR query is performed for peer address and then regexp is applied.
92+
Then, for result of PTR query, another DNS query is performed and all received addresses compared to peer address.
93+
Strongly recommended that regexp is ends with $
94+
All results of DNS requests are cached till server restart.
95+
-->
96+
<networks>
97+
<ip>::/0</ip>
98+
</networks>
99+
100+
<!-- Settings profile for user. -->
101+
<profile>default</profile>
102+
103+
<!-- Quota for user. -->
104+
<quota>default</quota>
105+
106+
<!-- User can create other users and grant rights to them. -->
107+
<!-- <access_management>1</access_management> -->
108+
</default>
109+
</users>
110+
111+
<!-- Quotas. -->
112+
<quotas>
113+
<!-- Name of quota. -->
114+
<default>
115+
<!-- Limits for time interval. You could specify many intervals with different limits. -->
116+
<interval>
117+
<!-- Length of interval. -->
118+
<duration>3600</duration>
119+
120+
<!-- No limits. Just calculate resource usage for time interval. -->
121+
<queries>0</queries>
122+
<errors>0</errors>
123+
<result_rows>0</result_rows>
124+
<read_rows>0</read_rows>
125+
<execution_time>0</execution_time>
126+
</interval>
127+
</default>
128+
</quotas>
129+
</yandex>

0 commit comments

Comments
 (0)