You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
add env for max level of flattening allowed for events
env `P_MAX_FLATTEN_LEVEL` to control the maximum level of flattening allowed
default to 10
this is to ensure nested list type fields do not get created
eg. with current implementation of hard coded level of 4,
field gets created with data type -
```
{
"name": "Records_resources_accountId",
"data_type": {
"List": {
"name": "item",
"data_type": {
"List": {
"name": "item",
"data_type": "Utf8",
"nullable": true,
"dict_id": 0,
"dict_is_ordered": false,
"metadata": {}
}
},
"nullable": true,
"dict_id": 0,
"dict_is_ordered": false,
"metadata": {}
}
},
"nullable": true,
"dict_id": 0,
"dict_is_ordered": false,
"metadata": {}
}
```
after this change, data type changes to -
```
{
"name": "Records_resources_accountId",
"data_type": "Utf8",
"nullable": true,
"dict_id": 0,
"dict_is_ordered": false,
"metadata": {}
}
```
0 commit comments