bug: fix Arrow schema nullability not respecting JSON required field

[AhmedBoutaraa57]

[AhmedBoutaraa57]

Summary

The "required" property in the table’s JSON schema was only used for validation on insert (in validate_data_against_schema inside db_manager.rs). It was not used when building the Arrow/DataFusion schema.

Because of this, the table’s "required": false setting was never applied to the Arrow schema, so DESCRIBE always showed is_nullable = NO.

When taking into account that we have updated Timon to handle table schema updates, and that a new field was added at the end. insert will work the same way without throwing any warnings or errors. However, when we call the cloud_sink_parquet API, you will get the following warning:

⚠️ Schema compatibility error during merge for 'activitydetails_2025-09-01.parquet': Invalid argument error: Column 'newFieldAdded' is declared as non-nullable but contains null values
Skipping merge and uploading local file as-is to avoid data corruption.

After this fix, the cloud_sink_parquet will work without any warnings or errors