automated clickhouse migrations

[iuwqyir]

TL;DR

Added database migration support using golang-migrate to manage Clickhouse schema changes.

Each storage type has it's own migrations due to them having different connection configurations.

Migration detects if any storage types are configured for the same database and will combine the migration files in a temporary folder (this is due to the limitation of the migration tool that does not allow specifying different migration tables per migration and doesn't allow multiple migrations in one table)

What changed?

• Created a new db package with migration functionality
• Added initial migration files for orchestrator, staging, and main storage types
• Integrated migration execution into the orchestrator startup process
• Moved existing SQL schema files into organized migration directories
• Added documentation for managing migrations

How to test?

1. Start the indexer orchestrator
2. Verify migrations run successfully on startup
3. Check Clickhouse for the presence of the following tables:
   • Orchestrator: block_failures, cursors
   • Staging: block_data
   • Main: blocks, logs, transactions, traces

Why make this change?

To establish a proper database schema management system that:

• Ensures consistent schema across environments
• Automates schema updates during deployment
• Reduces manual intervention in database maintenance

[iuwqyir]

• automated clickhouse migrations #133 👈 (View in Graphite)
• improve delete performance for clickhouse #132
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.