File System Implementation Refactoring

[Umang01-hash]

File System Implementation Refactoring

Background

Current file system implementations (S3, GCS, SFTP) contain significant code duplication and inconsistent metrics.

Required Changes

1. Create a unified metrics system

   • Move metrics definitions to the main file package
   • Add storage_type label to differentiate providers (AWS/GCP/FTP)

2. Separate common code from provider-specific code

   • Common file operations → main file package
   • Storage/retrieval specifics → provider packages

3. Standardize shared components

   • Create a single FileLog implementation in the main package
   • Move common constants (SUCCESS, ERROR) to the main package
   • Define standard histogram buckets in one place
   • Implement a CloudStorageClient interface for S3/GCS

Benefits

• Reduced code duplication
• Consistent metrics across all storage providers
• Simplified implementation of new providers
• Better visibility for application developers

[Umang01-hash]

File System Refactoring Update

I have made some progress regrading this in PR #2435 and currently updated only GCS implementation to follow the same. Once merged we can focus on other providers to first match the GCS pattern for common logger and metrics functionality and then we can go ahead with larger refactorings.

####Completed

• Extracted common observability components to main file package.
• Standardized logging and metrics across providers.
• Unified constants (StatusSuccess, StatusError) and histogram buckets
• Updated GCS implementation to use these components

Future Work (Post-Merge)

• Create CloudStorageClient interface for all providers
• Separate common operations from provider-specific storage logic
• Implement version key/locking for consistent remote writes

These changes have significantly reduced code duplication while maintaining compatibility with existing implementations. The remaining work will be addressed in a dedicated follow-up PR to complete the architecture improvements.