Dev

[kkappler]

This pull request is being prepared in advance. Merging dev into main in this case will correspond to closing issue #153 (and its duplicate #30), and significant progress towards #80 , #118, #132

SubTasks to address:

• test_synthetic_driver.py uses the updated processing configs.

• The initialize_config method in pipelines/helpers.py is updated

  • Handled this by creating helpers_new.py which will overwrite helpers.py when this PR completes

• Address storage of json files in synthetic processing config for the aurora_vs_emtf tests.

  • Created issue Example of reading a Processing object from json #172 to address this
  • This item is being pushed to the next PR with deprecations

• test_stft_methods_agree.py uses the updated processing configs.

• Reconcile two versions of get_data_from_mth5_new (in time_series_helpers and time_series_helpers_new.)

• remove duplicate process_mth5_run method from process_mth5_dev

• remove duplicate get_data_from_mth5 in (in time_series_helpers and time_series_helpers_new.)

• Confirm no issue removing close_mth5() from synthetic/test_stft_methods_agree

• replace time_series_helpers.py with time_series_helpers_new.py

  • Added deprecation warnings to time_series_helpers.py
    -This is ready, just need to deprecate old time_series_helpers dependencies

• replace transfer_function_helpers.py with transfer_function_helpers_new.py

  • Added deprecation warnings to transfer_function_helpers.py
  • This is ready, just need to deprecate old transfer_function_helpers dependencies

• replace pipelines/helpers.py with pipelines/helpers_new.py

  • Added deprecation warnings to helpers.py
    -This is ready, just need to deprecate old pipelines/helpers dependencies

• Modify Parkfield tests to use new processing class

  • test_calibrate_parkfield.py
  • test_process_parkfield.py
    • This was handled by removing this antiquated, and redundant test
  • test_process_parkfield_run.py
    • Created test_process_parkfield_run_new.py
    • Added deprecation warnings to test_process_parkfield_run.py
    • Overwrite test_process_parkfield_run.py with test_process_parkfield_run_new.py
  • test_process_parkfield_run_rr.py
    • Created test_process_parkfield_run_rr_new.py
    • Added deprecation warnings to test_process_parkfield_run_rr.py
    • Overwrite test_process_parkfield_run_rr.py with test_process_parkfield_run_rr_new.py
  • Deprecate make_processing_configs.py
    • Added deprecation warnings to parkfield/make_processing_configs.py

• Remove duplicate method channel_summary_to_dataset_definition()

  • One version in sandbox.mth5_channel_summary_helpers, and one in tf_kernel/dataset.py
  • rename as channel_summary_to_run_summary
  • File Issue in mth5 requesting, as a feature, that a similar method be put into mth5

• Test channel_scale_factors (can scale individual channels)

• created issue Add handling of channel_scale_factors to new Processing class #175 to address this

• Modify run_summary to return actual input/output channel lists based on what is present in channel summary

• Modify operate_aurora ipynb to use new pipeline tools
  -Made operate_aurora_new.ipynb to test

• Add start and end times usage to populate_dataset_df so that partial runs are supported

[kkappler]

The new config is now being used in new (duplicates) of all the tests.
The plan is to:

1. Merge this branch into main
2. Hop back onto a dev branch and apply the deprecations