Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add capability to run diff resolutions for marine anl and background #3238

Merged
merged 36 commits into from
Feb 20, 2025
Merged

Add capability to run diff resolutions for marine anl and background #3238

merged 36 commits into from
Feb 20, 2025

Conversation

guillaumevernieres
Copy link
Contributor

@guillaumevernieres guillaumevernieres commented Jan 16, 2025 •
edited
Loading

Description

This PR mostly includes configuration changes to allow the use a B-matrix at a defined resolution that may or may not be the same as the background.

Changes of note:

Dependencies

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? YES (If YES, please add a link to any PRs that are pending.)
    • EMC verif-global
    • GDAS
    • GFS-utils
    • GSI
    • GSI-monitor
    • GSI-utils
    • UFS-utils
    • UFS-weather-model
    • wxflow

How has this been tested?

Subset of the ci tests on hercules and hera

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

@guillaumevernieres guillaumevernieres mentioned this pull request Jan 16, 2025
Merged
10 tasks
DavidNew-NOAA
DavidNew-NOAA requested changes Jan 17, 2025
View reviewed changes
Copy link
Contributor

@DavidNew-NOAA DavidNew-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good except for the one comment

@guillaumevernieres guillaumevernieres added the JEDI Feature development to support JEDI-based DA label Jan 23, 2025
DavidNew-NOAA
DavidNew-NOAA previously approved these changes Jan 23, 2025
View reviewed changes
@emcbot emcbot added the CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules label Feb 19, 2025
@aerorahul
Copy link
Contributor

Hera and Hercules are enough; especially the automatic CI on WCOSS2 does not work for WCDA test (that was my understanding; happy to be corrected)

@aerorahul
Copy link
Contributor

The WCDA test is also set to be skipped on WCOSS2.

@emcbot emcbot added CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress and removed CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules labels Feb 19, 2025
@DavidHuber-NOAA
Copy link
Contributor

Manual CI has started on Hera. The clone is located in /scratch1/NCEPDEV/global/David.Huber/GW/pr_3238 and tests will be run in /scratch1/NCEPDEV/global/David.Huber/pr/3238.

@DavidHuber-NOAA DavidHuber-NOAA added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera labels Feb 19, 2025
@emcbot emcbot added CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully and removed CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress labels Feb 19, 2025
@emcbot
Copy link

emcbot commented Feb 19, 2025

CI Passed on Hercules in Build# 1
Built and ran in directory /work2/noaa/global/CI/HERCULES/3238


Experiment C48_ATM_b3782c9d Completed 1 Cycles: *SUCCESS* at Wed Feb 19 09:22:01 CST 2025
Experiment C48mx500_hybAOWCDA_b3782c9d Completed 2 Cycles: *SUCCESS* at Wed Feb 19 09:52:22 CST 2025
Experiment C96_S2SWA_gefs_replay_ics_b3782c9d Completed 1 Cycles: *SUCCESS* at Wed Feb 19 10:04:49 CST 2025
Experiment C96C48_hybatmDA_b3782c9d Completed 3 Cycles: *SUCCESS* at Wed Feb 19 11:11:19 CST 2025
Experiment C96_atm3DVar_b3782c9d Completed 3 Cycles: *SUCCESS* at Wed Feb 19 11:11:21 CST 2025
Experiment C48mx500_3DVarAOWCDA_b3782c9d Completed 2 Cycles: *SUCCESS* at Wed Feb 19 11:48:12 CST 2025
Experiment C48_S2SWA_gefs_b3782c9d Completed 1 Cycles: *SUCCESS* at Wed Feb 19 12:43:07 CST 2025
Experiment C48_S2SW_b3782c9d Completed 1 Cycles: *SUCCESS* at Wed Feb 19 13:00:32 CST 2025

@RussTreadon-NOAA
Copy link
Contributor

WCOSS2 g-w CI

Since GFS v17 includes JEDI components install guillaumevernieres:feature/lowres-b at b3782c9 on Cactus. Set up and run g-w CI with the following results

/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_ATM_pr3238
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 19 2025 14:39:33    Feb 19 2025 15:45:18
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48mx500_3DVarAOWCDA_pr3238
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Feb 19 2025 14:39:35    Feb 19 2025 14:55:29
202103250000        Done    Feb 19 2025 14:39:35    Feb 19 2025 16:40:27
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48mx500_hybAOWCDA_pr3238
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Feb 19 2025 14:39:38    Feb 19 2025 14:55:36
202103250000        Done    Feb 19 2025 14:39:38    Feb 19 2025 16:05:34
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_S2SWA_gefs_pr3238
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 19 2025 14:39:41    Feb 19 2025 16:35:52
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_S2SW_pr3238
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 19 2025 14:39:44    Feb 19 2025 15:50:35
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96_atm3DVar_pr3238
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Feb 19 2025 14:39:46    Feb 19 2025 15:01:05
202112210000        Done    Feb 19 2025 14:39:46    Feb 19 2025 17:10:40
202112210600        Done    Feb 19 2025 14:39:46    Feb 19 2025 16:50:38
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmaerosnowDA_pr3238
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Feb 19 2025 14:39:48    Feb 19 2025 15:01:08
202112201800        Done    Feb 19 2025 14:39:48    Feb 19 2025 17:05:36
202112210000        Done    Feb 19 2025 14:39:48    Feb 19 2025 16:50:42
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmDA_pr3238
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Feb 19 2025 14:39:50    Feb 19 2025 15:01:12
202112210000        Done    Feb 19 2025 14:39:50    Feb 19 2025 16:45:45
202112210600        Done    Feb 19 2025 14:39:50    Feb 19 2025 16:50:46
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_ufs_hybatmDA_pr3238
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202402231800        Done    Feb 19 2025 14:39:52    Feb 19 2025 15:01:16
202402240000        Done    Feb 19 2025 14:39:52    Feb 19 2025 19:40:25
202402240600        Done    Feb 19 2025 14:39:52    Feb 19 2025 19:45:06
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96_S2SWA_gefs_replay_ics_pr3238
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202011010000        Done    Feb 19 2025 14:39:56    Feb 19 2025 15:41:03

All jobs in all cases successfully ran to completion.

Note that WCOSS2 g-w CI on Cactus was set up to run the same jobs that are run on Hera. Thus, hybatmaerosnowDA, 3DVarAOWCDA, hybAOWCDA, and ufs_hybatmDA were run on Cactus. These cases exercise various JEDI (GDASApp) capabilities.

@RussTreadon-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA, @DavidHuber-NOAA , and @aerorahul : g-w CI successfully ran on Cactus. The CI-Wcoss2_Passed label can be applied to this PR.

@DavidHuber-NOAA DavidHuber-NOAA added CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Feb 19, 2025
@RussTreadon-NOAA
Copy link
Contributor

@DavidHuber-NOAA and @TerrenceMcGuinness-NOAA : While the CI-Hera-Running label remains, squeue on Hera does not return any pending or running jobs associated with this PR. Did Hera CI testing complete?

@DavidHuber-NOAA
Copy link
Contributor

@RussTreadon-NOAA All tests passed except C48_ATM. This was due to an operator error (me). I am rerunning this test now.

@DavidHuber-NOAA DavidHuber-NOAA mentioned this pull request Feb 20, 2025
Merged
13 tasks
@DavidHuber-NOAA
Copy link
Contributor

The C48_ATM test case and all other tests have now completed. Setting CI-Hera-Passed label.

@DavidHuber-NOAA DavidHuber-NOAA added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Feb 20, 2025
@RussTreadon-NOAA
Copy link
Contributor

Thank you @DavidHuber-NOAA for guiding this PR through Hera g-w CI.

@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Feb 20, 2025
@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Feb 20, 2025
@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit 6a287b7 into NOAA-EMC:develop Feb 20, 2025
9 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CoryMartin-NOAA CoryMartin-NOAA CoryMartin-NOAA left review comments

@AndrewEichmann-NOAA AndrewEichmann-NOAA AndrewEichmann-NOAA left review comments

@shlyaeva shlyaeva shlyaeva approved these changes

@RussTreadon-NOAA RussTreadon-NOAA RussTreadon-NOAA approved these changes

@aerorahul aerorahul aerorahul left review comments

@WalterKolczynski-NOAA WalterKolczynski-NOAA WalterKolczynski-NOAA left review comments

@DavidNew-NOAA DavidNew-NOAA DavidNew-NOAA left review comments

@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA Awaiting requested review from TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA is a code owner

Assignees
No one assigned
Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully JEDI Feature development to support JEDI-based DA
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Implement the marine analysis at a resolution different from the deterministic Background error at 1/2 deg