Iss57 (#58)

* Considering future runs of GWF WRF-CONUS II product

The future runs, unfortunately, follow a different file/folder structure
and they needed to be handled a bit differently than the historical
files which are all in `.tar` format.

This PR adds the capability to process this period of data for this
product to the script. Minimal adjustments were done to make the script
almost intact.

Reported-by: Logan Fang <[email protected]>
Signed-off-by: Kasra Keshavarz <[email protected]>

* Minimal typo corrections

* Adding information for the future runs files

---------

Signed-off-by: Kasra Keshavarz <[email protected]>

Author: kasra-keshavarz

scripts/gwf-ncar-conus_i/conus_i.sh

@@ -268,9 +268,9 @@ function date_match_idx () {
   # defining local variables
   local str="$1"	# string to be matched 
   local matchPos="$2"	# the position of the matching string within the "YYYY-MM-DD",
-  			# 1: year, 2: month, 3: day
-			# 1,2: year and month, 2,3: month and day, 1,3: year and day
-			# 1-3: complete date
+  # 1: year, 2: month, 3: day
+	# 1,2: year and month, 2,3: month and day, 1,3: year and day
+	# 1-3: complete date
   local delim="$3"	# delimiter
   shift	3		# shift argument positins by 3
   local strArr=("$@")	# arrays of string

scripts/gwf-ncar-conus_ii/README.md

@@ -14,9 +14,10 @@ In this file, the details of the dataset is explained.
 
 ## Location of Dataset Files
 
-The `WRF-CONUSII` simulation outputs are located under the following directory accessible from Compute Canada (CC) Graham Cluster:
+The `WRF-CONUSII` simulation outputs are located under the following directories accessible from Compute Canada (CC) Graham Cluster:
 ```
-/project/rpp-kshook/Model_Output/wrf-conus/CONUSII/hist
+/project/rpp-kshook/Model_Output/wrf-conus/CONUSII/hist # years 1995-2015
+/project/rpp-kshook/Model_Output/wrf-conus/CONUSII/fut  # years 2080-2100
 ```
 and the structure of the dataset hourly files is as following:
 ```console
@@ -106,7 +107,7 @@ foo@bar:~$ ncdump -h  /path/to/extracted/conusii/netcdf/file.nc
 The spatial extent of the `WRF-CONUSII` is on latitutes from `15.02852` to `73.27542` and longitudes from `-156.8242` to `-40.3046`.
 
 ## Temporal Extent
-As is obvious from the nomenclature of the dataset files, the time-steps are hourly covering from the January 1995 to December 2015.
+As is obvious from the nomenclature of the dataset files, the time-steps are hourly covering from the January 1995 to December 2015 (`hist` dataset directory). Also, the dataset covers data from January 2080 to December 2100 (`fut` dataset directory).
 
 # Short Description on `WRF-CONUSII` Variables
 In most hydrological modelling applications, usually 7 variables are needed detailed as following: 1) specific humidity at 2 meters, 2) surface pressure, 3) air temperature at 2 meters, 4) wind speed at 10 meters, 5) precipitation, 6) downward short wave radiation, and 7) downward long wave radiation. These variables are available through `WRF-CONUSII` dataset and their details are described in the table below:

scripts/gwf-ncar-conus_ii/conus_ii.sh

@@ -431,18 +431,34 @@ for yr in $yearsRange; do
 
   # extract variables from the forcing data files
   while [[ "$toDateUnix" -le "$endPointUnix" ]]; do
-    # date manipulations
-    toDateFormatted=$(date --date "$toDate" "+$tarFormat") # current timestamp formatted to conform to CONUSII naming convention
-    
-    # creating file name
-    file="${tarFileStruct}_${toDateFormatted}.tar" # current file name
+ 
+    if [[ "$yr" -ge "2080" ]] && [[ "$yr" -le "2100" ]]; then 
+      # date manipulations
+      # current timestamp formatted to conform to CONUSII naming convention
+      toDateFormatted=$(date --date "$toDate" "+$format")
+
+      # copy files to $cache
+      cp "${datasetDir}/${yr}/${fileStruct}_${toDateFormatted}" \
+         "${cacheDir}/${yr}/${fileStruct}_${toDateFormatted}"
+
+      # for compatibility in the next few lines
+      tarFiles=( "${fileStruct}_${toDateFormatted}" )
+
+    else
+      # date manipulations
+      # current timestamp formatted to conform to CONUSII naming convention
+      toDateFormatted=$(date --date "$toDate" "+$tarFormat")
 
-    # untar files one day at a time
-    tar --strip-components=6 -xf "$datasetDir/$yr/$file" -C "$cacheDir/$yr/"
+      # creating file name
+      file="${tarFileStruct}_${toDateFormatted}.tar" # current file name
 
-    # parse tar file contents
-    tarFiles="$(tar -tf $datasetDir/$yr/$file)"
-    IFS=' ' read -ra tarFiles <<< $(echo $tarFiles)
+      # untar files one day at a time
+      tar --strip-components=6 -xf "$datasetDir/$yr/$file" -C "$cacheDir/$yr/"
+
+      # parse tar file contents
+      tarFiles="$(tar -tf $datasetDir/$yr/$file)"
+      IFS=' ' read -ra tarFiles <<< $(echo $tarFiles)
+    fi
 
     # extracting variables from the files and spatial subsetting
     for f in "${tarFiles[@]}"; do
@@ -453,15 +469,21 @@ for yr in $yearsRange; do
            -d "$lonVar","$lonLimsIdx" \
            "$cacheDir/$yr/$f2" "$cacheDir/$yr/$f2"; do
 
-	   echo "$(basename $0): Process killed: restarting process" >$2
-	   sleep 5;
+	      echo "$(basename $0): Process killed: restarting process" >$2
+	      sleep 5;
       done
 
     done
 
     # increment time-step by one unit
-    toDate=$(date --date "$toDate 1day") # current time-step
-    toDateUnix=$(date --date="$toDate" "+%s") # current timestamp in unix EPOCH time
+    if [[ "$yr" -ge "2080" ]] && [[ "$yr" -le "2100" ]]; then 
+      toDate=$(date --date "$toDate 1hours") # current time-step
+    else
+      toDate=$(date --date "$toDate 1days") # current time-step
+    fi
+
+    # conversion current time-step to UNIX EPOCH
+    toDateUnix=$(date --date="$toDate" "+%s")
   done
 
   # go to the next year if necessary