gee-water-resources-management.Rmd

---
title: "Google Earth Engine for Water Resources Management (Full Course Material)"
author: "Ujaval Gandhi"
subtitle: Application-focused Introduction to Google Earth Engine.
output:
  # word_document:
  #   toc: yes
  #   toc_depth: '3'
  html_document:
    df_print: paged
    highlight: pygments
    toc: yes
    toc_depth: 3
  # pdf_document:
  #   latex_engine: xelatex
  #   toc: yes
  #   toc_depth: '3'
fontsize: 12pt
header-includes:
- \usepackage{fancyhdr}
- \pagestyle{fancy}
- \renewcommand{\footrulewidth}{0.4pt}
- \fancyhead[LE,RO]{\thepage}
- \geometry{left=1in,top=0.75in,bottom=0.75in}
- \fancyfoot[CE,CO]{{\includegraphics[height=0.5cm]{images/cc-by-nc.png}} Ujaval Gandhi
  http://www.spatialthoughts.com}
---
\newpage

***

```{r echo=FALSE, fig.align='center', out.width='75%', out.width='250pt'}
knitr::include_graphics('images/spatial_thoughts_logo.png')
```

***

\newpage

# Introduction 

GIS and Remote Sensing plays a critical role in the management of water resources. Many practitioners in this field are constrained by the availability of tools and computing resources to use these techniques effectively. Recent advances in cloud computing technology have given rise to platforms such as Google Earth Engine, which provide free access to a large pool of computational resources and datasets. The course is designed for researchers in the water sector, academicians, water managers, and stakeholders with basic knowledge of Remote Sensing. It will enable them to leverage this platform for water resource management applications.

[![View Presentation](images/gee_water_resources_management/course_overview.png){width="400px"}](https://docs.google.com/presentation/d/1-PqksKb2QB8YJTQic5M80ZAsKh1Ld6OxOoO6BG77Uhg/edit?usp=sharing){target="_blank"}

[View the Presentation ↗](https://docs.google.com/presentation/d/1-PqksKb2QB8YJTQic5M80ZAsKh1Ld6OxOoO6BG77Uhg/edit?usp=sharing){target="_blank"}


# Setting up the Environment

## Sign-up for Google Earth Engine

If you already have a Google Earth Engine account, you can skip this step.

Visit [https://signup.earthengine.google.com/](https://signup.earthengine.google.com/){target="_blank"} and sign-up with your Google account. You can use your existing gmail account to sign-up. It usually takes 1-2 days for approval. Hence do this step as soon as possible.

Tips to ensure smooth sign-up process:

- Use Google Chrome browser.
- When signing up for Earth Engine, please log out of all Google accounts and ensure you are logged into only 1 account which you want associated with Earth Engine. 
- Access to Google Earth Engine is granted via Google Groups. The default settings should be fine, but verify you have the correct setting enabled.
    - Visit [groups.google.com](http://groups.google.com/){target="_blank"}
    - Click on Settings (gear icon) and select Global Settings.
    - Make sure the option Allow group managers to add me to their groups is checked.


## Get the Course Materials

The course material and exercises are in the form of Earth Engine scripts shared via a code repository.

1. [Click this link](https://code.earthengine.google.co.in/?accept_repo=users/ujavalgandhi/GEE-Water-Resources-Management) to open Google Earth Engine code editor and add the repository to your account.
2. If successful, you will have a new repository named `users/ujavalgandhi/GEE-Water-Resources-Management` in the *Scripts* tab in the *Reader* section.
3. Verify that your code editor looks like below

> If you do not see the repository in the *Reader* section, refresh your browser tab and it will show up.

```{r echo=FALSE, fig.align='center', out.width='75%', fig.cap='Code Editor After Adding the Class Repository'}
knitr::include_graphics('images/gee_water_resources_management/repository.png')
```


\newpage


# Module 1: Google Earth Engine Fundamentals

In Module 1, you will gain the essential skills to find datasets, filter them to your region, clip them to your boundary, calculate remote sensing indices, extract water-bodies using binary thresholding, and export raster data.

## 01. Hello World

This script introduces the basic Javascript syntax and the video covers the programming concepts you need to learn when using Earth Engine. To learn more, visit [Introduction to JavaScript for Earth Engine](https://developers.google.com/earth-engine/tutorials/tutorial_js_01) section of the Earth Engine User Guide. 

[![Video](images/gee_water_resources_management/intro_to_javascript.png){width="400px"}](https://www.youtube.com/watch?v=RV3Sv5iogHs){target="_blank"}

- [Watch the Video](https://www.youtube.com/watch?v=RV3Sv5iogHs){target="_blank"}

The *Code Editor* is an Integrated Development Environment (IDE) for Earth Engine Javascript API. It offers an easy way to type, debug, run and manage code. Type the code below and click *Run* to execute it and see the output in the *Console* tab.

> Tip: You can use the keyboard shortcut *Ctrl+Enter* to run the code in the Code Editor

```{r echo=FALSE, fig.align='center', out.width='75%', fig.cap='Hello World'}
knitr::include_graphics('images/gee_water_resources_management/hello_world.png')
```

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/01b_Hello_World_(complete)')}
```

[Open in Code Editor ↗](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F01b_Hello_World_(complete)){target="_blank"}

### Exercise


```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/01c_Hello_World_(exercise)')}
```

[Try in Code Editor ↗](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F01c_Hello_World_(exercise)){target="_blank"}


### Saving Your Work

When you modify any script for the course repository, you may want to save a copy for yourself. If you try to click the *Save* button, you will get an error message like below

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/gee_water_resources_management/setup1.png')
```

This is because the shared class repository is a *Read-only* repository. You can click *Yes* to save a copy in your repository. If this is the first time you are using Earth Engine, you will be prompted to choose the name of your *home folder*. Choose the name carefully, as it cannot be changed once created.

```{r echo=FALSE, fig.align='center', out.width='60%'}
knitr::include_graphics('images/gee_water_resources_management/setup2.png')
```

## 02. Working with Image Collections

Most datasets in Earth Engine come as a `ImageCollection`. An ImageCollection is a dataset that consists of images takes at different time and locations - usually from the same satellite or data provider. You can load a collection by searching the [Earth Engine Data Catalog](https://developers.google.com/earth-engine/datasets) for the *ImageCollection ID*. Search for the *Sentinel-2 Level 2A* dataset and you will find its id `COPERNICUS/S2_SR`. Visit the [Sentinel-2, Level 2A page](https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S2_SR) and see *Explore in Earth Engine* section to find the code snippet to load and visualize the collection. This snippet is a great starting point for your work with this dataset. Click the **Copy Code Sample** button and paste the code into the code editor. Click *Run* and you will see the image tiles load in the map.


```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/gee_water_resources_management/image_collection1.png')
```

In the code snippet, You will see a function `Map.setCenter()` which sets the viewport to a specific location and zoom level. The function takes the X coordinate (longitude), Y coordinate (latitude) and Zoom Level parameters ranging from 1 (whole world) to 22 (pixel level). Replace the X and Y coordinates with the coordinates of your city and click *Run* to see the images of your city.

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/gee_water_resources_management/image_collection2.png')
```

[Open in Code Editor ↗](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F02b_Image_Collections_(complete)){target="_blank"}

### Exercise

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/02c_Image_Collections_(exercise)')}
```

[Try in Code Editor ↗](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F02c_Image_Collections_(exercise)){target="_blank"}

## 03. Filtering Image Collections

The collection contains all imagery ever collected by the sensor. The entire collections are not very useful. Most applications require a subset of the images. We use **filters** to select the appropriate images. There are many types of filter functions, look at `ee.Filter...` module to see all available filters. Select a filter and then run the `filter()` function with the filter parameters. 

We will learn about 3 main types of filtering techniques

* **Filter by metadata**: You can apply a filter on the image metadata using filters such as `ee.Filter.eq()`, `ee.Filter.lt()` etc. You can filter by PATH/ROW values, Orbit number, Cloud cover etc.
* **Filter by date**: You can select images in a particular date range using filters such as `ee.Filter.date()`.
* **Filter by location**: You can select the subset of images with a bounding box, location or geometry using the `ee.Filter.bounds()`. You can also use the drawing tools to draw a geometry for filtering.

After applying the filters, you can use the `.size()` function to check how many images match the filter criteria.

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/gee_water_resources_management/filters.png')
```

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/03b_Filtering_Image_Collection_(complete)')}
```

[Open in Code Editor ↗](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F03b_Filtering_Image_Collection_(complete)){target="_blank"}

### Exercise

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/03c_Filtering_Image_Collection_(exercise)')}
```

[Try in Code Editor ↗](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F03c_Filtering_Image_Collection_(exercise)){target="_blank"}

## 04. Mosaics and Composites

The default order of the collection is by date. So when you display the collection, it implicitly creates a mosaic with the latest pixels on top. You can call `.mosaic()` on a ImageCollection to create a mosaic image from the pixels at the top (i.e) first image in the stack.

We can also create a composite image by applying selection criteria to each pixel from all pixels in the stack. Here we use the `.median()` function to create a composite where each pixel value is the median of all pixels values at each pixel from the stack for all matching bands.

> Tip: If you need to create a mosaic where the images are in a specific order, you can use the `.sort()` function to sort your collection by a property first.


```{r echo=FALSE, fig.align='center', out.width='100%', fig.cap='Mosaic vs. Median Composite'}
knitr::include_graphics('images/gee_water_resources_management/mosaic_median.png')
```

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/04b_Mosaics_and_Composites_(complete)')}
```

[Open in Code Editor ↗](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F04b_Mosaics_and_Composites_(complete)){target="_blank"}

### Exercise

```{js eval=FALSE}
// Create a median composite for the year 2020 and load it to the map

// Compare both the composites to see the changes in your city
```

[Try in Code Editor ↗](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F04c_Mosaics_and_Composites_(exercise)){target="_blank"}

## 05. Feature Collection


Feature Collections are similar to Image Collections - but they contain *Features*, not images. They are equivalent to Vector Layers in a GIS. We can load, filter and display Feature Collections using similar techniques that we have learned so far. 

Search for *GAUL Second Level Administrative Boundaries* and load the collection. This is a global collection that contains all Admin2 boundaries. We can apply a filter using the `ADM1_NAME` property to get all Admin2 boundaries (i.e. Districts) from a state.

```{r echo=FALSE, fig.align='center', out.width='75%'}
knitr::include_graphics('images/gee_water_resources_management/feature_collection.png')
```


```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/05b_Feature_Collections_(complete)')}
```
[Open in Code Editor ↗](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F05b_Feature_Collections_(complete)){target="_blank"}


### Exercise

```{js eval=FALSE}
// Add the admin2 layer to the map using Map.addLayer() function
// Go to your home city and inspect the layer to find the name of the region
// Use the ADM2_NAME property and apply a filter
// Display only the selected polygon on the map.
```
[Try in Code Editor ↗](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F05c_Feature_Collections_(exercise)){target="_blank"}

## 06. Clipping

It is often desirable to clip the images to your area of interest. You can use the `clip()` function to mask an image using geometry.

> While in Desktop softwares, clipping is used to remove the unnecessary portion of a large image to save computation time, but in the Earth Engine, clipping can <span style="color:red">increase the computation time.</span> As described in the [Earth Engine Coding Best Practices](https://developers.google.com/earth-engine/guides/best_practices?hl=en#if-you-dont-need-to-clip,-dont-use-clip) guide, avoid clipping or do it at the very end of your process.


```{r echo=FALSE, fig.align='center', out.width='100%', fig.cap='Full Image vs. Clipped Image'}
knitr::include_graphics('images/gee_water_resources_management/clipping.png')
```

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/06b_Clipping_(complete)')}
```
[Open in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F06b_Clipping_(complete)){target="_blank"}


### Exercise

```{js eval=FALSE}
// Add the GAUL admin boundary to the Map
// Search for your city and inspect the ADM2_NAME
// Replace the name with the ADM2_NAME of your selected region
// And display the clipped composite on the map.
```
[Try in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F06c_Clipping_(exercise)){target="_blank"}

## 07. Calculating Indices

Spectral Indices are central to many aspects of remote sensing. For almost all research/work, you will need to compute a pixel-wise ratio of 2 or more bands. The most commonly used formula for calculating an index is computing the *Normalized Difference* between 2 bands. Earth Engine provides a helper function `normalizedDifference()` to help calculate normalized indices, such as Modified Normalized Difference Water Index (MNDWI). 

For more complex formulae such as Automated Water Extraction Index (AWEI), you can use the `expression()` function to describe the calculation. AWEI is a moden technique to do surface water mapping with high accuracy. [Learn more about AWEI](https://www.sciencedirect.com/science/article/abs/pii/S0034425713002873)


```{r echo=FALSE, fig.align='center', out.width='75%', fig.cap='RGB, MNDWI, NDVI and AWEI images'}
knitr::include_graphics('images/gee_water_resources_management/indices.png')
```

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/07b_Calculating_Indices_(complete)')}
```
[Open in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F07b_Calculating_Indices_(complete)){target="_blank"}



### Exercise

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/07c_Calculating_Indices_(exercise)')}
```
[Try in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F07c_Calculating_Indices_(exercise)){target="_blank"}

## 08. Computation on Images

A standard method to extract information from an image is to threshold the image. We can write conditions for thresholding. The result will be a binary image with pixel values 1 or 0. For all pixel matching, the condition value will be 1, and for other pixels where the condition fails value will be 0. The conditions can be written using the *logical operators* like greater than `.gt()`, lesser than `.lt()`, equal to `.eq()`, greater than or equal to `.gte()`, etc. We can also combine multiple layers to create a condition using the *boolean operators* like AND `.and()`, OR `.or()` functions.  

```{r echo=FALSE, fig.align='center', out.width='75%', fig.cap='RGB and Thresholded images'}
knitr::include_graphics('images/gee_water_resources_management/threshold.png')
```

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/08b_Computation_on_Images_(complete)')}
```
[Open in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F08b_Computation_on_Images_(complete)){target="_blank"}

### Exercise

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/08c_Computation_on_Images_(exercise)')}
```
[Try in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F08c_Computation_on_Images_(exercise)){target="_blank"}

## 09. Export

Earth Engine allows for exporting both vector and raster data to be used in an external program. Vector data can be exported as a `CSV` or a `Shapefile`, while Rasters can be exported as `GeoTIFF` files. We will now export the Sentinel-2 Composite as a GeoTIFF file.

During the export, we can export either a raw image or visualized image. The raw image will retain the original reflectance value of each pixel. The visualized image will generate RGB or grayscale visualization of the image. The image will look like it is in the earth engine map view, but the band reflectance information will be lost. The visualized image should not use it for analysis purposes. 

> Tip: Code Editor supports autocompletion of API functions using the combination *Ctrl+Space*. Type a few characters of a function and press *Ctrl+Space* to see autocomplete suggestions. You can also use the same key combination to fill all parameters of the function automatically.

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/01-Earth_Engine_Basics/09b_Export_(complete)')}
```
[Open in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F09b_Export_(complete)){target="_blank"}

Once you run this script, the *Tasks* tab will be highlighted. Switch to the tab and you will see the tasks waiting. Click *Run* next to each task to start the process. On clicking the *Run* button, you will be prompted for a confirmation dialog. Verify the settings and click *Run* to start the export.

```{r echo=FALSE, fig.align='center', out.width='75%', fig.cap='Console, Tasks and Confirmation'}
knitr::include_graphics('images/gee_water_resources_management/export_task.png')
```

Once the Export finishes, a GeoTiff file for each export task will be added to your Google Drive in the specified folder. You can download them and use it in a GIS software.

```{r echo=FALSE, fig.align='center', out.width='75%', fig.cap='Visualized vs Raw'}
knitr::include_graphics('images/gee_water_resources_management/export_image.png')
```

### Exercise

```{js eval=FALSE}
// Export the waterMndwi image to your drive 

```
[Try in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A01-Earth_Engine_Basics%2F09c_Export_(exercise)){target="_blank"}


# Module 2: Surface Water Mapping

In this Module, we will learn create an vectore surface water map for a water shed boundary in [hydroSheds](https://www.hydrosheds.org/page/hydrobasins) using [JRC - Global Surface Water](https://global-surface-water.appspot.com/#features). This data contains more accurate global-level surface water information. Using this, we will map the surface water, then learn to water mask and get specific information, execute morphological operations, and create a vector map for a hydro shed basin. 

[![View Presentation](images/gee_water_resources_management/Surface_Water_Mapping_ppt.png){width="400px"}](https://docs.google.com/presentation/d/15EIaoRWc-obUrl2kqzlw_mYIaSAzYn7aOOE9SyU6XKc/edit?usp=sharing){target="_blank"}

[View the Presentation &#8599;](https://docs.google.com/presentation/d/15EIaoRWc-obUrl2kqzlw_mYIaSAzYn7aOOE9SyU6XKc/edit?usp=sharing){target="_blank"}

## 01. Load Global Surface Water Data

Lets load the [JRC Global Surface Water Mapping Layers](https://developers.google.com/earth-engine/datasets/catalog/JRC_GSW1_3_GlobalSurfaceWater). This data is the Spatio-temporal distribution of surface water. It is a single image with 7 bands, where each band contains unique information. Let's use the `occurrence` band, which includes information on the frequency of the water present from 1984-2020. The pixel value ranges from 0-100, where 0 represents No trace of water in any year, and 100 represents water present in all 36 years.

Use the Inspector to find the frequency of water present in the location. 

```{r echo=FALSE, fig.align='center', out.width='75%', fig.cap='Frequency of water present over Bellandur lake'}
knitr::include_graphics('images/gee_water_resources_management/Load_Global_Surface_Water_Data_01.png')
```

[Open in Code Editor &#8599;](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F01b_Load_Global_Surface_Water_Data_(complete)){target="_blank"}

### Exercise

The `max_extent` band gives information on whether a pixel ever contained water or not, 0 represents the pixel has no water present in any of the months, and 1 represents the pixel has contained water in it for at least 1 month.

Load the `max_extent` band and visualize it with correct parameters.

```{js eval=FALSE}
// Create a map showing everywhere surface water was present.
// Select the 'max_extent' band and display the results
```

[Try in Code Editor &#8599;](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F01c_Load_Global_Surface_Water_Data_(exercise)){target="_blank"}

## 02. Create a Water Mask

Earth Engine has a `mask` function to represent the *No Data* values in an Image. Any masked pixel of an image will not be used for analysis or visualization. There are two types of masks `selfMask` and `updateMask.` 

The `selfMask` will remove all the pixel values with *zero*. In the `max_extent` band visualization, we saw the pixel value is either 0 or 1, the pixel value with 0 contains no information, but the pixel value with 1 represents the water occurrence. So to remove the No Data pixel within the image, we can use the selfMask function. This will mask all pixel value with 0.

The `updateMask` will mask a *primary image* using a *mask image*. A mask image is an image with 0 1 pixel value. The primary image pixels overlaying with 0-pixel value in mask image will be set as No Data and masked out.

```{r echo=FALSE, fig.align='center', out.width='75%', fig.cap='Before Mask vs After Mask'}
knitr::include_graphics('images/gee_water_resources_management/water_mask.png')
```


```{js eval=FALSE, code=readLines('code/gee_water_resources_management/02-Surface_Water_Mapping/02b_Create_a_Water_Mask_(complete)')}
```

[Open in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F02b_Create_a_Water_Mask_(complete)){target="_blank"}

### Exercise

The `seasonality` band gives information on the number of months water was present in a pixel. Pixel value ranges from 1 to 12, 1 represents water present for 1 month, and 12 represents water present for 12 months in a year.

Select all pixels in which water present for more than 5 months in a year.

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/02-Surface_Water_Mapping/02c_Create_a_Water_Mask_(exercise)')}
```

[Try in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F02c_Create_a_Water_Mask_(exercise)){target="_blank"}


## 03. Find Lost Waterbodies

The `transition` band has water class changes between 1986 and 2020. It has [10 classes](https://developers.google.com/earth-engine/datasets/catalog/JRC_GSW1_3_GlobalSurfaceWater#bands) representing different changes, in which classes *3* and *6* represent *permanent* and *seasonal water loss*. We can calculate the total area of surface water lost by selecting both classes. 

The *binary operator* `eq` is used to select a particular class and create a binary image representing the availability of the class. The *boolean operator* `or` is used to create a binary image where either class value is present.

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/02-Surface_Water_Mapping/03b_Find_Lost_Waterbodies_(complete)')}
```
[Open in Code Editor &#8599;](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F03b_Find_Lost_Waterbodies_(complete)){target="_blank"}

### Exercise

Use the `transition` band and find the total surface water gain. 

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/02-Surface_Water_Mapping/03c_Find_Lost_Waterbodies_(exercise)')}
```

[Try in Code Editor &#8599;](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F03c_Find_Lost_Waterbodies_(exercise)){target="_blank"}

## 04. Get Yearly Water History

The [JRC Yearly Water Classification History](https://developers.google.com/earth-engine/datasets/catalog/JRC_GSW1_3_YearlyHistory) image collection contains the spatiotemporal map of surface water from 1984 to 2020. A total of 37 images represent 1 image for each year.

Let us filter this collection to visualize the surface water map of the year *2020*. This data contains 4 bands, where bands 2 and 3 indicate water presence.

> After filtering the Image Collection, the resultant would be a collection even if it contains just 1 image. We can use the function `.first()` to extract the particular image.

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/02-Surface_Water_Mapping/04b_Get_Yearly_Water_History_(complete)')}
```
[Open in Code Editor &#8599;](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F04b_Get_Yearly_Water_History_(complete)){target="_blank"}

### Exercise

Use the `JRC Yearly Water Classification History` dataset and visualize the surface water map for the year *2010*. 

```{js eval=FALSE} 
// Get the surface water image for the year 2010
// Display it on the map
```

[Try in Code Editor &#8599;](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F04c_Get_Yearly_Water_History_(exercise)){target="_blank"}

## 05. Locate a SubBasin

The [hydroSHEDS](https://www.hydrosheds.org/) is a series of data primarily developed by the *World Wildlife Fund*. It contains a global level geo-reference stream network, watershed boundaries, and drainage directions. We will use the `hydroBASINS,` a feature collection containing drainage basin boundaries. This data is created using SRTM DEM data. There are multiple levels of hydroBASINS, ranging from 1 to 11. In level 1, the basin boundaries are delineated as huge groups. Moving levels up, these boundaries will be sub-divided into smaller boundaries.

We will use the Level 7 hydroBASINS feature collection to filter and locate a sun-basin called *Arkavathy*. The hydroBASINS doesn't contain basin names, so basins should be visually located and filtered using the *HYBAS_ID* property to filter a particular basin boundary.

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/02-Surface_Water_Mapping/05b_Locate_a_SubBasin_(complete)')}
```
[Open in Code Editor &#8599;](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F05b_Locate_a_SubBasin_(complete)){target="_blank"}

### Exercise

Using the Level 7 hydroBASINS, locate and filter a watershed basin in your region of interest. 

```{js eval=FALSE}
// Add the hydrobasins layer to the map
// Zoom to your region on interest and inspect the layer
// Find the id for a sub basin of interest and add it to the map.
```

[Try in Code Editor &#8599;](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F05c_Locate_a_SubBasin_(exercise)){target="_blank"}

## 06. Create a Surface Water Map

A raster image output can be post-processed using morphological operations to create a clean-looking surface map. There are different types of operations like *Morphological Opening*, *Morphological Closing*, and *Majority Filtering*. 

* The opening operation can reduce the salt and pepper noise in the image. 
* The closing operation can fill holes and create a complete class. 
* A majority filter will smooth the image. This operation should be used for mapping purposes only and not used while calculating statistics, as it will significantly alter the result.

Earth Engine has many functions to perform morphological operations under `ee.Image`. Let's use the `Max_extent` band from Global Surface Water and fill holes to create a surface water map. An *Erosion* operation follows a *Dilation* operation to perform a closing operation, which translates as the *focalMin* function chained with *focalMax* in the earth engine. Different *kernelType* are available, let's use the *Square* kernel with *30* as search *radius* as the JRC's Global Surface Water spatial resolution is 30m. We can also define a custom kernel if the required kernel is unavailable in kernelType. 

```{r echo=FALSE, fig.align='center', out.width='75%', fig.cap='Before Operation vs After Operation'}
knitr::include_graphics('images/gee_water_resources_management/morphological_operation.png')
```

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/02-Surface_Water_Mapping/06b_Create_a_Surface_Water_Map_(complete)')}
```
[Open in Code Editor &#8599;](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F06b_Create_a_Surface_Water_Map_(complete)){target="_blank"}

### Exercise

By default, the total iteration is 1. We can increase it as required, so the kernel will pass through the image to perform the operation as many times as required. 

Set the iteration parameter to 2 and execute the morphological closing operation. 

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/02-Surface_Water_Mapping/06c_Create_a_Surface_Water_Map_(exercise)')}
```

[Try in Code Editor &#8599;](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F06c_Create_a_Surface_Water_Map_(exercise)){target="_blank"}

## 07. Convert Raster to Vector

After creating a final raster map, we can vectorize it. The `reduceToVectors` function should be used with *countEvery* reducer to convert a raster to a vector. This reducer will connect all linked pixels of the same class into a single feature and return a feature collection. By default, the connectedness of a pixel is checked in cardinal directions only. To check the connectedness of a pixel in the diagonal direction, *eightConnected* should be set as *true*.



```{js eval=FALSE, code=readLines('code/gee_water_resources_management/02-Surface_Water_Mapping/07b_Convert_Raster_to_Vector_(complete)')}
```
[Open in Code Editor &#8599;](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F07b_Convert_Raster_to_Vector_(complete)){target="_blank"}

### Exercise

The final feature collection will contain the class value from the pixel as the *label* property. Since the raster is a binary image with 0 - 1 value, the label with value 1 will represent the water bodies. 

Use the *eq* filter and filter the water bodies alone.

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/02-Surface_Water_Mapping/07c_Convert_Raster_to_Vector_(exercise)')}
```

[Try in Code Editor &#8599;](https://code.earthengine.google.com/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A02-Surface_Water_Mapping%2F07c_Convert_Raster_to_Vector_(exercise)){target="_blank"}




# Module 3: Precipitation Time Series Analysis 

Time series analysis is plotting a sequence of data collected over a period of time, remote sensing observation are such big data and these data can be crunched to user needs to create a graphical representation of the data. With computation capacity Earth Engine offers this analysis can be done as large sacle. 

Unlike traditional network programming, where conditional statements and loops are used to filter data and perform a task over a collection, the Earth Engine servers use the [MapReduce](https://en.wikipedia.org/wiki/MapReduce) concept to increase the efficiency of computation power by parallel computing. Earth Engine servers don't understand anything apart from earth engine objects. So, all the objects before being mapped into a function should be converted as earth engine objects.


In this module we will take the do a find trend for a state in India, 

## 01. Mapping a Function

This script introduces the basics of the Earth Engine API. Here we map a list of objects over a function to perform a task on all the objects. While defining a function, it should return a result after computation, which can be stored in a variable. To learn more, visit the [Earth Engine Objects and Methods](https://developers.google.com/earth-engine/tutorial_js_02) section of the Earth Engine User Guide. 

Apart from other regular strings and numbers, the earth engine can understand dates. Under `ee.Date` there are many functions by which dates can be defined or converted from one format to another.

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/03-Time_Series_Analysis/01b_Mapping_a_Function_(complete)')}
```

[Open in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A03-Time_Series_Analysis%2F01b_Mapping_a_Function_(complete)){target="_blank"}

### Exercise

Define the function `createDate`, map the `weeks` list as input. The function should return dates incremented by the week number.

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/03-Time_Series_Analysis/01c_Mapping_a_Function_(exercise)')}
```
[Try in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A03-Time_Series_Analysis%2F01c_Mapping_a_Function_(exercise)){target="_blank"}


## 02. Reducers

A *Reduce* operation allows you to compute statistics on a large amount of inputs. In Earth Engine, you need to run reduction operation when creating composites, calculating statistics, doing regression analysis etc. The Earth Engine API comes with a large number of built-in reducer functions (such as `ee.Reducer.sum()`, `ee.Reducer.histogram()`, `ee.Reducer.linearFit()` etc.) that can perform a variety of statistical operations on input data. You can run reducers using the `reduce()` function. Earth Engine supports running reducers on all data structures that can hold multiple values, such as Images (reducers run on different bands), ImageCollection, FeatureCollection, List, Dictionary etc. The script below introduces basic concepts related to reducers.

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/03-Time_Series_Analysis/02b_Reducers_(complete)')}
```
[Open in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A03-Time_Series_Analysis%2F02b_Reducers_(complete)){target="_blank"}

### Exercise

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/03-Time_Series_Analysis/02c_Reducers_(exercise)')}
```
[Try in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A03-Time_Series_Analysis%2F02c_Reducers_(exercise)){target="_blank"}

## 03. Calculating Total Rainfall

The CHIRPS Pentad is 5 days composite of rainfall at ~6 km spatial resolution. To compute this data, both satellite and grounds values are considered. To read more about the methodology, visit [nature.com](https://www.nature.com/articles/sdata201566)

Using this image collection, lets filter all the images for 2017. To get a yearly rainfall we can reduce the collection using `reduce` function with `sum` reducer. This will return a single global image where each pixel represent the total annual rainfall value in *mm* at that location. Now to get a annual average rainfall at a particular region of interest, we can do a *reduceRegions* with *mean* reducer over the region. This reducer will reduce the region's image and return a dictionary. The dictionary will contain the average value of all the bands in the image, which can be extracted using the *get* function.


```{js eval=FALSE, code=readLines('code/gee_water_resources_management/03-Time_Series_Analysis/03b_Calculating_Total_Rainfall_(complete)')}
```
[Open in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A03-Time_Series_Analysis%2F03b_Calculating_Total_Rainfall_(complete)){target="_blank"}

### Exercise

Update the date rage for monsoon season(1 June - 30 September) in India. 

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/03-Time_Series_Analysis/03c_Calculating_Total_Rainfall_(exercise))')}
```
[Try in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A03-Time_Series_Analysis%2F03c_Calculating_Total_Rainfall_(exercise)){target="_blank"}

## 04. Aggregating Time Series

## 05. Charting Monthly Rainfall 

## 06. Import

You can import vector or raster data into Earth Engine. We will now import a shapefile of [Taluk](https://kgis.ksrsac.in/kgis/downloads.aspx) for a state in India. Unzip the `Taluk.rar` into a folder on your computer. In the Code Editor, go to *Assets &rarr; New &rarr; Table Upload &rarr; Shape Files*. Select the `.cpj`, `.sbn`, `.shp`, `.shx`, `.dbf`and .`prj` files. Enter `Taluk_Boundary` as the *Asset Name* and click  *Upload*. In the *Tasks* tab you can see the progress of the upload. Once the ingestion is complete you will have a new asset in the *Assets* tab. The shapefile will be imported as a Feature Collection in Earth Engine. Select the `Taluk_Boundary` asset and click *Import*.  You can then visualize the imported data.

```{r echo=FALSE, fig.align='center', out.width='75%', fig.cap='Importing a Shapefile'}
knitr::include_graphics('images/gee_water_resources_management/upload.png')
```

```{js eval=FALSE, code=readLines('code/gee_water_resources_management/03-Time_Series_Analysis/06b_Import_(complete)')}
```

[Open in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A03-Time_Series_Analysis%2F06b_Import_(complete)){target="_blank"}

### Exercise

Explore the uploaded Feature Collection, `SHAPE_STAr` property contains the area in meter_square, using `addArea` function area is converted into kilometer_square and stored as `area_sqkm`. Using this property filter all regions with area greater than 1000 kmsq. 


```{js eval=FALSE, code=readLines('code/gee_water_resources_management/03-Time_Series_Analysis/06c_Import_(exercise)')}
```
[Try in Code Editor &#8599;](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FGEE-Water-Resources-Management%3A03-Time_Series_Analysis%2F06c_Import_(exercise)){target="_blank"}

## 07. Zonal Statistics

## 08. Trend Analysis

# Module 4: Land Use Land Cover Mapping

# Module 5: Flood Mapping

# Module 6: Drought Mapping 

# Module 7: Creating Earth Engine Apps


# Data Credits

* **Sentinel-2 Level-1C, Level-2A** and **Sentinel-1 SAR GRD**: Contains Copernicus Sentinel data.
* **TerraClimate: Monthly Climate and Climatic Water Balance for Global Terrestrial Surfaces, University of Idaho**: Abatzoglou, J.T., S.Z. Dobrowski, S.A. Parks, K.C. Hegewisch, 2018, Terraclimate, a high-resolution global dataset of monthly climate and climatic water balance from 1958-2015, Scientific Data 5:170191, doi:10.1038/sdata.2017.191

* **FAO GAUL 500m: Global Administrative Unit Layers 2015, Second-Level Administrative Units**: Source of Administrative boundaries: The Global Administrative Unit Layers (GAUL) dataset, implemented by FAO within the CountrySTAT and Agricultural Market Information System (AMIS) projects.
* **CHIRPS Pentad: Climate Hazards Group InfraRed Precipitation with Station Data (version 2.0 final)**: Funk, Chris, Pete Peterson, Martin Landsfeld, Diego Pedreros, James Verdin, Shraddhanand Shukla, Gregory Husak, James Rowland, Laura Harrison, Andrew Hoell & Joel Michaelsen. "The climate hazards infrared precipitation with stations—a new environmental record for monitoring extremes". Scientific Data 2, 150066. doi:10.1038/sdata.2015.66 2015.
* **MOD13Q1.006 Terra Vegetation Indices 16-Day Global 250m**: Didan, K. (2015). <i>MOD13Q1 MODIS/Terra Vegetation Indices 16-Day L3 Global 250m SIN Grid V006</i> [Data set]. NASA EOSDIS Land Processes DAAC. Accessed 2021-05-06 from https://doi.org/10.5067/MODIS/MOD13Q1.006

# License

The course material (text, images, presentation, videos) is licensed under a [Creative Commons Attribution 4.0 International License](https://creativecommons.org/licenses/by/4.0/).

The code (scripts, Jupyter notebooks) is licensed under the MIT License. For a copy, see https://opensource.org/licenses/MIT

Kindly give appropriate credit to the original author as below:

Copyright &copy; 2021 Ujaval Gandhi [www.spatialthoughts.com](https://spatialthoughts.com)

If you would like to white-label these materials as part of a commercial offering, you can purchase a *Trainer License* for a small fee. 

Please [contact us](https://spatialthoughts.com/contact/) for pricing and terms.

***

**This course is offered as an instructor-led online class. Visit [Spatial Thoughts](https://spatialthoughts.com/events/) to know details of upcoming sessions.**