Ongoing and Completed Analysis of Sri Lankan Residential Electricity Consumption Using Multi-Round Longitudinal Surveys and Energy Provider Data

Overview

This dataset contains household electricity consumption data, and survey responses collected from households in multiple Sri Lankan cities, combining 20 months of smart-meter and non-smart-meter electricity consumption records with three rounds of longitudinal surveys conducted with 4000 households. The dataset captures household living conditions, appliance usage, and behavioral changes over time, offering an intersection of granular energy consumption and consumer-driven factors. Using this repository you can get a good understanding of how the data can be processed especially smart meter data that requires carefull processing before being utilized, how variables and features being used for priliminary analysis, and few examples of machine learning and deep-learning applications.

You can access the dataset through:

1. IEEE-dataport: https://dx.doi.org/10.21227/n1dk-q860
2. Zenodo: https://zenodo.org/records/15023048
3. Kaggle: https://www.kaggle.com/datasets/lirneasia/sri-lankan-residential-electricity-consumption

Summary of the Dataset and overview of the data collection process

The dataset is divided into two main parts:

1. Consumption Data: Includes smart meter readings and non-smart meter (monthly) consumption data.
2. Survey Data: Collected over three waves (three longitudinal surveys), providing details including household demographic, appliance usage details and there changes over time.

Each section includes a data dictionary that describes variable names and meanings. Additionally Survey Data includes data profilers for each individual data file.

---

1. Consumption Data

1.1 Non-Smart Meter Data

• Description: Monthly electricity consumption readings for 4,063 households over 20 months.
• File Path: consumption_data/non_smart_meter/
• File Name: monthly_consumption.csv
• Columns:
  • household_Id: Unique household identifier
  • month: Year-Month format (YYYY-MM)
  • consumption: Total consumed units in kilowatts (KW)

1.2 Smart Meter Data

1.2.1 15-Minute Interval Data

• Description: Smart meter readings collected every 15 minutes.

• File Path: consumption_data/smart_meter/15min_interval/

• Files and Data Points:

  • smart_15min_1.csv: 9,435,755 datapoints, collected during 2023-10-01 to 2024-03-01.
  • smart_15min_2.csv: 16,425,676 datapoints, collected during 2024-03-02 to 2024-08-01
  • smart_15min_3.csv: 19,751,849 datapoints, collected during 2024-08-02 to 2024-12-23

• Sample Columns:

  • household_Id: Unique household identifier
  • date: Date of the reading
  • time: Time of the reading
  • importkwh(kwh): : Total Cumulative consumed energy units in kilowatts (kW)
  • exportkwh(kwh): : Total Cumulative generated energy units in kilowatts (kW)

1.2.2 6-Hour Interval Data

• Description: Smart meter readings collected every 6 hours.
• File Path: consumption_data/smart_meter/6hour_interval/
• Files and Data Points:
  • smart_6hour_1.csv: 1,115,230 datapoints, collected during 2023-01-01 to 2023-06-01
  • smart_6hour_2.csv: 1,319,347 datapoints, collected during 2023-06-02 to 2023-11-01
  • smart_6hour_3.csv: 1,047,075 datapoints, collected during 2023-11-02 to 2024-04-01
  • smart_6hour_4.csv: 680,674 datapoints, collected during 2024-04-02 to 2024-09-01
  • smart_6hour_5.csv: 279,467 datapoints, collected during 2024-09-02 to 2024-12-31
• Sample Columns:
  • household_Id: Unique household identifier
  • date: Date of the reading
  • time: Time of the reading
  • TOTAL_IMPORT (kWh): : Total Cumulative consumed energy units in kilowatts (kW)
  • TOTAL_EXPORT (kWh): : Total Cumulative generated energy units in kilowatts (kW)

(See consumption_data/documentation/data_dictionary.csv for full details.)

---

2. Survey Data

2.1 Wave 1

• File Path: survey_data/wave_1/
• Tables:
  • w1_ac_roster.csv
  • w1_appliances.csv
  • w1_demographics.csv
  • w1_electricity_generation_water_heating_cooking.csv
  • w1_fan_roster.csv
  • w1_household_information_and_history.csv
  • w1_light_roster.csv
  • w1_room_roster.csv

2.2 Wave 2

• File Path: survey_data/wave_2/
• Tables:
  • w2_ac_roster.csv
  • w2_new_household_members.csv
  • w2_changed_appliances.csv
  • w2_members_who_were_in_w1.csv
  • w2_electricity_generation.csv
  • w2_room_roster.csv
  • w2_changed_rooms.csv
  • w2_appliance_usage.csv
  • w2_household_information.csv
  • w2_light_roster.csv
  • w2_fan_roster.csv
  • w2_behaviour.csv

2.3 Wave 3

• File Path: survey_data/wave_3/
• Tables:
  • w3_appliance_usage.csv
  • w3_behaviour_data.csv
  • w3_fan_roster.csv
  • w3_electricity_billing_info.csv
  • w3_light_roster.csv
  • w3_ac_roster.csv
  • w3_members_who_were_in_w2.csv
  • w3_new_household_members.csv
  • w3_changed_rooms.csv
  • w3_demolished_rooms.csv
  • w3_room_roster.csv
  • w3_awareness_of_time_of_use.csv
  • w3_household_information.csv
  • w3_electricity_generation.csv
  • w3_changed_appliances.csv

2.4 Survey Dates

• File Path: survey_data/
• File Name: surveyed_dates.csv
• Description: Contains timestamps of when surveys were conducted for each household.

(See survey_data/documentation/data_dictionaries and survey_data/documentation/table_structures for details.)

---

Documentation

Each dataset component has an accompanying data dictionary and table structure file stored in the documentation/ folder:

• data_dictionary.csv: Definitions of column names and data types.
• table_structure.csv: Relationships between tables and descriptions.