Skip to content

Commit

Permalink
Merge pull request #9848 from NREL/csv-iso-8601
Browse files Browse the repository at this point in the history 
Add ISO 8601 format support to native CSV and JSON output
  • Loading branch information
@Myoldmopar
Myoldmopar authored Feb 22, 2023
2 parents 8b12b8e + 609fd68 commit ff86a13
Show file tree
Hide file tree
Showing 10 changed files with 4,047 additions and 57 deletions.
62 changes: 62 additions & 0 deletions doc/input-output-reference/src/input-for-output.tex
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2381,3 +2381,65 @@ \subsubsection{Inputs}\label{outputcontrolfiles-inputs}
No , ! ExtShd
No ; ! Tarcog
\end{lstlisting}

\subsection{OutputControl:Timestamp}\label{outputcontrol_timestamp}

Controls the format of the timestamp used in the native CSV and JSON outputs. The default timestamp is formated
as ``$MM$/$DD$ $hh$:$mm$:$ss$'', where

\begin{wherelist}
\item[MM] the two-digit month,
\item[DD] the two-digit day of the month,
\item[hh] the two-digit hours since the beginning of the day,
\item[mm] the two-digit minutes since the beginning of the last hour, and
\item[ss] the two-digit seconds (which is always ``00'').
\end{wherelist}

The time portion of this timestamp has traditionally been the elapsed time since the beginning of the day and associates values
with the elapsed time at the end of the interval of interest. For example, the timestamp for the interval from 1~AM to 2~AM on
December 7 is written as ``12/07 02:00:00''. Unfortunately, some common software used to postprocess results has trouble
interpreting the last timestamp for each day, which will include ``24:00:00''. An alternative that avoids this problem is to
associate values with the elapsed time at the beginning of the interval of interest. In this case, the interval from 1~AM to
2~AM on December 7 is written as ``12/07 01:00:00'' and 24 will never appear as the hours value (but 00 will). The ISO 8601
timestamp format is ``$YYYY$-$MM$-$DD$T$hh$:$mm$:$ss$'' where

\begin{wherelist}
\item[YYYY] the four-digit year,
\item[MM] the two-digit month,
\item[DD] the two-digit day of the month,
\item[hh] the two-digit hours since the beginning of the day,
\item[mm] the two-digit minutes since the beginning of the last hour,
\item[ss] the two-digit seconds (which is always ``00''),
\end{wherelist}

and a capital letter ``T'' separates the time portion of the timestamp from the date portion. The timestamp for the interval from
1~AM to 2~AM on December 7, 2022 is written either as

\begin{itemize}
\item ``2022-12-07T02:00:00'' at the end of the interval or
\item ``2022-12-07T01:00:00'' at the beginning of the interval.
\end{itemize}

Output workflows using the ReadVars postprocessing program will not be impacted by the selections made here.

\subsubsection{Inputs}\label{inputs-for-timestamp}

\paragraph{Field: ISO 8601 Format}\label{field-timestamp-iso8601format}

This field determines whether the ISO 8601 or traditional format is used. If the field is ``\textbf{No}'', the traditional format is
used. If the field is ``\textbf{Yes}'', the ISO 8601 format is used. The default is ``\textbf{No}''.

\paragraph{Field: Timestamp at Beginning of Interval}\label{field-timestamp-beginning}

This field determines whether the timestamp is placed at the beginning of the interval (``\textbf{Yes}'') or at the end of the
interval (``\textbf{No}''). The default is ``\textbf{No}''.

\subsubsection{Example}\label{example-for-timestamp}

An example IDF input object that uses the ISO 8601 format and places the timestamp at beginning of the interval.
\begin{lstlisting}
Output:SQLite,
Yes, !- ISO 8601 Format
Yes; !- Timestamp at Beginning of Interval
\end{lstlisting}

16 changes: 16 additions & 0 deletions idd/Energy+.idd.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -105700,6 +105700,22 @@ OutputControl:Files,
\key No
\default Yes

OutputControl:Timestamp,
\memo Control timestamp format, currently applies only to JSON and native CSV (not CSV via ReadVars)
\unique-object
A1, \field ISO 8601 Format
\note Use the ISO 8601 format YYYY-MM-DDThh:mm:ss
\type choice
\key Yes
\key No
\default No
A2; \field Timestamp at Beginning of Interval
\note Determines where the timestamp is produced, either at the beginning (Yes) or end (No) of the interval
\type choice
\key Yes
\key No
\default No

Output:JSON,
\memo Output from EnergyPlus can be written to JSON format files.
\unique-object
Expand Down
27 changes: 24 additions & 3 deletions src/EnergyPlus/IOFiles.cc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@
#include "FileSystem.hh"
#include "InputProcessing/EmbeddedEpJSONSchema.hh"
#include "InputProcessing/InputProcessor.hh"
#include "ResultsFramework.hh"
#include "UtilityRoutines.hh"

#include <algorithm>
Expand Down Expand Up @@ -316,8 +317,9 @@ std::vector<std::string> InputOutputFile::getLines()

void IOFiles::OutputControl::getInput(EnergyPlusData &state)
{
auto const instances = state.dataInputProcessing->inputProcessor->epJSON.find("OutputControl:Files");
if (instances != state.dataInputProcessing->inputProcessor->epJSON.end()) {
auto &ip = state.dataInputProcessing->inputProcessor;
auto const instances = ip->epJSON.find("OutputControl:Files");
if (instances != ip->epJSON.end()) {

auto find_input = [=, &state](nlohmann::json const &fields, std::string const &field_name) -> std::string {
std::string input;
Expand Down Expand Up @@ -345,7 +347,7 @@ void IOFiles::OutputControl::getInput(EnergyPlusData &state)
for (auto instance = instancesValue.begin(); instance != instancesValue.end(); ++instance) {
auto const &fields = instance.value();

state.dataInputProcessing->inputProcessor->markObjectAsUsed("OutputControl:Files", instance.key());
ip->markObjectAsUsed("OutputControl:Files", instance.key());

{ // "output_csv"
csv = boolean_choice(find_input(fields, "output_csv"));
Expand Down Expand Up @@ -442,6 +444,25 @@ void IOFiles::OutputControl::getInput(EnergyPlusData &state)
}
}
}

auto const timestamp_instances = ip->epJSON.find("OutputControl:Timestamp");
if (timestamp_instances != ip->epJSON.end()) {
auto const &instancesValue = timestamp_instances.value();
for (auto instance = instancesValue.begin(); instance != instancesValue.end(); ++instance) {
auto const &fields = instance.value();
ip->markObjectAsUsed("OutputControl:Timestamp", instance.key());

auto item = fields.find("iso_8601_format");
if (item != fields.end()) {
state.dataResultsFramework->resultsFramework->setISO8601(item->get<std::string>() == "Yes");
}

item = fields.find("timestamp_at_beginning_of_interval");
if (item != fields.end()) {
state.dataResultsFramework->resultsFramework->setBeginningOfInterval(item->get<std::string>() == "Yes");
}
}
}
}

void IOFiles::flushAll()
Expand Down
43 changes: 26 additions & 17 deletions src/EnergyPlus/OutputProcessor.cc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2465,7 +2465,7 @@ namespace OutputProcessor {
ScheduleManager::dayTypeNames[CurDayType]);
if (state.dataResultsFramework->resultsFramework->TSMeters.rDataFrameEnabled()) {
state.dataResultsFramework->resultsFramework->TSMeters.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, EndMinute);
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, EndMinute, state.dataGlobal->CalendarYear);
}
PrintTimeStamp = false;
PrintTimeStampToSQL = false;
Expand Down Expand Up @@ -2573,7 +2573,7 @@ namespace OutputProcessor {
ScheduleManager::dayTypeNames[CurDayType]);
if (state.dataResultsFramework->resultsFramework->HRMeters.rDataFrameEnabled()) {
state.dataResultsFramework->resultsFramework->HRMeters.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0);
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0, state.dataGlobal->CalendarYear);
}
PrintTimeStamp = false;
PrintTimeStampToSQL = false;
Expand Down Expand Up @@ -2656,7 +2656,7 @@ namespace OutputProcessor {
ScheduleManager::dayTypeNames[CurDayType]);
if (state.dataResultsFramework->resultsFramework->DYMeters.rDataFrameEnabled()) {
state.dataResultsFramework->resultsFramework->DYMeters.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0);
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0, state.dataGlobal->CalendarYear);
}
PrintTimeStamp = false;
PrintTimeStampToSQL = false;
Expand Down Expand Up @@ -2729,7 +2729,7 @@ namespace OutputProcessor {
state.dataEnvrn->Month);
if (state.dataResultsFramework->resultsFramework->MNMeters.rDataFrameEnabled()) {
state.dataResultsFramework->resultsFramework->MNMeters.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0);
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0, state.dataGlobal->CalendarYear);
}
PrintTimeStamp = false;
PrintTimeStampToSQL = false;
Expand Down Expand Up @@ -2795,7 +2795,7 @@ namespace OutputProcessor {
state, state.files.mtr, op->YearlyStampReportChr, state.dataGlobal->CalendarYearChr, PrintTimeStamp && PrintTimeStampToSQL);
if (state.dataResultsFramework->resultsFramework->YRMeters.rDataFrameEnabled()) {
state.dataResultsFramework->resultsFramework->YRMeters.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0);
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0, state.dataGlobal->CalendarYear);
}
PrintTimeStamp = false;
PrintTimeStampToSQL = false;
Expand Down Expand Up @@ -2872,7 +2872,7 @@ namespace OutputProcessor {
PrintTimeStamp && PrintTimeStampToSQL);
if (state.dataResultsFramework->resultsFramework->SMMeters.rDataFrameEnabled()) {
state.dataResultsFramework->resultsFramework->SMMeters.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0);
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0, state.dataGlobal->CalendarYear);
}
PrintTimeStamp = false;
PrintTimeStampToSQL = false;
Expand Down Expand Up @@ -4991,13 +4991,19 @@ void UpdateDataandReport(EnergyPlusData &state, OutputProcessor::TimeStepType co

if (state.dataResultsFramework->resultsFramework->timeSeriesEnabled()) {
if (t_TimeStepTypeKey == TimeStepType::Zone) {
state.dataResultsFramework->resultsFramework->RIDetailedZoneTSData.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, op->TimeValue.at(TimeStepType::Zone).CurMinute);
state.dataResultsFramework->resultsFramework->RIDetailedZoneTSData.newRow(state.dataEnvrn->Month,
state.dataEnvrn->DayOfMonth,
state.dataGlobal->HourOfDay,
op->TimeValue.at(TimeStepType::Zone).CurMinute,
state.dataGlobal->CalendarYear);
}
if (t_TimeStepTypeKey == TimeStepType::System) {
// TODO this was an error probably, was using TimeValue(1)
state.dataResultsFramework->resultsFramework->RIDetailedHVACTSData.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, op->TimeValue.at(TimeStepType::System).CurMinute);
state.dataResultsFramework->resultsFramework->RIDetailedHVACTSData.newRow(state.dataEnvrn->Month,
state.dataEnvrn->DayOfMonth,
state.dataGlobal->HourOfDay,
op->TimeValue.at(TimeStepType::System).CurMinute,
state.dataGlobal->CalendarYear);
}
}

Expand Down Expand Up @@ -5187,8 +5193,11 @@ void UpdateDataandReport(EnergyPlusData &state, OutputProcessor::TimeStepType co
state.dataResultsFramework->resultsFramework->initializeITSDataFrame(
ReportingFrequency::TimeStep, op->IVariableTypes, op->NumOfIVariable);
}
state.dataResultsFramework->resultsFramework->RITimestepTSData.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, op->TimeValue.at(TimeStepType::Zone).CurMinute);
state.dataResultsFramework->resultsFramework->RITimestepTSData.newRow(state.dataEnvrn->Month,
state.dataEnvrn->DayOfMonth,
state.dataGlobal->HourOfDay,
op->TimeValue.at(TimeStepType::Zone).CurMinute,
state.dataGlobal->CalendarYear);
}

for (auto &thisTimeStepType : {TimeStepType::Zone, TimeStepType::System}) { // Zone, HVAC
Expand Down Expand Up @@ -5358,7 +5367,7 @@ void UpdateDataandReport(EnergyPlusData &state, OutputProcessor::TimeStepType co
ReportingFrequency::Hourly, op->IVariableTypes, op->NumOfIVariable);
}
state.dataResultsFramework->resultsFramework->RIHourlyTSData.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0);
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0, state.dataGlobal->CalendarYear);
}

for (auto &thisTimeStepType : {TimeStepType::Zone, TimeStepType::System}) { // Zone, HVAC
Expand Down Expand Up @@ -5461,7 +5470,7 @@ void UpdateDataandReport(EnergyPlusData &state, OutputProcessor::TimeStepType co
ReportingFrequency::Daily, op->IVariableTypes, op->NumOfIVariable);
}
state.dataResultsFramework->resultsFramework->RIDailyTSData.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0);
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0, state.dataGlobal->CalendarYear);
}

op->NumHoursInMonth += 24;
Expand Down Expand Up @@ -5510,7 +5519,7 @@ void UpdateDataandReport(EnergyPlusData &state, OutputProcessor::TimeStepType co
ReportingFrequency::Monthly, op->IVariableTypes, op->NumOfIVariable);
}
state.dataResultsFramework->resultsFramework->RIMonthlyTSData.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0);
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0, state.dataGlobal->CalendarYear);
}

op->NumHoursInSim += op->NumHoursInMonth;
Expand Down Expand Up @@ -5557,7 +5566,7 @@ void UpdateDataandReport(EnergyPlusData &state, OutputProcessor::TimeStepType co
ReportingFrequency::Simulation, op->IVariableTypes, op->NumOfIVariable);
}
state.dataResultsFramework->resultsFramework->RIRunPeriodTSData.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0);
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0, state.dataGlobal->CalendarYear);
}
for (auto &thisTimeStepType : {TimeStepType::Zone, TimeStepType::System}) { // Zone, HVAC
for (int Loop = 1; Loop <= op->NumOfRVariable; ++Loop) {
Expand Down Expand Up @@ -5594,7 +5603,7 @@ void UpdateDataandReport(EnergyPlusData &state, OutputProcessor::TimeStepType co
ReportingFrequency::Yearly, op->IVariableTypes, op->NumOfIVariable);
}
state.dataResultsFramework->resultsFramework->RIYearlyTSData.newRow(
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0);
state.dataEnvrn->Month, state.dataEnvrn->DayOfMonth, state.dataGlobal->HourOfDay, 0, state.dataGlobal->CalendarYear);
}
for (auto &thisTimeStepType : {TimeStepType::Zone, TimeStepType::System}) { // Zone, HVAC
for (int Loop = 1; Loop <= op->NumOfRVariable; ++Loop) {
Expand Down
Loading

5 comments on commit ff86a13

@nrel-bot-3
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

develop (Myoldmopar) - x86_64-MacOS-10.17-clang-13.0.0: OK (2616 of 2616 tests passed, 0 test warnings)

Build Badge Test Badge

@nrel-bot
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

develop (Myoldmopar) - Win64-Windows-10-VisualStudio-16: OK (2615 of 2615 tests passed, 0 test warnings)

Build Badge Test Badge

@nrel-bot-2c
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

develop (Myoldmopar) - x86_64-Linux-Ubuntu-18.04-gcc-7.5: OK (2637 of 2637 tests passed, 0 test warnings)

Build Badge Test Badge

@nrel-bot-2c
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

develop (Myoldmopar) - x86_64-Linux-Ubuntu-18.04-gcc-7.5-UnitTestsCoverage-Debug: OK (1843 of 1843 tests passed, 0 test warnings)

Build Badge Test Badge Coverage Badge

@nrel-bot-2
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

develop (Myoldmopar) - x86_64-Linux-Ubuntu-18.04-gcc-7.5-IntegrationCoverage-Debug: OK (776 of 776 tests passed, 0 test warnings)

Build Badge Test Badge Coverage Badge

Please sign in to comment.