Skip to content

Commit

Permalink
Release 2.20.0 (again) (#4273)
Browse files Browse the repository at this point in the history
* Remove Deprecated APIs (#4049)

* Remove deprecated endpoints

* Update tests

* Remove additional code related to API v2

* Add ubid_threshold to expected org response

* Lock `google-chrome` to v114 (#4165)

Lock chrome version to webdriver version

* Bump word-wrap from 1.2.3 to 1.2.4 (#4154)

Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4.
- [Release notes](https://github.com/jonschlinkert/word-wrap/releases)
- [Commits](jonschlinkert/word-wrap@1.2.3...1.2.4)

---
updated-dependencies:
- dependency-name: word-wrap
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alex Swindler <[email protected]>

* Fix a few code-formatting issues (#4172)

* run autopep to cleanup a few straggling issues

* Run precommit against migrations

* Cleanup unnecessary logging

---------

Co-authored-by: Alex Swindler <[email protected]>

* Add Open Street Map to CSP rules (#4169)

* add error handling for sf edge case (#4159)

add error handldling for sf edge case

Co-authored-by: Hannah Eslinger <[email protected]>
Co-authored-by: Alex Swindler <[email protected]>

* Remove ID from inventory document display name (#4125)

remove id from inventory doc on inv det

* Set the MEDIA_ROOT path back to original value during test (#4174)

* set the mediaroot path back to where it started

* naming

* Optimize list analyses endpoint (#4206)

* Optimize list analyses endpoint

* Add outputfiles back in

* Fix

* Make it even faster

* Move analyses endpoint (#4210)

* Add new `create_property` and `update_with_espm` endpoint (#4012)

* 179d create_property endpoint

* precommit

* adding property id, state id, and view id to create response

* Partial fixes for handling bad data

* Verify property exists and belongs to org

* start of new serializer and controlling extra_data fields for 179d

* fix property create endpoint

* Formatting

* fix org_id on column views

* add report_format to PM download so spreadsheet can be downloaded

* updating BAE and small tweak since assets are now typed

* update BAE processing in reader

* updating BAE to v0.1.14

* remove uneeded view

* Add ESPM download for single property (via API) (#4158)

* add single property download method

* add viewset for downloading single espm report

* fix BAE migration and update swagger schema

* rename pm report single to download

* update lokalise notes and pull from latest from website

* initial commit with a new update with ESPM function. need to get the backend method finished

* enable udpate_with_espm

* add test and column mapping profile helper to load in the data

* fix example file that was downloaded from espm

* fix comment formats

* file paths

* style espm modal and fix both upload methods

* put tmp files into the projects media dir

* add column mapping profile param to update_with_espm

* put tmp files into the projects media dir

* add status to dresponse for pyseed

* do not save in media_root

* precommit

* allow different urls for better and at during testing

* return the mapping profile object when creating from file

* add tests to ensure read_only fields are protected

* Update models.py

* Update models.py

* - Removed broken `property_id` option
- Fixed extra_data columns to only ones that actually apply
- Changed `merge_state` from 0 to 1
- Added ValueError handling

* Small fixes

* Fix 179d ESPM update endpoint to run in foreground with Celery enabled (#4211)

* Fixed bae object accessors

* create two new methods for running save_raw and map to the foreground

* fix type

---------

Co-authored-by: Alex Swindler <[email protected]>

---------

Co-authored-by: Nicholas Long <[email protected]>
Co-authored-by: Alex Swindler <[email protected]>
Co-authored-by: Nicholas Long <[email protected]>

* Fix display name (#4223)

* Fix refresh_metadata (#4221)

* Fix refresh_metadata

* Rename

* Bump uwsgi from 2.0.17.1 to 2.0.22 in /requirements (#4209)

Bumps [uwsgi](https://github.com/unbit/uwsgi-docs) from 2.0.17.1 to 2.0.22.
- [Commits](https://github.com/unbit/uwsgi-docs/commits)

---
updated-dependencies:
- dependency-name: uwsgi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alex Swindler <[email protected]>

* Remove Beta label from text of analysis button  (#4225)

remove beta from analysis

Co-authored-by: Nicholas Long <[email protected]>

* Fix Meters Without `source_id` (#4229)

Fix showing meter readings for meters without `source_id`

* Fix performance for fetching analyses and force user email to lowercase (#4213)

* Set username to lowercase when updating user info

* Removed unnecessary prefetch from analyses retrieval

---------

Co-authored-by: Hannah Eslinger <[email protected]>
Co-authored-by: Nicholas Long <[email protected]>

* EEEJ Analysis Functionality (#4208)

* eeej analysis

* precommit

* precommit

* add EJScreen

* update eeej analysis to link to EJScreen report

* adding analysis description for eeej

* Fix display name

* fix for analysis reruns

* cleanup

* fix eeej tests

* cleanup

* fix searching for string in a NoneType

* set manual geocoding

* fix geocode tests and add census to geocode summary

* fix typos

* geocode with lat/lng over address if available

* add copyright text

* small doc formatting cleanup

* cleanup

* oops, missed this formatting

---------

Co-authored-by: Nicholas Long <[email protected]>
Co-authored-by: Hannah Eslinger <[email protected]>
Co-authored-by: Nicholas Long <[email protected]>

* Show Analysis `debug_message` If Available (#4198)

Show analysis debug_message if available

Co-authored-by: Katherine Fleming <[email protected]>

* Use display name in custom reports legend (#4060)

* Use display name in custom reports legend

* Fix docker compose

* Fix Tests

* Address comment

* Clean

---------

Co-authored-by: Katherine Fleming <[email protected]>

* Allow analysis property view related properties to be null (#4227)

* update ondelete buildingfile > event > scenarios

* update relationhsip to null when a property is deleted

* add warning triangle

* remove checkbox to delete analyses with properties

* precommit

* reorder migrations

* reset

* remove commit from outside branch

* add warning to analyses page

* styles

* Update migrations

---------

Co-authored-by: Hannah Eslinger <[email protected]>

* Add more API documentation for meter and meter readings (#4237)

* add more api doc info on meter and meter reading endpoints

* Improved nested documentation, added swagger support for enums, sorted meter readings

---------

Co-authored-by: Alex Swindler <[email protected]>

* Update inventory selection to include current inventory type only (#3862)

filter selection by current invetory type and return its length

Co-authored-by: Alex Swindler <[email protected]>

* Update default meter range selection for BETTER analysis (#3819)

* Show all navs in org

* reverse the default better meter selection

* Allow analysis to be by range or cycle

* html updates to add cycle to BETTER

* revert to allow merge

* select cycle for better

* update docker-dev better api endpoint

* modify valid meter data to accept cycle start and end dates

* precommit

* formatting

* remove console and updat better url

* merge conflicts

* order

* default current cycle to eui and better on select_cycle

* Simplified

---------

Co-authored-by: Hannah Eslinger <[email protected]>
Co-authored-by: Alex Chapin <[email protected]>
Co-authored-by: Ryo <[email protected]>
Co-authored-by: Alex Swindler <[email protected]>

* Delete BuildingFiles and Related Events on PropertyState delete (#4217)

* update ondelete buildingfile > event > scenarios

* precommit

* delete scenario if event is deleted

* cascade over nothing

* Renumber migration

---------

Co-authored-by: Alex Swindler <[email protected]>

* Fixed Issues with Scenario/Measure CRUD Operations (#4240)

Fixed issues with scenario/measure CRUD operations, cherry-picked from a640926

* Add a cycle indicator to the analysis modal and filter analyses for current cycle (#3837)

* expose current cycle on new analysis modal

* display analysis associated with cycle

* refactor help text

* add cycle indicator to analysis_card

* syntax

* refactor for clarity

* add cycle_name to analysis display tables

* add cycle dropdown to inventory detail analyses page

* find analyses relavent to current cycle

* precommit

* formatting

* remove debug logs

* clarifying note

* consistency

* Fixed ui-router navigation

---------

Co-authored-by: Alex Swindler <[email protected]>

* Disable ability to remove matching criteria fields once inventory is added (#4222)

* update ondelete buildingfile > event > scenarios

* base for locked matching criteria

* refactor for clarity

* remove matching preview from frontend

* remove matching preview from backend

* translations

* precommit

* remove unrelated migration

* revert

* update tests

---------

Co-authored-by: Hannah Eslinger <[email protected]>
Co-authored-by: Alex Swindler <[email protected]>

* Add smaller EEEJ data files for testing (#4243)

add smaller EEEJ data files for testing

* Add Uniformat Table (#4238)

* Add uniformat

* Minor fixes

---------

Co-authored-by: Nicholas Long <[email protected]>

* Fix Meter Overwrites (#4250)

* add test to make sure meter readings do not overwrite

* Fixed major bug where inserting a single meter reading would delete any existing readings for any meter with the same start time
- Added the ability for inserting bulk meter readings to update existing data
- Sanitized sql inputs in `copy_readings`

* Simplification

* EEEJ test fix

* add test to test overwrite of bulk meter import

---------

Co-authored-by: Nicholas Long <[email protected]>
Co-authored-by: Nicholas Long <[email protected]>

* Fix filter group dropdown bugs (#4249)

Fix for filter group dropdown issues

Co-authored-by: Alex Swindler <[email protected]>

* Add Import Export to Audit Template (#4215)

* base for at get buildings

* base batch at import functional from frontend button

* frontend confirm before upating with at

* frontend triggers backend batch update

* functional but missing real time progress updates

* functional with progress

progress is moving, but cant get back to frontend

properties updating, progress bar only 50%

functional with progress

* cleanup

* precommit

* summary and date formatting

* bad credentials guards

* test for full at import and property update

* precommit

* sample buildingsync xml file

* swagger documentation

* request validation

* precommit

* comment cleanup

* comment cleanup

* improve tests and disable update until selection

* translate

* lokalise translations

* function description

* update view permissions

* clarify error message logic

* precommit

* add name to at building summary

* add at_updated_at to pstate on at import, prevent duplicate uploads

* create column on at import to display and compare

* include translation

* update tests

* precommit

* precommit

* remove translation

* build bsync xml from pstate

* multiple iterations for xml export

* base for batch export to at

* add audit_template_report_type to org model

* test cleanup

* add audit_template_report_type to org model

* add at report type to org settings

* update batch test

* toggle at password visibility

* move batch export to celery

* export properties to at from inv list actions

* backfill org audit_template_report_type

* mock tests for export to audit template

* phrasing

* refactor export results

* build out export results table

* precommit

* revert

* precommit

* download current lokalise files

* add default display field to failed exports

* incorporate audit template report type in test

* request body to be a dict not list

* add translations

* update ondelete buildingfile > event > scenarios

* remove property name from inventory and expand modal

* update states to reflect at_updated

* add export to at from inv det actions

* precommit

* merge develop

* migration conflict

* Renumber migrations

* Small fixes

* Privilege enforcement

---------

Co-authored-by: Alex Swindler <[email protected]>

* Fix EEEJ analysis errors when there are no lat/long or address (#4247)

* fix eeej analysis errors

* add eeej to tasks and fix for non-eager celery

* Fix for dict key types changing based on ALWAYS_EAGER

* fixes!

* Simplification

---------

Co-authored-by: Nicholas Long <[email protected]>
Co-authored-by: Alex Swindler <[email protected]>

* Add envvar to specify loading test EEEJ dataset (#4254)

* add envvar to specify loading test EEEJ dataset

* slight mod to support strings as boolean values

---------

Co-authored-by: Nicholas Long <[email protected]>

* Default to excluding Address Line 1 as a matching criteria (#4256)

* Default to excluding Address Line 1 for matching

* [WIP] Fix some tests

* update tests for address_line_1 default matching criteria removal

---------

Co-authored-by: Nicholas Long <[email protected]>
Co-authored-by: Hannah Eslinger <[email protected]>
Co-authored-by: Nicholas Long <[email protected]>

* Bump version (#4259)

* bump versions

* current change log

* add remaining two PRs before release

* Refactor map page for functionality and performance (#4260)

* Major map/eeej/layout improvements

* Added tract shading, database indices

* Fix test failure

* Release 2.20.0 (#4262)

Merge main release commit into develop.

* Fix the items in the 2.20.0 changelog  (#4263)

put the latest PRs in the right place

* ArcGIS CSP rule (#4265)

Add ArcGIS CSP rule

* Add documentation for cleaning up conflicting column names during migration (#4264)

* add more documentation around cleaning up columns

* Update docs/source/migrations.rst

Co-authored-by: Katherine Fleming <[email protected]>

* Update docs/source/migrations.rst

Co-authored-by: Katherine Fleming <[email protected]>

---------

Co-authored-by: Katherine Fleming <[email protected]>

* Update changelog for 2.20.0 (#4268)

add 2 remaining items for release

* Fix edge case for map label filtering (#4270)

Fix ids that are used for map filtering

* Update changelog (#4272)

one more entry

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Alex Swindler <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Katherine Fleming <[email protected]>
Co-authored-by: Hannah Eslinger <[email protected]>
Co-authored-by: Ross Perry <[email protected]>
Co-authored-by: ebeers-png <[email protected]>
Co-authored-by: Alex Chapin <[email protected]>
Co-authored-by: Ryo <[email protected]>
  • Loading branch information
9 people authored Oct 6, 2023
1 parent eda7c0b commit e8820eb
Show file tree
Hide file tree
Showing 4 changed files with 29 additions and 9 deletions.
7 changes: 6 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# SEED Version 2.20.0

<!-- Release notes generated using configuration in .github/release.yml at prep-release-2.20.0 -->
<!-- Release notes generated using configuration in .github/release.yml at develop -->

## What's Changed
### New Features 🎉
Expand All @@ -18,6 +18,8 @@
* Update default meter range selection for BETTER analysis by @perryr16 in https://github.com/SEED-platform/seed/pull/3819
* Add a cycle indicator to the analysis modal and filter analyses for current cycle by @perryr16 in https://github.com/SEED-platform/seed/pull/3837
* Add envvar to specify loading test EEEJ dataset by @kflemin in https://github.com/SEED-platform/seed/pull/4254
* Default to excluding Address Line 1 as a matching criteria by @axelstudios in https://github.com/SEED-platform/seed/pull/4256
* Refactor map page for functionality and performance by @axelstudios in https://github.com/SEED-platform/seed/pull/4260
### Maintenance 🧹
* Remove Deprecated APIs by @axelstudios in https://github.com/SEED-platform/seed/pull/4049
* Lock `google-chrome` to v114 by @axelstudios in https://github.com/SEED-platform/seed/pull/4165
Expand All @@ -26,6 +28,7 @@
* Bump uwsgi from 2.0.17.1 to 2.0.22 in /requirements by @dependabot in https://github.com/SEED-platform/seed/pull/4209
* Allow analysis property view related properties to be null by @perryr16 in https://github.com/SEED-platform/seed/pull/4227
* Disable ability to remove matching criteria fields once inventory is added by @perryr16 in https://github.com/SEED-platform/seed/pull/4222
* Add documentation for cleaning up conflicting column names during migration by @nllong in https://github.com/SEED-platform/seed/pull/4264
### Bug Fixes 🐛
* Add Open Street Map to CSP rules by @axelstudios in https://github.com/SEED-platform/seed/pull/4169
* add error handling for sf edge case by @kflemin in https://github.com/SEED-platform/seed/pull/4159
Expand All @@ -39,6 +42,8 @@
* Fix Meter Overwrites by @axelstudios in https://github.com/SEED-platform/seed/pull/4250
* Fix filter group dropdown bugs by @ebeers-png in https://github.com/SEED-platform/seed/pull/4249
* Fix EEEJ analysis errors when there are no lat/long or address by @kflemin in https://github.com/SEED-platform/seed/pull/4247
* ArcGIS CSP rule by @axelstudios in https://github.com/SEED-platform/seed/pull/4265
* Fix edge case for map label filtering by @axelstudios in https://github.com/SEED-platform/seed/pull/4270


**Full Changelog**: https://github.com/SEED-platform/seed/compare/v2.19.0...v2.20.0
Expand Down
4 changes: 3 additions & 1 deletion docker/nginx/seed-csp.conf
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
# https://www.html5rocks.com/en/tutorials/security/content-security-policy/#inline-code-considered-harmful
set $DEFAULT "default-src 'self'";

set $CONNECT "connect-src 'self' https://services.arcgis.com/";

set $SCRIPT "script-src 'self' 'unsafe-inline' 'unsafe-eval'";
set $SCRIPT "${SCRIPT} https://better-lbnl-development.herokuapp.com";
set $SCRIPT "${SCRIPT} https://better.lbl.gov";
Expand Down Expand Up @@ -42,4 +44,4 @@ set $IMG "${IMG} https://validator.swagger.io";

set $OBJECT "object-src 'none'";

set $CSP "${DEFAULT}; ${SCRIPT}; ${STYLE}; ${FONT}; ${FRAME}; ${IMG}; ${OBJECT}";
set $CSP "${DEFAULT}; ${CONNECT}; ${SCRIPT}; ${STYLE}; ${FONT}; ${FRAME}; ${IMG}; ${OBJECT}";
14 changes: 13 additions & 1 deletion docs/source/migrations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,19 @@ Version 2.20.0

Version 2.19.0
--------------
- There are no special migrations needed for this version. Simply run `./manage.py migrate`.
- Run `./manage.py migrate`.
- There is a new migration in this release that requires column names to be unique across `organization`, `table_name`, and `is_extra_data`. This migration will fail if there are duplicate column names. If you have duplicate column names, you will need to manually fix them in your database before running the migration. The following steps will help you identify and fix the duplicate column names:
- Check the organization age to gauge the impact of the change. If it is a deprecated org, impact of the change will be low. Often this issue arose in older organizations when units were not part of the columns. The old mapping columns were not upserts with the units, so typically the columns impacted are the ones with units.
- Query the `seed_column` table for the organization and column name displayed on the screen (e.g., `organization_id = 300 and column_name = 'Source EUI (kBtu/ft2)'`). If there is no `table_name` set, it is likely an import file column name and can easily be cleaned up without causing issues. In such cases, there will be two rows, and you want to keep the one with the `units_pint` column set.
- More complex columns may require deleting or updating the `column_id` in the `seed_columnmapping_*` tables. If there is a foreign key constraint with `seed_columnmapping_*`, take note of the ID you want to remove and the ID you want it to be replaced with (preferably keep the one with units_pint).
- If the constraint is on `seed_columnmapping_column_raw`:
- The field should be an import file column (i.e., no `table_name` item). Query for the old column in `seed_columnmapping_column_raw` (e.g., `column_name = <old_id>`).
- Replace the old ID with the new one. If it errors because it already exists, then the row can be deleted.
- Return to the `seed_column` table and remove the old ID.
- If the constraint is on `seed_columnmapping_column_mapped`:
- The mapped column should have a `table_name` in the field. If not, it is likely an older organization.
- If there is no `table_name`, remove the row from the `seed_columnmapping_column_mapped` table.
- Return to the `seed_column` table and remove the old ID.

Version 2.18.1
--------------
Expand Down
13 changes: 7 additions & 6 deletions seed/static/seed/js/controllers/inventory_map_controller.js
Original file line number Diff line number Diff line change
Expand Up @@ -556,21 +556,22 @@ angular.module('BE.seed.controller.inventory_map', [])
return rerenderPoints(records);
}

const viewIdProperty = isPropertiesTab ? 'property_view_id' : 'taxlot_view_id';
if ($scope.labelLogic === 'and') {
// Find properties/taxlots with all labels
const ids = new Set($scope.selected_labels.map(({is_applied}) => is_applied).reduce((acc, ids) => acc.filter(id => ids.includes(id))));
records = $scope.geocoded_data.filter(({id}) => ids.has(id));
const viewIds = new Set($scope.selected_labels.map(({is_applied}) => is_applied).reduce((acc, ids) => acc.filter(id => ids.includes(id))));
records = $scope.geocoded_data.filter((record) => viewIds.has(record[viewIdProperty]));
} else if ($scope.labelLogic === 'or') {
// Find properties/taxlots with any label
const ids = $scope.selected_labels.map(({is_applied}) => is_applied).reduce((acc, ids) => {
const viewIds = $scope.selected_labels.map(({is_applied}) => is_applied).reduce((acc, ids) => {
for (const id of ids) acc.add(id);
return acc;
}, new Set());
records = $scope.geocoded_data.filter(({id}) => ids.has(id));
records = $scope.geocoded_data.filter((record) => viewIds.has(record[viewIdProperty]));
} else if ($scope.labelLogic === 'exclude') {
// Find properties/taxlots with all labels, return everything else
const ids = new Set($scope.selected_labels.map(({is_applied}) => is_applied).reduce((acc, ids) => acc.filter(id => ids.includes(id))));
records = $scope.geocoded_data.filter(({id}) => !ids.has(id));
const viewIds = new Set($scope.selected_labels.map(({is_applied}) => is_applied).reduce((acc, ids) => acc.filter(id => ids.includes(id))));
records = $scope.geocoded_data.filter((record) => !viewIds.has(record[viewIdProperty]));
}

inventory_service.saveSelectedLabels(localStorageLabelKey, $scope.selected_labels.map(({id}) => id));
Expand Down

0 comments on commit e8820eb

Please sign in to comment.