v3.1.0

Researchers

Researchers can now expand/collapse the Guidance and Comments sections when editing their DMPs. This gives the researcher more space to see their answers. Clicking on the 'Comments & Guidance' label will collapse/expand the section for that question. (Thanks to @benjaminfaure for contributing this feature #3123)

Screenshot of the new collapsible guidance/comments section:

Changes to the 'Project Details' page (must be enabled):

• Users can now specify a primary research domain for their project (e.g. Biology, Computer Science, Humanities, etc.). This information appears in the DMP's JSON as a 'tag'.
• Users can now indicate and describe any ethical concerns that concern their research data. Clicking the checkbox will display a field to describe the concerns and a URL field to link to a report. This information appears in the DMP's JSON.

A new 'Research Outputs' tab that allows users to specify information about their intended research outputs (must be enabled) (thanks to @briri and @nicolasfranck for these contributions #2738 #2814 #3134)

Screenshot of the new Research Outputs tab:

The 'Research Outputs' tab allows users to:

• Specify the type (e.g. dataset, software, image, etc.), name and a description of the output, size (if applicable), whether or not it contains sensitive info or PII, what the initial access level will be (e.g. open, restricted, closed) and the anticipated publication date.
• Select an appropriate license (must be enabled) (note that the options come from the SPDX license registry) (#2607)
• Select any metadata standards that will be followed (e.g. Dublin Core) (must be enabled) (note that the list of standards comes from the RDA Metadata standards catalog) (#2798)
• Select the repositories they intend to preserve the object in (e.g. Dryad, Zenodo, GitHub, or an institutional repository) (must be enabled) (note that the list of repositories comes from the re3data registry) (#2605)

Screenshot of the Repository selection window (functionality for Metadata Standards is similar):

After a user enters information about their research outputs, the data will appear in the PDF as an addendum in a table format (#2812) (the Download page includes a checkbox to include/exclude this new addendum)

Screenshot of Research Outputs as an addendum to the PDF:

The research output information will also appear in the JSON output (and API v1 output) for the DMP:

Screenshot of new JSON Output (also delivered by API v1) showing a Research Output translated into the RDA Common Standard:

Organisational Administrators

Admins may now specify a helpdesk email address when editing their Organisation's details. This email will be used in the email signature for emails sent from the system (assuming the recipient of the email is a member of the same Organisation) (thanks to @johnpinto1 #3140)

Note that, if enabled, the new 'Research Outputs' tab may duplicate data entry for users if your templates are already asking for specific information about the intended research outputs (e.g. which repositories, what licenses, etc.). You can prevent this duplication of effort by editing your template and removing/rewording those questions. The new research output info appears in an addendum to the PDF of the DMP. It also appears as 'Dataset' information in the JSON output (see above screenshots). If you do change your template, do not forget to re-publish once you have made your changes!

Bug Fixes:

• Fixed an issue that was preventing Org Admins from seeing the 'Share' tab #3131
• Fixed an issue that was causing errors when Org Admins tried to save changes to a template #3038
• Patched an issue that continued to show users as collaborators on a Plan's Share tab even after they had removed themselves from the Plan #3126
• Fixed an issue that was causing TinyMCE editors to have an incorrect size when they initially load #3141
• Various fixes for the APIs
  • Patched issue that was always using the default template when creating a new DMP in API v1 #3137
  • Fixed an issue that was preventing plans from being created via API v0 #3135
  • Addressed an issue that was preventing users from being unassigned from a department in API v0 #3132

Developers

Enabling new functionality

By default, your guidance and comments section of the 'Write Plan' page will be expanded. If you want to change this so that it is collapsed by default, you can change the guidance_comments_opened_by_default setting in the config/initializers/_dmproadmap.rb file. If you want to disable this functionality entirely you can change the guidance_comments_toggleable setting.

The 'Research domain' dropdown and 'Ethical issues' fields on the Project Details page and the new 'Research outputs' tab are disabled by default. If you would like to use them, you must enable them (enable_research_domain, enable_ethical_issues and enable_research_outputs) in the config/initializers/_dmproadmap.rb file. Note that changes to this file require a restart of the application.
There are a few additional configuration options for components of the 'Research outputs' page:

• To allow your users to select metadata standards for a research output, you should:
  • enable the setting in the _dmproadmap.rb initializer - enable_metadata_standard_selection
  • Run bin/rails external_api:load_rdamsc_standards RAILS_ENV=[env] to populate the metadata_standards table
• To allow your users to select repositories for a research output, you should:
  • enable the setting in the _dmproadmap.rb initializer - enable_repository_selection
  • Run bin/rails external_api:load_re3data_repos RAILS_ENV=[env] to populate the repositories table
• To allow your users to select a license for their research outputs, you should:
  • enable the setting in the _dmproadmap.rb initializer - enable_license_selection
  • Run bin/rails external_api:load_spdx_licenses RAILS_ENV=[env] to populate the licenses table with data
  • Review the list of 'preferred' licenses in the _dmproadmap.rb initializer and modify if needed. Note that if you should use the %{latest} token we 'preferring' a license that have a version (see example in the config file).

We have added information to the Administration Wiki page about enabling/disabling functionality and our recommendations on which Rake tasks should be run on a schedule.

Additional changes:

• Updated API V1 to convey the new Research Output info as Datasets in the JSON output (see screenshot above)
• Added a new generic modal search dialog (currently used for repository and metadata standard selection). This component allows the page to define its own search/filter options and display search results in its own way. (Included a README.md in the app/views/layouts/modal_search dir to help describe how to implement for new data)
• Modification to the way the Plan is associated with its grant (basically a shortcut to get at the identifier record)
• Added new Services (and corresponding config files) to services/external_apis that fetch data for the License, Metadata Standard and Repository tables. These also have callable Rake tasks in lib/tasks/utils/external_api.rake to help you initialize the tables and update the contents on a regular basis
• Reverted change to our use of string format tokens that was made during an upgrade of Rubocop. The preferred format is once again Some %{variable_name} text #3128
• Synced latest translations from Translation.io
• Updated all Gem and JS dependencies

v3.0.6

• Fixes for downloading PDFs

v3.0.5

Bug Fixes:

• Fix for pagination of plans retreived using the REST API V0 #3105
• Security vulnerability patches #3100, #3097, #3093, #3083, #3077
• Fixed issues/inconsistencies with policies #3099, #3084
• Fix for issue downloading PDFs #3098, #3054, #3049
• Fix for issue displaying users as collaborators on a plan after they have removed themselves #3095
• Fix for API v0 not supplying the owner email address #3094
• Fix for bug preventing Super Admins from creating new Orgs #3091
• Fix for issue causing deleted plans to appear in the Org Admin list of plans #3087
• Fix for CSV downloads #3085, #3075
• Fix for pagination and search on the Admin plans page #3069, #3073
• Fix for strings that were not included in translation content #3081, #3050
• Fix for issue with contributor org logic not adhering to the restrict_orgs config flag #3078, #3060
• Fix for adding contributors #3071
• Removed phone number field from contributor page #3067
• Fixed issue with merging users #3065
• Fixed issue with emails #3052
• Fix for users unable to see API page #3047
• Upgraded Rubocop and updated all files accordingly #3045, #3048

We have also updated most Gem and JS dependencies

Upgrade instructions:

• Run bundle install
• Run yarn install

v3.0.4

Researchers

• Fixed an issue that was causing Guidance selections to display incorrectly on the Project Details page #2989
• Corrected an issue that was preventing Funder selection on the Project Details page #3029
• Addressed an issue that was causing the 'Search' functionality to fail on tables displaying plans #3000
• Added a red asterisk next to the 'Organization' label on the create account tab since it is a required field #2889
• Adds a "Funding status" selection box to the Project Details page #2920
• Adds a "Research domain" selection box to the Project Details page #2673 (if enabled in config/initializers/_dmproadmap.rb)
• Adds the "Ethical Issues" checkbox and corresponding "Description" and "Report URL" to the Project Details page #2432 (if enabled in config/initializers/_dmproadmap.rb)
• Added spellcheck to all of the text area / TinyMCE editors
• Fixed an issue when downloading a plan that has unanswered questions #2958

Admins

• Updates to API V1:
  • Use the plan.owner as the data contact if available (otherwise the contributor who is the :data_curator)
  • Include the new ethical_issues fields
  • Include the research_domain as a :keyword
  • include the plan.org as a :dmproadmap_funded_affilaition to deal with scenarios where the plan.org does not match the orgs defined for the owner or contributors
  • include the user entered plan.identifier as :dmproadmap_funding_opportunity_number

Developers

• Pulled in the latest translations from Translation.io
• Updated Gem and JS dependencies
• Added grant getter and setter overrides to the Plan model to allow the grant to be loaded and set properly from incoming UI params.
• Added new config settings to allow the Research Domain and Ethical Issues to be enabled/disabled in config/initializers/_dmproadmap.rb
• Fixed issues with ReCaptcha implementation #3011
• Addressed issue with HTML Downloads #3030
• Addressed an accessibility issue that was improperly attaching the wrong label to TinyMCE editors #3032
• Cleaned out old unused/legacy tables and columns. See #2818 for details (note that despite orgs.is_other and users.other_organization being listed on the ticket, they were not removed)
• Added a utils/conditionalFields.js file that can be used to hide/show form elements based on the state of a checkbox within a <conditional></conditional> block.
• Refactored JS on 'Plan Details' page to use generalized utils/conditionalFields.js
• Removed old unused feedback_confirmation mailer
• Removed unnecessary link between Plan and ApiClient
• Removes the mime_types table which we decided not to use #2816
• Add a Rake task rails external_apis:add_field_of_science_to_research_domains that populates the new research_domains table

Warning! This update will drop the old user_identifiers and org_identifiers tables. You should evaluate the data in these tables to ensure that there are no recent record (the transition to the consolidated identifiers table occurred several versions ago). It is advisable to backup your database prior to the upgrade.

If you wish to enable the new Research Domain selection box and/or Ethical Issues questions on the 'Project Details' page, you will need to run bin/rails external_apis:add_field_of_science_to_research_domains RAILS_ENV=[env] and set the enable_research_domain and/or enable_ethical_issues flags in the config/initializers/_dmproadmap.rb file (note that you will need to restart the Rails server after updating the config).

v3.0.3

Researchers:

• Your project start and end dates and grant information now appear on the coversheet of the PDF export #2972
• Fixed an issue with sharing plans #2874
• Fixed a bug that was preventing plans with a quote in the title from being exported #2883
• Fixed an issue with user's appearing on the collaborators list even after they have been removed #2954
• You may now specify 'other' as a role on the contributors page #2961

Admins:

• patched a bug with API V0 (full text API) #2892, #2965, #2966
• fixed an issue with downloading some of the CSV files on the usage dashboard #2866
• The sort, search and pagination options are now retained in the URL on the Public Plans and Funder Requirements pages so that you may copy, paste and share search results #2959

Developers:

Note that the new JS spinner was added to the app/views/layouts/application.html.erb. You have likely branded your own version of this file so be sure to copy the spinner div to your copy of this file.

• The system has been translated into the Turkish language (tr-TR). If you wish to offer this language in your installation, you will need to add it to your local DB. INSERT INTO languages (abbreviation, name, default_language) VALUES ('tr-TR', 'Türk', 0); (Set the last value to '1' if you want this to be the default language for your site)
• Added a new tab to the Super Admin's Org page that allows you to merge Orgs #2763
• Added a JS spinner that appears when the site performs an AJAX call #2960, #2976
• Added the ability to restrict_orgs in the configuration file so that the system will not query the ROR API. #2836, #2851, #2858, #2871
• Fixed a bug that was generating a 500 error if a user left the email address blank when submitting the 'add collaborator' form
• Fixed issues with the workflow for inviting users #2863, #2946

v3.0.2

This release contains a series of bug fixes introduced by the v3.0 upgrade to Rails 5

Developers

We have written up a high level description of the DB schema along with explanations of each table and field's use within the system: https://github.com/DMPRoadmap/roadmap/wiki/DB-Schema

• Addressed an issue that was allowing a user to select non-funder organizations on the Plan details page
• Fixed an issue that was preventing the JSON Web Tokens from being decoded properly
• Fixed an issue with the mysql tests GitHub action
• Fixed an issue with API V1 deserialization test that was improperly comparing dates #2774
• Fixed an issue that was causing the plans/ endpoint in API V1 to return a 500 error #2776
• Fixed an issue with one-click plan creation #2800
• Fixed an issue with table sorting/pagination #2785
• Fixed an issue that prevents branded content from being used when running Rails as a daemon #2779
• Addressed an issue that displayed the 'Create plan' icon on the public 'Funder Templates' page to users who are not logged in #2804
• Updates to optimize system performance by changing ActiveRecord method calls on associations to Array methods. For example: @plan.answers.where(foo: 'bar').order(:created_at) to @plan.answers.select { |a| a.foo == 'bar' }.sort { |a,b| a.created_at <=> b.created_at }
• Removed the old app/helpers/version_helper.rbwhich attempted to execute a git command. The version/commit information it collected was used on the app/views/layouts/application.html.erb file which is typically overridden. Having the code in place was problematic in situations where people use tools like Capistrano for application deployment and the deployed code is not a git repository.
• Updated the Perm model to use the Rails cache so that the system is not querying the DB multiple times
• Fixed an issue with the Metadata Standard question type that was preventing the answer from saving properly. #2826
• Updated all calls to administerable_by? and editable_by? on the Plan model to require the user's id instead of the object. For example: @plan.administerable_by?(current_user) to @plan.administerable_by?(current_user.id)
• Fixed a bug in the bin/setup file

Dependency updates

• Bump the wkhtmltopdf-binary gem from 0.12.6 to 0.12.6.5
• Bump the elliptic gem from 6.5.3 to 6.5.4

Thanks to @nicolasfranck and @yjarosz for their contributions to this release 👍🏻

v3.0.1

Researchers:

• Added new Download as JSON (RDA Common metadata standard format) feature #2438
• Removed the ROR link from the Contributors list on the Contributors tab (now just lists the Org name) #2705

Administrators:

• Added feature that lets admins generate or refresh their API token #2410
• Fixed bug with sorting template sections via the drag and drop feature #2735
• Fixed a bug that was preventing the plans.complete flag from being set which prevented the plan from appearing in the list of Organizational plans #2745
• Fixed several bugs on the edit Org details page:
  • You can now update and delete a shibboleth entityID #2288
  • You can now add/edit/remove links #2737
  • You can now edit the Org type #2760

Developers:

A bug in the Organization typeahead boxes on the create account and edit profile pages prevented the v3.0.0 code from calling out to the ROR API. This may have resulted in Org records created via manual user entry with names that do not match the official ROR record and that do not have ROR or Crossref funder ids.

We have added 2 new Rake tasks to assist with cleanup of any affected data.

• rails org_cleanup:rorify Will collect all of your Orgs that do not have a ROR id in the identifiers table. It will then call out to the ROR API and attempt to locate their identifiers. If found, it will add them to the database. Note that the names must match closely for this task to determine that it found a match in the ROR API.
• rails org_cleanup:find_dups Will review your Org records and attempt to find duplicates by comparing their names and abbreviations. It will generate a report upon completion that identifies possible duplicates. You will then need to either manually clean up these duplicates in the database or wait until the next release which will include a UI function that allows Super Admins to merge Orgs.

Bug fixes and administrative tasks

• Updated the README to clarify our level of support #2756
• Added DB and models to support future UI work to allow users to define multiple research outputs for their plans #2739
• Add api_clients.org_id as a nillable fkey on that table. Updated the v1 API to use this fkey when determining access to a plan #2725
• Updated the api_clients.last_access from a t.date to t.datetime in the DB
• Bug fix for Shibboleth login #2686
• Fixed a bug that was preventing Org typeahead boxes from searching the ROR API #2746
• Fixed a bug with the Google Analytics tracker #2732
• Improved performance of loading an Org's Plans by using Rails.cache #2726
• Fixed issue with ORCID and Shib auth from the Edit profile page opening in a new tab/window #2752
• Fixed an issue with using Socket.hostname which was causing issues in certain scenarios #2729
• Removed uglifier gem since this is now handled via Webpacker #2741
• Fixed a typo in action for deleting a theme and fixed rendering in shib actions on orgs_controller #2771
• Synced up the latest translations #2772

v3.0.0

Researchers

No major changes to report.

Administrators

• Introduction of API v1. This new API conforms to the new [common standard metadata format for DMPs](RDA Common Standard Metadata schema v1.0) that was defined an RDA working group. It currently supports: retrieving the list of your DMPs (and any public DMPs), retrieving the metadata for a single DMP, and creating DMPs. All of the JSON that this new API consumes and produces follows the metadata standard. You should continue using API v0 if you need access to a plan's full text or any other features that have not yet been ported over to v1.

---

Developers

• Fixed a bug that was preventing users from creating a plan by clicking the icon on the Public Plans page and from the organizational plans table on the My Dashaboard page - Issue 2580
• Soft launch of the new APIClients section (introduced in v2.2 but worth re-mentioning) that allows you to define systems that will integrate with your installation (as opposed to organizational admins). These new API clients can only API v1 and have very restricted access to DMPs. The functionality available for these clients will continue to grow over the coming years. Please refer to our development roadmap for details.

Upgrade to Rails 5.2

This upgrade brings the codebase from Rails 4.2.x to Rails 5.2.x. This was a major version upgrade and we advise you to review Rails' official upgrade guide to see what has changed. If you have customized the code in any way, you may need to make changes to work with this new version. We have highlighted some of the major changes below along with some changes we've made to consolidate the application's configuration.

The upgrade guides can be found here:

• Rails 4.2 to 5.0
• Rails 5.0 to 5.1
• Rails 5.1 to 5.2

• Dependencies - This release includes an upgrade to Rails 5.2, Ruby 2.6.3, Node 10.x+ and Yarn 1.22+
  • Upgraded most Gems and Yarn dependencies (see the Gemfile.lock and yarn.lock for details). The primary exceptions are TinyMCE and Bootstrap.
• Credentials - The application's sensitive credentials have moved from the various YAML files and initializers ito Rails' new encrypted credentials system.
  • Run EDITOR=vi rails credentials:edit (Note: you can use editors other than vi) and copy the content of the config/credentials.yml.example.
  • Update any custom code you have that references those old values (e.g. Rails.application.credentials.devise[:secret_key])
• Configuration - All custom configuration has been moved from config/application.rb, config/branding.yml and various config/initializers/*.rb into a single config/initializers/_dmproadmap.rb file. This was done to consolidate the applications configuration and make it easier for installations to manage their config.
  • You will need to evaluate your existing configuration and update this new file accordingly.
  • You will also need to evaluate any custom code you may have to use these new files. For example: Rails.configuration.branding[:organisation][:name] becomes Rails.configuration.x.organisation[:name]
• Strong Parameters - Rails 5.2 no longer passes request parameters around as a Hash. They have instead moved to ActionController::StrongParemeters. This is a breaking change so you will need to review any customizations you have made to controllers.
• Rake - You no longer run any 'rake' commands. You should instead use 'rails' (e.g. rails db:migrate RAILS_ENV=production)
• Warning -- check your customized code! If you have placed any customizations to the base DMPRoadmap code in the lib/ directory, you will need to move them into the app/ directory. Rails 5+ has disabled the autoload feature in the production environment. You will need to move your code into the app/ directory and update the module/class hierarchies accordingly (e.g. MyApp::Controllers::PublicPages which was in lib/my_app/controllers/public_pages.rb would become MyApp::PublicPages if moved to app/controllers/my_app/public_pages.rb)
• Enabled Rubocop for verification that our Ruby code follows the latest community standards
• The timeago JS strings are now translatable. The 'updated by [email protected] - 2 days ago' messages are now part of the translation ecosystem - issue 1974

Introduction of Translation.io

We're moving our translation backend to Translation.io. Previously, we were using a combination of FastGetText and I18n for translation. The translation gem simplifies the management of these two transation systems, and provides a web-based service to allow translators to collaborate. This approach allows us to retain the Gettext style translations (full strings in the code and the _("") markup for translation), and gives the flexibility to separately maintain translations for the open-source DMPRoadmap codebase, along with your customization to the codebase and to static-pages.

For more details on how the translations will be managed, along with instructions for translating your customization, see the wiki page on translations

Upgrade guide

Please review the following changes and back up your DB before upgrading!

• Install the latest versions of Ruby, Node and Yarn (See above for specific version info)
• Merge in the latest changes (resolving any conflicts it may have with your customizations)
• Run EDITOR=vi rails credentials:edit (swap 'vi' with your favorite editor) and copy the contents of config/credentials.yml.example into the editor. Replace all of the values with the ones appropriate for your installation. This will generate a config/master.key and config/credentials.yml.enc file. DO NOT commit your master.key into GitHub! See the Rails 5.2 docs for more info on the new approach to managing credentials.
• Run rails db:migrate
• Run rails v3:upgrade_3_0_0. This will run scripts that sets your orgs.language_id, users.language_id, orgs.feedback_email_subject and orgs.feedback_email_msg values to the default.
• Update your customizations to comply with Rails 5.2 (see notes above)
• Thoroughly test your application

v2.2.1

Minor patches to v2.2.0. Including some that address issues with the v2.2.0 upgrade tasks

• added missing DB indexes
• fixed bug with one-click plan creation
• fixed bug with missing contributor names in API v1
• fixed several bugs with the v2.2.0 upgrade scripts that transform data

If you have not already done so, please follow the upgrade notes in the developer section for release v2.2.0

v2.2.0

Researchers

• You can now provide Project Start and End Dates on the Project Details tab when editing your DMP. #2409
• You can now provide information on your Project's collaborators / contributors on the new 'Contributors' tab when editing your project. For example, you can identify more than one PI, project administrators and data curators. #2349 #2408
• The new 'Contributors' tab provides a typeahead box for organizational affiliation. This box searches the Research Organization Registry (ROR) as you type. #2339
• Fixed a bug that was sometimes confusing the funder and template names on PDF exports #2419

Organizational Admins

• You can now make some checkboxes, dropdown lists or radio buttons on your templates trigger conditional logic. For example if a user selects a particular value from a dropdown then logic can be triggered to send an email or hide other questions related to their response. #1772 #2476 #2405 #2451
• The Usage Statistics page now allows you to filter out test plans and the filters now apply to the same graphs instead of the static plan and user counts (which have been removed) #2158 #2343
• You can now click on one of the bars in the bar charts on the Usage Dashboard page which will drill into the results.
• You can now filter the results of the Plans and Users pages by month and year #2450
• You can now provide us with a Google Analytics code that you can then use to monitor your user's activity within the system. #2344
• Updated the API to allow you to update your User's departmental affiliations. See the wiki documentation for more details #2235
• Fixed a bug that prevented you from viewing historical versions of your templates #2509
• Fixed a bug that was giving new templates the incorrect visibility selection by default. This was preventing published templates from being used. #2518
• Fixed a bug that was prevent users from creating more than 5 templates #2458
• Fixed a bug that was preventing you from deleting questions from a published template #2304
• ORCID and Shibboleth ids now appear on the Users page #2353

Super Admins

• You can now deactivate a notification #2382
• You now have the ability to export all plans #2346
• Added filter by permission level to the Users page #2352

Developers
This release contains many structural DB changes and data transformations to help your system move towards the RDA Common Metadata Standard for DMPs. These changes will make future integrations with external systems easier. Please read the following instructions carefully before proceeding with your upgrade.

Please read through these notes BEFORE deploying this release!

If you are running on a machine that has 2 or fewer CPUs then you will need to modify the upgrade Rake tasks. See Issue 2723

• All of the Org selection boxes have been updated. The ones on the 'Create account', 'Edit profile', 'New Org', 'Funder selection Project Details' and the new 'Contributors' tab provide a typeahead that queries the list of Orgs in your DB as well as the [Research Organization Registry (ROR) API] (https://ror.org). The results prioritize results from your DB. It also allows the user to enter their own value for scenarios where an Organization does not appear in the ROR API. If the user selects (or enters their own value) in the box, a new Org record is created and the user is associated with the org.
• The old user_identifiers and org_identifiers tables have been consolidated into the new polymorphic identifiers table. The upgrade script will migrate your existing records into this new table. These legacy tables will be dropped in the next release.
• Two new Identifier Schemes have been added: Fundref and ROR. When new Orgs are created via the org selection box described above, the ROR and Fundref (if applicable) identifiers are added to the new identifiers table. An upgrade script is provided that will retrieve the ROR and Fundref identifiers for all of your existing Org records.
• Introduced a 'managed' flag to the Orgs table. This will eventually replace the old 'is_other' flag. A 'managed' org has administrators. New Orgs created in the future by a user's selection of a ROR Org (or entering free text) will be 'unmanaged' by default
• An upgrade script is supplied that will attempt to match any User's associated with the default is_other Org with a result from the ROR API (using the user's email domain and the org's name stored on the user record). If an appropriate match is found, the org record will be created and the user will be associated with the new org.
• Introduced the 1st half of the RDA metadata standard for DMP compliant API api/v1/. The API currently only allows for the following: templates#index, plans#index (no filters yet), plans#show and plans#create #2390 (Note that the 2nd half of the API work is scheduled for late summer / early fall .. the draft of the new spec can be found here)
• Added a new 'Api Clients' section to allow you register clients for the new v1 API. The v1 API uses JSON Web Tokens for security and allows either an Api Client or User to authenticate. Api Clients are meant for communication with external systems. See the API spec linked above for further details

Updates to the Gemfile dependencies

• We added Httparty to handle communications with external APIs
• We added JWT to handle auth for the new v1 API
• We added Capybara-Webmock to ensure that calls are not made to external apis when running tests.

Most gems were upgraded to their latest patch release. The following gems had minor or major version updates:

Dependency	from	to
Annotate gem	3.0	3.1
Better Errors	2.5	2.7
Brakeman	4.7	4.8
Bullet	6.0	6.1
Bundler Audit	0.6	0.7
Byebug	11.0	11.1
Capybara	3.29	3.32
Database Cleaner	1.7	1.8
Excon	0.71	0.75
FactoryBot & FactoryBot Rails	5.1	5.2
Faraday	0.17	1.0
FFI	1.11	1.13
FOG AWS	3.5	3.6
FOG Core	2.1	2.2
Hashie	3.6	4.1
Launchy	2.4	2.5
Libv8	6.9	7.3
Loofah	2.5	2.6
Lumberjack	1.0	1.2
Method Source	0.9	1.0
Parser	2.6	2.7
Pry	0.12	0.13
MiniProfiler	1.1	2.0
Recaptcha	5.2	5.5
Regex Parser	1.6	1.7
Rollbar	2.22	2.25
Rspec	3.9	4.0
Rubocop	0.77	0.85
Sassc	2.2	2.4
Shoulda	3.6	4.0
Simplecov	0.17	0.18
Thor	0.20	1.0
Unicode Display Width	1.6	1.7
Webmock	3.7	3.8
----------	-----	-----

Updates to the Yarn JS dependencies

• Added bootstrap-select to support conditional questions feature
• Most JS files were upgraded to their latest minor or patch release.

Database Changes

• New table conditions to support conditional questions
• New table api_clients
• New contributors table
• New table identifiers (replaces the user_identifiers and org_identifiers)
• New table trackers to store org Google Analytics codes
• Add a orgs.managed flag
• Added plans.funder_id and plans.org_id (replaces old funder string) relationships
• Added plans.start_date and plans.end_date
• Added plans.api_client_id to identify the provenance of plans created via api v1
• Added a notifications.enabled flag
• Added a users.last_api_access field
• Added a identifier_schemes.context bit flag
• Added question_options.versionable_id
• Added filtered to the stats tables

Upgrade process
This upgrade involves changes to the database structure (see above). It also requires scripts to be run that will transform your existing data to fit into these new structures. We recommend that you run through the upgrade process in a development or staging environment prior to production. Any cleanup that may need to be done can then be scripted to help speed up the upgrade process in production.

1. Back up your database!

2. Update your branding.yml file as follows:

• Add use_recaptcha: false to the application section. Set the value to 'true' to add a reCAPTCHA to the create account form.
• Review the config/initializers/external_apis/ files. The DOI one is a placeholder, this functionality will appear in a future release. You can set the open_aire.rb file's active attribute to enable/disable the API typeahead for the 'Grant ID' field on the Project Details page. You can set the ror.rb file's active attribute to enable/disable calls to the ROR API (this will force the new Org selection boxes to only look at your local database ... users may still enter free text into the field to create a new Org record)

3. Run bundle exec rake db:migrate RAILS_ENV=[env] to make the necessary changes to the DB.

4. We highly recommend that you run the following scripts in the latest version of v2.2.x! For example, v2.2.1 includes several bug fixes to the v2_0_0_part2 script.

5. Run the 1st data transformation script that will populate the new question_options.versionable_id field. This step can take over 5 minutes to run depending on the size of your table: bundle exec rake upgrade:v2_1_6 RAILS_ENV=[env]

6. Run the upgrade:v2_2_0_part1 upgrade script that will add 'fundref' and 'ror' to the 'identifier_schemes' table, populate the 'context' field on the 'identifier_schemes' table, migrate your 'user_identifiers' and 'org_identifiers' into the new 'identifiers' table and search the ROR api and add th...