[redcap] enhancements

[regisoc]

Description

This a short todo list of features or enhancement that can benefit the REDCap module.
More will be added later.

• extract authorize instrument list from config module (medium task/difficulty):
  • extract the single list from Configuration module elsewhere (to define)
  • make it per REDCap instance/project (can be different).
• special field names (_date, _status) handling (medium+).
• make _timestamp field mandatory (source) (easy).
• ...

[MaximeBICMTL]

I am moving my comment from https://github.com/aces/CBIGR/issues/745 to here:

Hhhhm, I must say I have never used the REDCap module with different projects configured that use it, and since the module was originally single-project on HBCD, I guess it is to be expected that there might be bugs with several projects.

Anyway, the redcap_importable_instrument config variable is clearly from a time before the multi-project feature was added. I think the correct fix is to move the variable from the generic database configuration to the per-project redcap configuration so that each project can have its own list of instruments, like the following:

<redcap>
  <instance>
    <redcap-url>https://www.example.net/redcap/</redcap-url>
    <project>
      <redcap-project-id>123</redcap-project-id>
      <redcap-api-token>ABCDEFGGHIJKLMNOPQRSTUVWXYZ</redcap-api-token>
      <prefix-instrument-variable>false</prefix-instrument-variable>
      <redcap-participant-id>record_id</redcap-participant-id>
      <candidate-id>pscid</candidate-id>
      <instruments>
        <instrument>instrument_1</instrument>
        <instrument>instrument_2</instrument>
        <instrument>instrument_3</instrument>
      </instruments>
      <visit>
        <visit-label>visit_1</visit-label>
        <redcap-arm-name>arm_1</redcap-arm-name>
        <redcap-event-name>event_1</redcap-event-name>
      </visit>
    </project>
  </instance>
</redcap>

Here I placed the <instruments> node in the <project> node directly, but those could also be placed in the <visit> node I guess. Ideally we would want to have some sort of cascading configuration for the REDCap module, but that is too far-stretched IMO. I think the default if the <instruments> node is not present should be to import all instruments (unlike now, where none is imported), which would be consistent with the current approach that imports all visits when they are not specified, and would make default configuration easier for new projects.

[regisoc]

When implementing this new feature, I made a mix of HBCD/IBIS to test data import.
What we are seeing now, for each new project using redcap module is that there is (yet) another way of handling data in REDCap.
Due to the REDCap event/arm/... flexibility, I bet we will discover new ways of representing data later on, and so new "bugs" which will be basically [yet another possible case/mapping we did/could not have anticipated].
Regarding authorized/importable instrument list in configuration, yes and each list should be per REDCap project.
Agreed for the case where we want to import everything, not having an <instruments> tag is lighter and easier to understand.
The main endpoint is already an open public endpoint, so by default it would allow to import what LORIS receives.
I am a bit on the fence for this: I like the explicit and centralized location, but I am just not a fan of having possible really long lists of instruments in this configuration. But at least, that is better than what we have right now.

[adamdaudrich]

Can we place instruments under the visit node?

[regisoc]

Is there a reason for that? Else, I would place it outside of it (i.e. what Maxime posted above), as the <visit> node is optional and the list of instruments per visit is defined in the battery. Meaning, this list should be authorizing insturments: per project, for all visits.

[adamdaudrich]

That is correct, my bad.
Will the instruments listed in the semi-structured form (xml) be inserted into config? How will this work for a multi-project instance like CBIGR? An enum in config per-project?

[regisoc]

Yes, it is detailed in the configuration file, you can repeat <instance> and <project> multiple times inside the root <redcap> tag.

[adamdaudrich]

I meant the instruments in the config table. How do we handle it there.

[MaximeBICMTL]

@adamdaudrich You don't add anything to the config table, you just remove the redcap_importable_instrument variable. Regarding how to parse/store the REDCap per-project configuration, the code is located in the modules/redcap/php/config directory. Just add parsing here and change uses of the redcap_importable_instrument variable to use the REDCap project configuration instead.

[adamdaudrich]

Ah ok brilliant