feat(9586): implement freetext search in cht datasource

[sugat009]

Description

Closes: #9586

Code review checklist

• Readable: Concise, well named, follows the style guide, documented if necessary.
• Documented: Configuration and user documentation on cht-docs
• Tested: Unit and/or e2e where appropriate
• Internationalised: All user facing text
• Backwards compatible: Works with existing data and configuration or includes a migration. Any breaking changes documented in the release notes.

Compose URLs

If Build CI hasn't passed, these may 404:

• Core
• CouchDB Single
• CouchDB Cluster

License

The software is provided under AGPL-3.0. Contributions to this project are accepted under the same license.

[sugat009]

@jkuester I did not go with the hook implementation as it would mean checking for filenames to run because this auth setup needs to run only for a few test suites, which would have resulted in a not-so-fruitful hook file with checks.

[jkuester]

Alright! We are getting very close here.

[jkuester]

nitpick: I guess now the getPrimaryContactIds, hydratePrimaryContact, hydrateLineage functions do not need to be exported since they are only getting used in this file.

[jkuester]

Lets 👏 freaking 👏 go 👏 !!!!! 🚀 🚀

Thanks @sugat009 for hanging in with me on this absolute tour-de-force! 💪

[jkuester]

note: I am fine going with whatever way you think is most maintainable, but I originally was thinking we could re-use the existing fetchAndFilter function instead of duplicating most of that logic again here... 😬

export const fetchAndFilterUuids = (
  getFunction: (limit: number, skip: number) => Promise<string[]>,
  limit: number,
): ReturnType<typeof fetchAndFilter<string>> => {
  const uuidSet = new Set<string>();
  const filterFn = (uuid: Nullable<string>): boolean => {
    if (!uuid) {
      return false;
    }
    const { size } = uuidSet;
    uuidSet.add(uuid);
    return uuidSet.size !== size;
  };

  return fetchAndFilter(
    getFunction,
    filterFn,
    limit
  );
};

I see you have a + 1 fetch in your fetchAndFilterUuids which is a bit different behavior (but maybe that should be built into the base fetchAndFilter? Otherwise, I do not see much performance difference between these right?

[sugat009]

Yes, there is no performance difference between these two. I implemented this function separately because when I started working on this I saw a situation, not a bug I guess, where if you query with limit 10. There are 10 objects in the database, the cursor is also returned as 10, which means the end-user might re-query again, and then in the next request the user gets cursor as null, resulting in 1 extra query. The +1 way is a solution for that.

I did not implement the +1 way directly into the already-existing fetchAndFilter functionality because it might affect the places where it is used but now that I think about fetchAndFilter VS fetchAndFilterUuids, they have a bit of an inconsistent cursor returns. I think implementing fetchAndFilterUuids the above way and re-doing fetchAndFilter with the +1 functionality could be done. However, I think we can move on with the above suggestion and keep the +1 way as a future enhancement as it's not a breaking issue. What do you think?

[jkuester]

I think we can move on with the above suggestion and keep the +1 way as a future enhancement as it's not a breaking issue. What do you think?

Sounds good! 👍

[mrjones-plip]

Lets 👏 freaking 👏 go 👏 !!!!! 🚀 🚀

YYYYYYYYEEEESSS!!! you two are amazing - so great to see this work come together