Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement functional/performance test for the Query Agent #203

Open
andre-senna opened this issue Jan 24, 2025 · 0 comments
Open

Implement functional/performance test for the Query Agent #203

andre-senna opened this issue Jan 24, 2025 · 0 comments
Assignees
Milestone

Comments

@andre-senna
Copy link
Contributor

We need an integrated test (Query agent integrated with AtomDB) running on a large KB to serve as functional and performance test.

  • Ideally the test should use 3 containers, one for AtomDB, one for Query server and one for client (the test code itself). This is desired but is not a hard requirement. Having everything running in the same container may work just as well.
  • To create the KB, we can re-use the code in https://github.com/singnet/das-nunet-rejuve-poc/tree/bb23a5453ef671b6ca7a451c240efc04c5648a91/presentation
  • Tests are expected to run (by hand, command line) in one of our large RAM bare metal servers. No automation is expected at this stage.
  • Performance test should cover different cases with different features. At least:
    • Queries with large responses for LinkTemplates
    • Queries with deep/complex AND, OR operations
    • KB of different sizes. At least small (few thousands sentences), medium and large (after loaded, near 80% of RAM is used)
  • Functional test can be implemented as a regression test on top of the results of (some of) the performance tests. We can run the selected tests once and record the query outputs, then check the result manually once and use the correct output to create baseline files. Afterwards, the actual functional tests should run the code and compare query outputs against the baseline.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants