Search Relevance Test Bed

Note

This repository is being gradually transitioned to a new approach to measure relevancy and so some instructions may be out of date. The original readme has been kept for now to aid the transition.

A comprehensive tool for testing and comparing search algorithm relevance across different configurations and datasets.

Quick Start

# 1. Run Docker (we prefer colima) 
colima start

# 2. Run tool
make compare

Development

Running Tests

# All tests
make test

# With coverage
make test-coverage

# With race detection
make test-race

Code Quality

# Format code
make fmt

# Run linter
make lint

# Security audit
make audit

# All checks
make check

Tooling

We use some tooling for development that you will need to install.

Linting

For running lint checks against the JSON files you will need to run Node > v20 (or use the version in the .nvmrc file) and have prettier installed:

npm install -g prettier

For running lint checks against Go template files for JSON - we are currently experimenting with our own linting tool, dis-json-template-linter.

Integration testing

To ensure we're producing valid ElasticSearch queries, we run them against a docker run instance of ElasticSearch using testcontainers.

To get setup, follow our guidance on using testcontainers

If you're already setup, you will just need to ensure a docker daemon is running, for example via colima start.

To skip these tests you can pass the -short flag to the go test command, e.g.

go test ./... -short