Skip to content

feat: PostgreSQL-backed search/document storage as ElasticSearch alternative#700

Open
pandor4u wants to merge 1 commit intomoqui:masterfrom
pandor4u:feature/postgres-search-backend
Open

feat: PostgreSQL-backed search/document storage as ElasticSearch alternative#700
pandor4u wants to merge 1 commit intomoqui:masterfrom
pandor4u:feature/postgres-search-backend

Conversation

@pandor4u
Copy link

PostgreSQL-Backed Search & Document Storage

This PR adds a new postgres ElasticClient type that implements the full ElasticFacade.ElasticClient interface using PostgreSQL — eliminating the need for a separate ElasticSearch/OpenSearch cluster.

Motivation

Many Moqui deployments already run PostgreSQL. Requiring a separate ElasticSearch/OpenSearch cluster adds operational complexity, memory overhead, and cost — especially for small-to-medium deployments. This provides a zero-dependency alternative that uses PostgreSQL's native JSONB, tsvector full-text search, and GIN indexes.

Changes

New Files

  • PostgresElasticClient.groovy — Full ElasticClient implementation: index/get/update/delete/bulk/search/count operations backed by PostgreSQL tables
  • ElasticQueryTranslator.groovy — Translates ElasticSearch Query DSL (bool, term, terms, range, nested, exists, match_all, query_string, ids) into parameterized PostgreSQL SQL
  • PostgresSearchLogger.groovy — Log4j2 appender that writes structured logs to PostgreSQL instead of ES
  • SearchEntities.xml — Moqui entity definitions for moqui_search_index, moqui_document, moqui_logs, moqui_http_log with JSONB columns, tsvector, and GIN/BRIN indexes
  • moqui-postgres-only-compose.yml — Docker Compose for postgres-only deployment

Modified Files

  • ElasticFacadeImpl.groovy — Added type="postgres" client instantiation path
  • ElasticRequestLogFilter.groovy — Updated to work with postgres client
  • MoquiDefaultConf.xml — Default configuration for postgres search
  • moqui-conf-3.xsd — Schema update for type attribute on elastic-client
  • build.gradle — PostgreSQL JDBC driver dependency
  • MoquiSuite.groovy — Test suite registration

Tests (83 tests, all passing)

  • PostgresSearchTranslatorTests.groovy — 46 unit tests for query translation including 13 SQL injection prevention tests
  • PostgresElasticClientTests.groovy — 37 integration tests for CRUD, bulk, search, count, delete-by-query
  • PostgresSearchSuite.groovy — JUnit test suite

Security

  • All field names validated against ^[a-zA-Z0-9_@][a-zA-Z0-9_.\-]*$ regex before SQL interpolation
  • Double-dash (--) SQL comments explicitly blocked
  • LIMIT/OFFSET use JDBC parameterized queries (not string interpolation)
  • Docker compose uses environment variable references for credentials
  • TLS 1.2 minimum enforced in Docker nginx proxy
  • 13 dedicated security tests verify SQL injection rejection

Configuration

<elastic-client name="default" type="postgres" cluster-name="my-app"/>

Set type="postgres" on any elastic-client element in your Moqui XML configuration. The client will use the transactional entity group's database connection.

Testing

All 166 Postgres-related tests pass (83 per suite × 2 suite runs). No regressions in existing framework tests.

…rnative

Add a new 'postgres' ElasticClient type that implements the full
ElasticFacade.ElasticClient interface using PostgreSQL with JSONB
document storage, tsvector full-text search, and GIN indexes.

Key changes:
- PostgresElasticClient: full ElasticClient implementation backed by
  PostgreSQL tables (moqui_search_index, moqui_logs, moqui_http_log)
- ElasticQueryTranslator: translates ES Query DSL (bool, term, terms,
  range, nested, exists, match_all, query_string, ids) into
  parameterized PostgreSQL SQL with sanitized field names
- PostgresSearchLogger: Log4j2 appender writing to PostgreSQL
- SearchEntities.xml: entity definitions with JSONB, tsvector, GIN indexes
- Security hardening: field name sanitization, parameterized queries,
  env-var credentials in Docker, TLS 1.2 minimum
- Comprehensive test suite: 83 tests covering query translation, CRUD,
  bulk indexing, search, and SQL injection prevention

Configuration: set elastic-client type="postgres" in Moqui XML conf.
No external search engine dependency required.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant