feat: PostgreSQL-backed search/document storage as ElasticSearch alternative#700
Open
pandor4u wants to merge 1 commit intomoqui:masterfrom
Open
feat: PostgreSQL-backed search/document storage as ElasticSearch alternative#700pandor4u wants to merge 1 commit intomoqui:masterfrom
pandor4u wants to merge 1 commit intomoqui:masterfrom
Conversation
…rnative Add a new 'postgres' ElasticClient type that implements the full ElasticFacade.ElasticClient interface using PostgreSQL with JSONB document storage, tsvector full-text search, and GIN indexes. Key changes: - PostgresElasticClient: full ElasticClient implementation backed by PostgreSQL tables (moqui_search_index, moqui_logs, moqui_http_log) - ElasticQueryTranslator: translates ES Query DSL (bool, term, terms, range, nested, exists, match_all, query_string, ids) into parameterized PostgreSQL SQL with sanitized field names - PostgresSearchLogger: Log4j2 appender writing to PostgreSQL - SearchEntities.xml: entity definitions with JSONB, tsvector, GIN indexes - Security hardening: field name sanitization, parameterized queries, env-var credentials in Docker, TLS 1.2 minimum - Comprehensive test suite: 83 tests covering query translation, CRUD, bulk indexing, search, and SQL injection prevention Configuration: set elastic-client type="postgres" in Moqui XML conf. No external search engine dependency required.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PostgreSQL-Backed Search & Document Storage
This PR adds a new
postgresElasticClient type that implements the fullElasticFacade.ElasticClientinterface using PostgreSQL — eliminating the need for a separate ElasticSearch/OpenSearch cluster.Motivation
Many Moqui deployments already run PostgreSQL. Requiring a separate ElasticSearch/OpenSearch cluster adds operational complexity, memory overhead, and cost — especially for small-to-medium deployments. This provides a zero-dependency alternative that uses PostgreSQL's native JSONB, tsvector full-text search, and GIN indexes.
Changes
New Files
PostgresElasticClient.groovy— FullElasticClientimplementation: index/get/update/delete/bulk/search/count operations backed by PostgreSQL tablesElasticQueryTranslator.groovy— Translates ElasticSearch Query DSL (bool,term,terms,range,nested,exists,match_all,query_string,ids) into parameterized PostgreSQL SQLPostgresSearchLogger.groovy— Log4j2 appender that writes structured logs to PostgreSQL instead of ESSearchEntities.xml— Moqui entity definitions formoqui_search_index,moqui_document,moqui_logs,moqui_http_logwith JSONB columns, tsvector, and GIN/BRIN indexesmoqui-postgres-only-compose.yml— Docker Compose for postgres-only deploymentModified Files
ElasticFacadeImpl.groovy— Addedtype="postgres"client instantiation pathElasticRequestLogFilter.groovy— Updated to work with postgres clientMoquiDefaultConf.xml— Default configuration for postgres searchmoqui-conf-3.xsd— Schema update fortypeattribute onelastic-clientbuild.gradle— PostgreSQL JDBC driver dependencyMoquiSuite.groovy— Test suite registrationTests (83 tests, all passing)
PostgresSearchTranslatorTests.groovy— 46 unit tests for query translation including 13 SQL injection prevention testsPostgresElasticClientTests.groovy— 37 integration tests for CRUD, bulk, search, count, delete-by-queryPostgresSearchSuite.groovy— JUnit test suiteSecurity
^[a-zA-Z0-9_@][a-zA-Z0-9_.\-]*$regex before SQL interpolation--) SQL comments explicitly blockedConfiguration
Set
type="postgres"on anyelastic-clientelement in your Moqui XML configuration. The client will use thetransactionalentity group's database connection.Testing
All 166 Postgres-related tests pass (83 per suite × 2 suite runs). No regressions in existing framework tests.