Apache Polaris (incubating)

Apache Polaris™ is an open-source, fully-featured catalog for Apache Iceberg™. It implements Iceberg's REST API, enabling seamless multi-engine interoperability across a wide range of platforms, including Apache Doris™, Apache Flink®, Apache Spark™, StarRocks, and Trino.

Documentation is available at https://polaris.apache.org, including Polaris management API doc and Apache Iceberg REST API doc.

Subscribe to the dev mailing list to join discussions via email or browse the archives. Check out the CONTRIBUTING guide for contribution guidelines.

Building and Running

Apache Polaris is organized into the following modules:

• polaris-core - The main Polaris entity definitions and core business logic
• API modules (implementing the Iceberg REST API and Polaris management API):
  • polaris-api-management-model - The Polaris management model
  • polaris-api-management-service - The Polaris management service
  • polaris-api-iceberg-service - The Iceberg REST service
• Service modules:
  • polaris-service-common - The main components of the Polaris server
• Quarkus runtime modules:
  • polaris-quarkus-service - The Quarkus-specific components of the Polaris server
  • polaris-quarkus-defaults - The Quarkus-specific configuration defaults
  • polaris-quarkus-server - The Polaris server runtime
  • polaris-quarkus-admin-tool - The Polaris admin & maintenance tool
• Persistence modules:
  • polaris-jpa-model - The JPA entity definitions
  • polaris-eclipselink - The Eclipselink implementation of the MetaStoreManager interface

Apache Polaris is built using Gradle with Java 21+ and Docker 27+.

• ./gradlew build - To build and run tests. Make sure Docker is running, as the integration tests depend on it.

• ./gradlew assemble - To skip tests.

• ./gradlew test - To run unit tests and integration tests.

• ./gradlew polarisServerRun - To run the Polaris server locally, with profile prod; the server is reachable at localhost:8181.

• java -Dquarkus.profile=test -jar quarkus/server/build/quarkus-app/quarkus-run.jar - To run the Polaris server locally, with profile test. With this profile, Polaris uses the test Authenticator and test TokenBroker; this configuration is suitable for running regressions tests, or for connecting with Spark.

• ./regtests/run_spark_sql.sh - To connect from Spark SQL. Here are some example commands to run in the Spark SQL shell:

create database db1;
show databases;
create table db1.table1 (id int, name string);
insert into db1.table1 values (1, 'a');
select * from db1.table1;

More build and run options

Running in Docker

• ./gradlew clean :polaris-quarkus-server:assemble -Dquarkus.container-image.build=true - To build the image locally.
• docker run -p 8181:8181 -p 8182:8182 apache/polaris:latest - To run the image.

Running in Kubernetes

• ./run.sh - To run Polaris as a mini-deployment locally. This will create a Kind cluster, then deploy one pod and one service. The service is available on ports 8181 and 8182.
• kubectl port-forward svc/polaris-service -n polaris 8181:8181 8182:8182 - To create secure connections between a local machine and a pod within the cluster for both service and health/metrics endpoints:
  • http://localhost:8182/q/metrics
  • http://localhost:8182/q/health
• kubectl get pods -n polaris - To check the status of the pods.
• kubectl get deployment -n polaris - To check the status of the deployment.
• kubectl describe deployment polaris-deployment -n polaris - To troubleshoot if things aren't working as expected.

Running regression tests

Regression tests can be run in a local environment or in a Docker environment.

To run regression tests locally, you need to have a Polaris server running locally, with the test Authenticator enabled. You can do this by running Polaris as below:

java -Dquarkus.profile=test -jar quarkus/server/build/quarkus-app/quarkus-run.jar

Then, you can run the regression tests using the following command:

env POLARIS_HOST=localhost ./regtests/run.sh

To run regression tests in a Docker environment, you can use the following command:

docker compose -f regtests/docker-compose.yml up --build --exit-code-from regtest

Building docs

• Docs are generated using Hugo using the Docsy theme.
• To view the site locally, run

  site/bin/run-hugo-in-docker.sh

• See README in site/ for more information.

License

Apache Polaris is under the Apache License Version 2.0. See the LICENSE.

ASF Incubator disclaimer

Apache Polaris™ is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.

_{Apache®, Apache Polaris™, Apache Iceberg™, Apache Spark™ are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.}