Installing Dependencies

This guide serves as an introduction to several key entities that can be managed with Apache Polaris (Incubating), describes how to build and deploy Polaris locally, and finally includes examples of how to use Polaris with Apache Spark™.

Prerequisites

This guide covers building Polaris, deploying it locally or via Docker, and interacting with it using the command-line interface and Apache Spark. Before proceeding with Polaris, be sure to satisfy the relevant prerequisites listed here.

Git

To get the latest Polaris code, you’ll need to clone the repository using git. You can install git using homebrew on MacOS:

brew install git

Please follow instructions from the Git Documentation for instructions on installing Git on other platforms.

Then, use git to clone the Polaris repo:

cd ~
git clone https://github.com/apache/polaris.git

Docker

It is recommended to deploy Polaris inside Docker for the Quickstart workflow. Instructions for deploying the Quickstart workflow on the supported Cloud Providers (AWS, Azure, GCP) will be provided only with Docker. However, non-Docker deployment instructions for local deployments can also be followed on Cloud Providers.

Instructions to install Docker can be found on the Docker website. Ensure that Docker and the Docker Compose plugin are both installed.

Docker on MacOS

Docker can be installed using homebrew:

brew install --cask docker

There could be a Docker permission issues related to seccomp configuration. To resolve these issues, set the seccomp profile to “unconfined” when running a container. For example:

docker run --security-opt seccomp=unconfined apache/polaris:latest

Note: Setting the seccomp profile to “unconfined” disables the default system call filtering, which may pose security risks. Use this configuration with caution, especially in production environments.

Docker on Amazon Linux

Docker can be installed using a modification to the CentOS instructions. For example:

sudo dnf update -y
# Remove old version
sudo dnf remove -y docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-engine
# Install dnf plugin
sudo dnf -y install dnf-plugins-core
# Add CentOS repository
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
# Adjust release server version in the path as it will not match with Amazon Linux 2023
sudo sed -i 's/$releasever/9/g' /etc/yum.repos.d/docker-ce.repo
# Install as usual
sudo dnf -y install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Confirm Docker Installation

Once installed, make sure that both Docker and the Docker Compose plugin are installed:

docker version
docker compose version

Also make sure Docker is running and is able to run a sample Docker container:

docker run hello-world

Java

If you plan to build Polaris from source yourself or using this tutorial’s instructions on a Cloud Provider, you will need to satisfy a few prerequisites first.

Polaris is built using gradle and is compatible with Java 21. We recommend the use of jenv to manage multiple Java versions. For example, to install Java 21 via homebrew and configure it with jenv:

cd ~/polaris
brew install openjdk@21 jenv
jenv add $(brew --prefix openjdk@21)
jenv local 21

Ensure that java --version and javac both return non-zero responses.

jq

Most Polaris Quickstart scripts require jq. Follow the instructions from the jq website to download this tool.