How can I install DataHub on an AWS EC2 instance?

I want to install DataHub on an AWS instance. I specifically don't want to run it on a kubernetes cluster. Just a local installation is fine.

The below installation guide has been tested last on 06.05.2023.
Tests were executed on an AWS EC2 instance running Ubuntu with 2 CPUs, 8 GB of RAM and 30GB of storage.

The below code will guide you through the process of installing and properly configuring DataHub.

If working with Docker is or will become part of your job please read this guide from start to end.

A Docker Tutorial for Beginners (docker-curriculum.com)

In addition to general knowledge about Docker it will teach you about Docker Compose and volumes.
This knowledge is needed to understand what happens during the set up process.

Install Docker:

Install necessary software

• sudo apt-get install curl --> Enables file transfer between
  servers.
• sudo apt-get install gnupg --> For encrypting communication
  between server.
• sudo apt-get install ca-certificates --> For checking
  trustworthyness of servers.
• sudo apt-get install lsb-release --> To
  get info about linux distribution.

Create folder for docker GPG (encryption) file and download it.

• sudo mkdir -p /etc/apt/keyrings
• curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

Reference the latest stable docker release

• echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Update packages on the instance

• sudo apt-get update

Install Docker

• sudo apt-get install docker-ce
• sudo apt-get install docker-ce-cli
• sudo apt-get install containerd.io
• sudo apt-get install docker-compose-plugin

Check whether docker was installed successfully

• sudo docker --version --> Will show version of Docker.
• docker compose version --> Will show version of Docker compose.
• sudo docker run hello-world --> Will run a container and therefor
  proof functionality.

Enable Docker to run without requiring root user privileges

Create a docker permission group

• sudo groupadd docker

Add your user to the docker group

• sudo usermod -aG docker $USER

Update groups on your system

• newgrp docker

Check whether docker runs without sudo

• docker run hello-world

Install DataHub

Update the package list before installing pip

• sudo apt-get update

Install pip

• sudo apt-get install python3-pip

check for success

• pip3 --version

Install Wheel and setuptools

• python3 -m pip install --upgrade pip wheel setuptools

Install datahub

• python3 -m pip install --upgrade acryl-datahub

Add datahub to the path of linux

• export PATH="/home/ubuntu/.local/bin:$PATH"

Check whether datahub was installed

• datahub version

Restart the instance

Change frontend root user credentials

Navigate to home

• cd

Create datahub folder and open it

• mkdir datahub
• cd datahub

Create volumes folder and open it

• mkdir volumes
• cd volumes

Create frontend-react folder and open it

• mkdir frontend-react
• cd frontend-react

Create user.props file

• > user.props

Edit user.props

• nano user.props

Copy new user credentials

• // new user.props 
  datahub:NewPassword
  

Pres CTRL+X then Y and then Enter

Check contents of new file

• cat user.props

Run datahub quickstart once to download the .yml file.

• datahub docker quickstart

After the first container is healthy, stop the process with STRG+C. If your terminal gets stuck or is showing running text without end, restart it.

Stop all containers, remove them and also delete all volumes

• docker stop $(docker ps -a -q)
• docker rm $(docker ps -a -q)
• docker volume rm $(docker volume ls -q)

Navigate to the docker compose YAML file

• cd
• cd .datahub
• cd quickstart

Edit the docker-compose.yml file

• nano docker-compose.yml

Add the below line in the volume section of the frontend react container

• - ${HOME}/datahub/volumes/frontend-react/user.props:/datahub-frontend/conf/user.props
• 

Press CTRL+X then Y and then Enter

Execute docker compose up (If the terminal gets stuck afte a container was unhealthy or ran into an error execute the same command again until all containers are up)

• docker compose -f docker-compose.yml up -d

When running the docker ps command you should see the follwing now: