Ubuntu/Debian · Fedora/RHEL · optional NVIDIA GPU

Install Jozie AI on Linux

Linux is the most direct path of the three — no WSL2 layer, no Docker Desktop GUI, just Docker Engine talking straight to the kernel and (optionally) straight to your GPU driver. This covers a fresh server or workstation end to end.

~20–30 min (CPU) · +10 min (GPU) Ubuntu 22.04+ / Debian 12+ / Fedora / RHEL sudo access NVIDIA GPU optional

Overview

Everything runs in Docker: backend, worker, frontend, PostgreSQL+pgvector, and Redis — the exact same images and docker-compose.yml already proven on Mac and Windows. Only two things stay native, for the same reason on every OS: Ollama (needs direct GPU access) and, if you have an NVIDIA GPU, the driver itself.

Fine-tuning (Module 8) works here too if you have an NVIDIA GPU — steps 3–4 cover that; skip them if you're CPU-only and Fine-Tuning just won't be available, same honest behavior as any other unsupported-hardware machine.

Requirements

A recent distro

Ubuntu 22.04+, Debian 12+, Fedora, or RHEL/CentOS/Rocky. Commands below cover both apt and dnf families.

sudo / root access

Needed to install Docker and (if applicable) the NVIDIA container toolkit.

~20 GB free disk

Docker images plus any base models downloaded for fine-tuning.

NVIDIA GPU (optional)

Only needed for Module 8 (Fine-Tuning). A reasonably current proprietary NVIDIA driver, not the open-source nouveau driver.

Install Docker Engine

Native Docker Engine — not Docker Desktop, which is a Mac/Windows GUI product. Docker's official repository, not the distro's own (often outdated) package, keeps you on a current release.

terminal

# add Docker's official GPG key and repository
sudo apt-get update
sudo apt-get install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

# install
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

terminal

# add Docker's official repository
sudo dnf -y install dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo

# install
sudo dnf install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# start it now and on every boot
sudo systemctl enable --now docker

Confirm it's installed:

terminal

docker --version
docker compose version

Run Docker without sudo

Optional, but saves typing sudo before every command for the rest of this guide.

terminal

sudo usermod -aG docker $USER
newgrp docker   # or log out and back in

Verify: docker run hello-world should work without sudo.

GPU: install the container toolkit Skip if no NVIDIA GPU

Unlike Windows (where Docker Desktop bundles GPU support), Linux needs an explicit extra package — the NVIDIA Container Toolkit — so Docker knows how to hand a GPU to a container. Install your regular NVIDIA driver first if you haven't (nvidia-smi should already work outside Docker before you proceed).

terminal

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

# wire it into Docker's runtime config, then restart Docker
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

terminal

curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo

sudo dnf install -y nvidia-container-toolkit

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

GPU: verify passthrough Skip if no NVIDIA GPU

Same checkpoint used on every OS this app supports — confirm the GPU reaches a container before involving this app at all, so a failure here is unambiguously a driver/toolkit issue, not an app bug.

terminal

docker run --rm --gpus=all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

This should print your GPU. If it fails, see Troubleshooting below.

Install Ollama

Runs natively (not in Docker) for direct GPU access, same as every other OS.

terminal

curl -fsSL https://ollama.com/install.sh | sh

ollama pull llama3.1:8b
ollama pull embeddinggemma

Ollama installs itself as a systemd service and starts automatically — confirm with systemctl status ollama.

Get the app & configure

Copy the project onto this machine, then from inside its directory:

terminal — inside the project directory

cp .env.docker.example .env

Edit .env and set a real JWT_SECRET (openssl rand -hex 32) before exposing this beyond your own machine — the placeholder value is intentionally insecure.

Start the stack

CPU-only stack:

terminal

docker compose up -d --build

With GPU fine-tuning enabled (after steps 3–4 above) — a Compose override file that swaps just the worker service, everything else is unchanged:

terminal

docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d --build

Watch it come up:

terminal

docker compose logs -f backend worker

Then open http://localhost.

If you're on a headless server (no browser on the machine itself), replace localhost with the server's IP or hostname from another machine on the same network, and make sure port 80 (and 8000, if you want direct API access) is allowed through ufw/firewalld if either is active.

First walkthrough

Click Create org, register your account.
Knowledge Base → upload a document → wait for cleaned.
Datasets → create a dataset → Generate from the document → approve at least 6 examples.
If you set up the GPU path: Fine-Tuning → confirm the badge says NVIDIA CUDA → start a job against the smallest base model first.
Once a fine-tuned model completes, it's a normal Ollama tag — try it in Conversation Testing or Evaluation immediately.

Troubleshooting

Step 4's GPU test fails›

Check, in order:

nvidia-smi works outside Docker first — if not, it's a driver problem, fix that before anything else
nvidia-container-toolkit is actually installed: dpkg -l | grep nvidia-container-toolkit (or rpm -q on Fedora/RHEL)
You ran nvidia-ctk runtime configure --runtime=docker and restarted Docker afterward — this step edits /etc/docker/daemon.json, and Docker won't pick it up without a restart
You're not running the open-source nouveau driver by accident (lsmod | grep nouveau should show nothing if the proprietary driver is active)

permission denied on /var/run/docker.sock›

Step 2 wasn't applied yet, or your shell session predates it. Run newgrp docker, or fully log out and back in, then retry.

Port 80 or 8000 already in use›

Something else on this machine (nginx, another app) is holding that port. Either stop it, or change the host side of the ports: mapping in docker-compose.yml (e.g. "80:80" → "8080:80").

Can't reach the app from another machine on the network›

Check the firewall: sudo ufw status (Debian/Ubuntu) or sudo firewall-cmd --list-all (Fedora/RHEL). Allow the port if it's active: sudo ufw allow 80/tcp, for example.

docker compose logs worker shows a Python/CUDA error during fine-tuning›

This CUDA training path was built without NVIDIA hardware available to test against — a version conflict or API mismatch on first real run is plausible, not a sign of a deeper problem. Copy the full traceback; it's usually a quick pin adjustment in backend/requirements-cuda.txt or a small argument fix in backend/app/services/finetune_scripts/cuda_train.py/cuda_fuse.py.

Command reference

What	Command
Start (CPU stack)	`docker compose up -d --build`
Start with GPU fine-tuning	`docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d --build`
Tail logs	`docker compose logs -f backend worker`
Check GPU is visible to the worker	`docker compose exec worker nvidia-smi`
Stop, keep data	`docker compose down`
Stop, delete all data	`docker compose down -v`
Check Docker's own status	`systemctl status docker`