|
@@ -0,0 +1,343 @@
|
|
|
|
|
+# Open WebUI – Docker kasutusjuhend
|
|
|
|
|
+
|
|
|
|
|
+Professionaalne LLM veebiliides. Vestlus, RAG (dokumentide põhine küsimine), veebiotsing.
|
|
|
|
|
+
|
|
|
|
|
+## Arhitektuur (2025-07-05 seisuga)
|
|
|
|
|
+
|
|
|
|
|
+| Komponent | Asukoht | Staatus |
|
|
|
|
|
+|-----------|---------|---------|
|
|
|
|
|
+| **Open WebUI** | Docker konteiner (`open-webui`) | port 3010 |
|
|
|
|
|
+| **Ollama** | Native systemd teenus (`ollama.service`) | port 11434 |
|
|
|
|
|
+| **Ollama GPU** | ROCm 7.2.2, gfx1151 (Radeon 8060S) | hostil |
|
|
|
|
|
+| **mcp-filesystem-sse** | Docker konteiner (`mcp-filesystem-sse`) | port 3100 |
|
|
|
|
|
+| **mcp-terminal** | Docker konteiner (`mcp-terminal`) | port 3110 |
|
|
|
|
|
+| **tika** | Docker konteiner (`tika`) | port 9998 |
|
|
|
|
|
+| **Qdrant** | Docker konteiner (`qdrant`) | port 6333-6334 |
|
|
|
|
|
+| **SearXNG** | Docker konteiner (`searxng`) | port 8888 |
|
|
|
|
|
+| **PostgreSQL** | Docker konteiner (`postgres_postgis`) | port 5432 |
|
|
|
|
|
+
|
|
|
|
|
+**NB!** Ollama EI OLE enam Dockeris. See jookseb native systemd teenusena koos ROCm GPU toega.
|
|
|
|
|
+Vana Dockeri konteiner (`ollama-rocm`) on eemaldatud.
|
|
|
|
|
+
|
|
|
|
|
+## GPU / ROCm
|
|
|
|
|
+
|
|
|
|
|
+Server: **AZW GTR Pro** mini-PC
|
|
|
|
|
+- CPU: AMD Ryzen AI Max+ 395 (16C/32T, Zen 5)
|
|
|
|
|
+- GPU: AMD Radeon 8060S (RDNA 3.5, gfx1151, 40 CU)
|
|
|
|
|
+- ROCm: 7.2.2 installeeritud hostil (`/opt/rocm/`)
|
|
|
|
|
+- NPU: AMD RyzenAI (aie2p) – hetkel kasutamata
|
|
|
|
|
+
|
|
|
|
|
+ROCm teegid on registreeritud süsteemis:
|
|
|
|
|
+
|
|
|
|
|
+ # Kontrolli GPU olekut
|
|
|
|
|
+ rocm-smi
|
|
|
|
|
+ rocminfo
|
|
|
|
|
+ cat /sys/class/drm/card1/device/gpu_busy_percent
|
|
|
|
|
+
|
|
|
|
|
+ # ROCm teegid
|
|
|
|
|
+ ls /opt/rocm/lib/libamdhip64*
|
|
|
|
|
+ ldconfig -p | grep amdhip64
|
|
|
|
|
+
|
|
|
|
|
+## Ollama haldus (native, systemd)
|
|
|
|
|
+
|
|
|
|
|
+Ollama teenus jookseb hostil `ollama.service` nimega:
|
|
|
|
|
+
|
|
|
|
|
+ # Ollama teenuse olek
|
|
|
|
|
+ sudo systemctl status ollama
|
|
|
|
|
+
|
|
|
|
|
+ # Taaskäivita
|
|
|
|
|
+ sudo systemctl restart ollama
|
|
|
|
|
+
|
|
|
|
|
+ # Logid
|
|
|
|
|
+ journalctl -u ollama -f
|
|
|
|
|
+ journalctl -u ollama --no-pager -n 50
|
|
|
|
|
+
|
|
|
|
|
+ # Kontrolli GPU tuvastust
|
|
|
|
|
+ journalctl -u ollama --no-pager | grep -iE 'gpu|rocm|gfx|inference'
|
|
|
|
|
+
|
|
|
|
|
+ # Kasuta ollama CLI-d
|
|
|
|
|
+ ollama list # mudelite nimekiri
|
|
|
|
|
+ ollama ps # hetkel laetud mudelid
|
|
|
|
|
+ ollama show <nimi> # mudeli detailid
|
|
|
|
|
+
|
|
|
|
|
+Teenuse konfiguratsioon:
|
|
|
|
|
+- Service fail: `/etc/systemd/system/ollama.service`
|
|
|
|
|
+- Override: `/etc/systemd/system/ollama.service.d/override.conf`
|
|
|
|
|
+- Mudelite asukoht: `/home/ardo/.ollama/models`
|
|
|
|
|
+- Kasutaja: `ollama` (grupid: `render`, `video`, `ardo`)
|
|
|
|
|
+- GPU override: `HSA_OVERRIDE_GFX_VERSION=11.5.1`
|
|
|
|
|
+- ROCm teegid: `LD_LIBRARY_PATH=/opt/rocm/lib`
|
|
|
|
|
+
|
|
|
|
|
+### Paigaldatud mudelid
|
|
|
|
|
+
|
|
|
|
|
+| Mudel | Suurus | Kasutus |
|
|
|
|
|
+|-------|--------|---------|
|
|
|
|
|
+| `nomic-embed-text:latest` | 274 MB | **RAG embedding (GPU)** |
|
|
|
|
|
+| `qwen3:30b-fast` | 18 GB | LLM vestlus |
|
|
|
|
|
+| `deepseek-r1:14b` | 9.0 GB | LLM vestlus (reasoning) |
|
|
|
|
|
+| `translategemma:12b` | 8.1 GB | Tõlkimine |
|
|
|
|
|
+| `bge-m3:latest` | 1.2 GB | Varu-embedding |
|
|
|
|
|
+
|
|
|
|
|
+### Mudelite haldus
|
|
|
|
|
+
|
|
|
|
|
+ # Lae mudel
|
|
|
|
|
+ ollama pull qwen3:30b-fast
|
|
|
|
|
+
|
|
|
|
|
+ # Eemalda mudel
|
|
|
|
|
+ ollama rm nomic-embed-text-cpu
|
|
|
|
|
+
|
|
|
|
|
+ # Testi mudelit (verbose näitab tok/sek)
|
|
|
|
|
+ ollama run nomic-embed-text "test" --verbose
|
|
|
|
|
+
|
|
|
|
|
+## Embedding konfiguratsioon (RAG)
|
|
|
|
|
+
|
|
|
|
|
+OpenWebUI embeddingud on seadistatud kasutama **nomic-embed-text:latest** mudelit
|
|
|
|
|
+koos GPU kiirendusega. **Ära kasuta** `nomic-embed-text-cpu` mudelit – see on
|
|
|
|
|
+ainult CPU jaoks (`num_gpu=0`) ja oluliselt aeglasem.
|
|
|
|
|
+
|
|
|
|
|
+Compose failis: `RAG_EMBEDDING_MODEL=nomic-embed-text`
|
|
|
|
|
+
|
|
|
|
|
+### Jõudluse võrdlus
|
|
|
|
|
+
|
|
|
|
|
+| Mõõdik | CPU mudel (-cpu) | GPU mudel (latest) | Kiirendus |
|
|
|
|
|
+|--------|-------------------|---------------------|-----------|
|
|
|
|
|
+| 901 embedit (batch) | ~39 s | ~9.5 s | **4x** |
|
|
|
|
|
+| 848 embedit (batch) | ~39 s | ~8.1 s | **5x** |
|
|
|
|
|
+| Üksik API päring | ~28 s | ~0.3 s | **100x** |
|
|
|
|
|
+
|
|
|
|
|
+### Embedding API test
|
|
|
|
|
+
|
|
|
|
|
+ # Testi embeddingut
|
|
|
|
|
+ curl http://localhost:11434/api/embeddings -d '{
|
|
|
|
|
+ "model": "nomic-embed-text",
|
|
|
|
|
+ "prompt": "How to implement semantic search with Python"
|
|
|
|
|
+ }'
|
|
|
|
|
+
|
|
|
|
|
+### Vektorbaasi puhastamine
|
|
|
|
|
+
|
|
|
|
|
+ # Qdrant kollektsiooni kustutamine
|
|
|
|
|
+ curl -X DELETE http://qdrant:6333/collections/open-webui_files
|
|
|
|
|
+
|
|
|
|
|
+ # Läbi OpenWebUI skripti
|
|
|
|
|
+ docker exec -ti open-webui bash -c "python /scripts/clean_qdrant.py"
|
|
|
|
|
+
|
|
|
|
|
+## Seadistus
|
|
|
|
|
+
|
|
|
|
|
+Kasutab:
|
|
|
|
|
+- **Ollama** (native, systemd) – LLM mudelid ja embeddingud
|
|
|
|
|
+- **SearXNG** – Veebiotsing
|
|
|
|
|
+- **Qdrant** – Vektorbaas (väljaspool seda compose'i)
|
|
|
|
|
+- **PostgreSQL** – Andmebaas
|
|
|
|
|
+
|
|
|
|
|
+Olulised keskkonnamuutujad compose failis:
|
|
|
|
|
+- `USE_OLLAMA_DOCKER=false` – väldib OpenWebUI sisemist ollama käivitust
|
|
|
|
|
+- `RAG_EMBEDDING_MODEL=nomic-embed-text` – GPU embedding mudel
|
|
|
|
|
+- `RAG_EMBEDDING_ENGINE=ollama` – embedding API läbi ollama
|
|
|
|
|
+- `OLLAMA_API_BASE_URL=http://100.87.1.24:11434` – ollama API
|
|
|
|
|
+- `VECTOR_DB=qdrant` – vektorbaasi tüüp
|
|
|
|
|
+
|
|
|
|
|
+## Docker käsud
|
|
|
|
|
+
|
|
|
|
|
+ # Ava kaust
|
|
|
|
|
+ cd /opt/docker-compose/third/server/openwebui/
|
|
|
|
|
+
|
|
|
|
|
+ # Käivita (ainult open-webui teenus)
|
|
|
|
|
+ docker-compose up -d open-webui
|
|
|
|
|
+
|
|
|
|
|
+ # Käivita kõik teenused
|
|
|
|
|
+ docker-compose up -d
|
|
|
|
|
+
|
|
|
|
|
+ # Peata
|
|
|
|
|
+ docker-compose down
|
|
|
|
|
+
|
|
|
|
|
+ # Taaskäivita
|
|
|
|
|
+ docker-compose restart open-webui
|
|
|
|
|
+
|
|
|
|
|
+ # Logid (ainult open-webui)
|
|
|
|
|
+ docker-compose logs -f open-webui
|
|
|
|
|
+
|
|
|
|
|
+ # Logid (kõik)
|
|
|
|
|
+ docker-compose logs -f
|
|
|
|
|
+
|
|
|
|
|
+ # Uuenda
|
|
|
|
|
+ docker-compose pull open-webui
|
|
|
|
|
+ docker-compose up -d open-webui
|
|
|
|
|
+
|
|
|
|
|
+## Docker otse
|
|
|
|
|
+
|
|
|
|
|
+ # Konteinerite seis
|
|
|
|
|
+ docker ps | grep -E "open-webui|tika|mcp"
|
|
|
|
|
+
|
|
|
|
|
+ # Logid nime järgi
|
|
|
|
|
+ docker logs open-webui
|
|
|
|
|
+ docker logs tika
|
|
|
|
|
+
|
|
|
|
|
+ # Sisene konteinerisse
|
|
|
|
|
+ docker exec -it open-webui /bin/bash
|
|
|
|
|
+ docker exec -it tika /bin/bash
|
|
|
|
|
+
|
|
|
|
|
+## Teenused
|
|
|
|
|
+
|
|
|
|
|
+| Teenus | Port | URL |
|
|
|
|
|
+|--------|------|-----|
|
|
|
|
|
+| Open WebUI | 3010 | http://100.87.1.24:3010 |
|
|
|
|
|
+| Ollama API | 11434 | http://100.87.1.24:11434 (native, mitte Docker) |
|
|
|
|
|
+| MCP Filesystem | 3100 | http://127.0.0.1:3100/mcp (ainult localhost) |
|
|
|
|
|
+| MCP Terminal | 3110 | http://127.0.0.1:3110/mcp (ainult localhost) |
|
|
|
|
|
+| Apache Tika | 9998 | http://100.87.1.24:9998 |
|
|
|
|
|
+
|
|
|
|
|
+## Tervisekontroll
|
|
|
|
|
+
|
|
|
|
|
+Open WebUI kontrollib iseenda tervist iga 30 sekundi järel:
|
|
|
|
|
+
|
|
|
|
|
+ # Testi käsitsi
|
|
|
|
|
+ curl http://100.87.1.24:3010/health
|
|
|
|
|
+
|
|
|
|
|
+ # Ollama tervisekontroll
|
|
|
|
|
+ curl http://100.87.1.24:11434/api/tags
|
|
|
|
|
+
|
|
|
|
|
+## Olulised kataloogid
|
|
|
|
|
+
|
|
|
|
|
+| Asukoht | Sisu |
|
|
|
|
|
+|---------|------|
|
|
|
|
|
+| `/opt/docker-compose/third/server/openwebui/` | Compose failid ja see dokument |
|
|
|
|
|
+| `/data/docker/open-webui/scripts` | Skriptid |
|
|
|
|
|
+| `/backups/docker/open-webui` | Varukoopiad |
|
|
|
|
|
+| `/home/ardo/.ollama/models` | Ollama mudelid (35 GB) |
|
|
|
|
|
+| `/opt/rocm/` | ROCm installatsioon |
|
|
|
|
|
+| `/etc/systemd/system/ollama.service` | Ollama systemd teenus |
|
|
|
|
|
+| `/etc/systemd/system/ollama.service.d/override.conf` | Ollama teenuse override |
|
|
|
|
|
+
|
|
|
|
|
+## Fallback: Dockeri ollama taastamine
|
|
|
|
|
+
|
|
|
|
|
+Kui on vaja minna tagasi Dockeri ollama peale (nt probleemid native installiga):
|
|
|
|
|
+
|
|
|
|
|
+ # Peata native ollama
|
|
|
|
|
+ sudo systemctl stop ollama
|
|
|
|
|
+
|
|
|
|
|
+ # Käivita Docker ollama (vana startup.sh põhjal)
|
|
|
|
|
+ docker run -d --name ollama-rocm --restart unless-stopped --device /dev/kfd --device /dev/dri --security-opt seccomp=unconfined --security-opt label=disable -p 11434:11434 -v /home/ardo/.ollama/:/root/.ollama/ -e OLLAMA_HOST=0.0.0.0:11434 -e HSA_OVERRIDE_GFX_VERSION=11.5.1 -e OLLAMA_NUM_PARALLEL=4 ollama/ollama:rocm
|
|
|
|
|
+
|
|
|
|
|
+ # Vajadusel muuda OpenWebUI compose failis:
|
|
|
|
|
+ # USE_OLLAMA_DOCKER=true
|
|
|
|
|
+ # ja taaskäivita:
|
|
|
|
|
+ # docker-compose up -d open-webui
|
|
|
|
|
+
|
|
|
|
|
+## MCP tööriistade kasutamine OpenWebUIs
|
|
|
|
|
+
|
|
|
|
|
+Vestluses ava `@` menüü või klõpsa `+` nuppu ja vali **Filesystem** või **Terminal**.
|
|
|
|
|
+
|
|
|
|
|
+### @Filesystem — failihaldus (AI tööriist)
|
|
|
|
|
+
|
|
|
|
|
+| Tööriist | Näide |
|
|
|
|
|
+|----------|-------|
|
|
|
|
|
+| `list_directory` | `@Filesystem list directory path=/data` |
|
|
|
|
|
+| `read_file` | `@Filesystem read file path=/scripts/clean_qdrant.py` |
|
|
|
|
|
+| `write_file` | `@Filesystem write file path=/backups/märkmed.txt content="Tere!"` |
|
|
|
|
|
+| `search_files` | `@Filesystem search files path=/scripts pattern=*.py` |
|
|
|
|
|
+| `get_file_info` | `@Filesystem get file info path=/data` |
|
|
|
|
|
+| `create_directory` | `@Filesystem create directory path=/backups/test` |
|
|
|
|
|
+
|
|
|
|
|
+Lubatud kataloogid: `/data`, `/scripts`, `/backups`
|
|
|
|
|
+
|
|
|
|
|
+### @Terminal — käsurida (AI tööriist)
|
|
|
|
|
+
|
|
|
|
|
+| Tööriist | Näide |
|
|
|
|
|
+|----------|-------|
|
|
|
|
|
+| `run_command` | `@Terminal run_command: docker ps` |
|
|
|
|
|
+| | `@Terminal run_command: df -h /` |
|
|
|
|
|
+| | `@Terminal run_command: docker logs open-webui --tail 20` |
|
|
|
|
|
+| | `@Terminal run_command: curl -s http://open-webui:8080/health` |
|
|
|
|
|
+| | `@Terminal run_command: uname -a` |
|
|
|
|
|
+
|
|
|
|
|
+Lubatud käsud: `ls`, `cat`, `grep`, `df`, `du`, `ps`, `docker ps/logs/inspect`, `curl`, `ping`, `uname`, `echo`, `stat`, `which`, `whoami`, `env`, `tree`, `find`, `head`, `tail`, `sort`, `uniq`, `file`, `free`, `uptime`, `id`, `date`, `host`
|
|
|
|
|
+Blokeeritud: `rm`, `sudo`, `shutdown`, `|` (torud), `$()`, `>`, `&&`, `docker rm/stop/kill`
|
|
|
|
|
+
|
|
|
|
|
+Näidisvestlus AI-ga:
|
|
|
|
|
+
|
|
|
|
|
+> **Sina:** "Vaata, mis toimub – docker logs open-webui --tail 10"
|
|
|
|
|
+> **AI:** (kasutab @Terminal run_command) "Siin on viimased logiread..."
|
|
|
|
|
+>
|
|
|
|
|
+> **Sina:** "Leia /scripts kataloogist kõik Python skriptid"
|
|
|
|
|
+> **AI:** (kasutab @Filesystem search_files) "Leidsin faili clean_qdrant.py"
|
|
|
|
|
+
|
|
|
|
|
+## Tailscale MCP serverite eksponeerimiseks
|
|
|
|
|
+
|
|
|
|
|
+MCP serverid on seotud ainult `127.0.0.1`-ga (turvalisuse mõttes).
|
|
|
|
|
+Kui vajad neile ligipääsu läbi Tailscale'i, kasuta:
|
|
|
|
|
+
|
|
|
|
|
+ # MCP Filesystem (Streamable HTTP)
|
|
|
|
|
+ sudo tailscale serve --bg --https 3100 http://127.0.0.1:3100
|
|
|
|
|
+
|
|
|
|
|
+ # MCP Terminal (Streamable HTTP)
|
|
|
|
|
+ sudo tailscale serve --bg --https 3110 http://127.0.0.1:3110
|
|
|
|
|
+
|
|
|
|
|
+ # Kontrolli staatust
|
|
|
|
|
+ sudo tailscale serve status
|
|
|
|
|
+
|
|
|
|
|
+ # Keela
|
|
|
|
|
+ sudo tailscale serve --https=3100 off
|
|
|
|
|
+ sudo tailscale serve --https=3110 off
|
|
|
|
|
+
|
|
|
|
|
+Pärast seda on MCP serverid kättesaadavad:
|
|
|
|
|
+- `https://100.87.1.24:3100/mcp`
|
|
|
|
|
+- `https://100.87.1.24:3110/mcp`
|
|
|
|
|
+
|
|
|
|
|
+**NB!** Brauseris `/sse` avamine ei tööta, sest SSE on püsiv ühendus (Streamable HTTP),
|
|
|
|
|
+mitte tavalise GET-päringuga laetav leht. MCP klient (OpenWebUI) haldab ühendust ise.
|
|
|
|
|
+
|
|
|
|
|
+## Võrguinfo
|
|
|
|
|
+
|
|
|
|
|
+- Kõik konteinerid on `main_ai_network` peal (compose'is `ai_network`)
|
|
|
|
|
+- Ligipääs Ollamale, SearXNG-le ja Qdrantile läbi `host.docker.internal`
|
|
|
|
|
+- `extra_hosts` on seadistatud compose failis
|
|
|
|
|
+
|
|
|
|
|
+## Vead ja tõrkeotsing
|
|
|
|
|
+
|
|
|
|
|
+### Embedimine aeglane
|
|
|
|
|
+- Kontrolli, et `RAG_EMBEDDING_MODEL=nomic-embed-text` (ilma `-cpu`-ta)
|
|
|
|
|
+- Kontrolli `ollama ps` – peaks näitama "100% GPU"
|
|
|
|
|
+- Kui mudel on `nomic-embed-text-cpu`, eemalda see: `ollama rm nomic-embed-text-cpu`
|
|
|
|
|
+- Kontrolli GPU olekut: `cat /sys/class/drm/card1/device/gpu_busy_percent`
|
|
|
|
|
+
|
|
|
|
|
+### Failide üleslaadimine ebaõnnestub – "Too many open files"
|
|
|
|
|
+- Põhjus: compose failis puudub `ulimits:` seadistus, Dockeri vaikimisi limiit on 1024
|
|
|
|
|
+- Lahendus: kontrolli, et compose failis on `ulimits.nofile.soft=65536` ja `hard=65536`
|
|
|
|
|
+- Kontrolli konteineri limiiti: `docker exec open-webui cat /proc/1/limits | grep "open files"`
|
|
|
|
|
+- Peaks näitama: `Max open files 65536 65536`
|
|
|
|
|
+
|
|
|
|
|
+### Ollama ei leia GPU-d
|
|
|
|
|
+
|
|
|
|
|
+ # Kontrolli ROCm teeke
|
|
|
|
|
+ ldconfig -p | grep amdhip64
|
|
|
|
|
+
|
|
|
|
|
+ # Kontrolli HSA override
|
|
|
|
|
+ journalctl -u ollama --no-pager | grep HSA_OVERRIDE
|
|
|
|
|
+
|
|
|
|
|
+ # Taaskäivita ollama
|
|
|
|
|
+ sudo systemctl restart ollama
|
|
|
|
|
+
|
|
|
|
|
+ # Kontrolli logisid
|
|
|
|
|
+ journalctl -u ollama -f | grep -i gpu
|
|
|
|
|
+
|
|
|
|
|
+### OpenWebUI ei ühendu Ollamaga
|
|
|
|
|
+
|
|
|
|
|
+ # Testi ollama API-t
|
|
|
|
|
+ curl http://100.87.1.24:11434/api/tags
|
|
|
|
|
+
|
|
|
|
|
+ # Kontrolli compose faili
|
|
|
|
|
+ grep OLLAMA /opt/docker-compose/third/server/open-webui/docker-compose.yml
|
|
|
|
|
+
|
|
|
|
|
+ # Taaskäivita OpenWebUI
|
|
|
|
|
+ cd /opt/docker-compose/third/server/openwebui/
|
|
|
|
|
+ docker-compose restart open-webui
|
|
|
|
|
+
|
|
|
|
|
+## Ajalugu
|
|
|
|
|
+
|
|
|
|
|
+| Kuupäev | Muudatus |
|
|
|
|
|
+|---------|----------|
|
|
|
|
|
+| 2025-07-05 | Ollama viidud Dockerist native systemd peale (ROCm GPU tugi) |
|
|
|
|
|
+| 2025-07-05 | Embedding mudel muudetud `nomic-embed-text-cpu` -> `nomic-embed-text:latest` |
|
|
|
|
|
+| 2025-07-05 | Lisatud `USE_OLLAMA_DOCKER=false` compose faili |
|
|
|
|
|
+| 2025-07-05 | `startup.sh` OpenWebUI osa asendatud docker-compose haldusega |
|
|
|
|
|
+| 2025-07-05 | Lisatud `ulimits.nofile=65536` compose faili (parandab "Too many open files") |
|