Cloudflare'i möödahiilimise teenus teadusartiklite kraapimiseks.
Praegune paigaldus: /opt/docker-compose/third/server/flaresolverr/docker-compose.yml
version: '3.8'
services:
flaresolverr:
image: ghcr.io/flaresolverr/flaresolverr:latest
container_name: flaresolverr
restart: unless-stopped
ports:
- "8191:8191"
environment:
- LOG_LEVEL=info
- TZ=Europe/Tallinn
networks:
- ai_network
extra_hosts:
- "host.docker.internal:host-gateway"
shm_size: '512mb'
networks:
ai_network:
external: true
name: main_ai_network
# Käivita
docker-compose -f /opt/docker-compose/third/server/flaresolverr/docker-compose.yml up -d
# Peata
docker-compose -f /opt/docker-compose/third/server/flaresolverr/docker-compose.yml down
# Taaskäivita
docker-compose -f /opt/docker-compose/third/server/flaresolverr/docker-compose.yml restart
# Logid
docker-compose -f /opt/docker-compose/third/server/flaresolverr/docker-compose.yml logs -f
# Logid (viimased 50 rida)
docker-compose -f /opt/docker-compose/third/server/flaresolverr/docker-compose.yml logs --tail=50
# Vaata konteineri staatust
docker ps | grep flaresolverr
# Logid konteineri nime järgi
docker logs flaresolverr
# Jälgi logisid reaalajas
docker logs -f flaresolverr
# Sisene konteinerisse (veatuvastuseks)
docker exec -it flaresolverr /bin/bash
# Vaata konteineri üksikasju
docker inspect flaresolverr
# Taaskäivita konteiner
docker restart flaresolverr
# Peata ja eemalda konteiner
docker stop flaresolverr && docker rm flaresolverr
# Tõmba uusim image ja käivita uuesti
docker-compose -f /opt/docker-compose/third/server/flaresolverr/docker-compose.yml pull
docker-compose -f /opt/docker-compose/third/server/flaresolverr/docker-compose.yml up -d
# Test kas teenus töötab
curl http://100.87.1.24:8191/v1
# Päris testpäring
curl -X POST http://100.87.1.24:8191/v1 \
-H "Content-Type: application/json" \
-d '{"cmd":"request.get","url":"https://httpbin.org/ip","maxTimeout":10000}'
import requests
response = requests.post("http://100.87.1.24:8191/v1", json={
"cmd": "request.get",
"url": "https://www.sciencedirect.com/search?qs=etnomatematika",
"session": "minu_session",
"maxTimeout": 60000,
})
data = response.json()
html = data["solution"]["response"]
Muudetav failis docker-compose.yml:
LOG_LEVEL=info – tavaline (soovitatud)LOG_LEVEL=debug – detailne (veatuvastuseks)LOG_LEVEL=warning – ainult hoiatused ja veadLOG_LEVEL=error – ainult veadMuutmise järel taaskäivita: docker-compose restart
ai_network (aka main_ai_network)8191 on avatud kõikidel liidestelhost.docker.internalTestiks kõikidel saitidel korraga:
cd ~/Dropbox/Project && .venv/bin/python -m kraapija.test_flaresolverr
Tulemused (mai 2026):
| # | Sait | Tulemus | Aeg |
|---|---|---|---|
| 1 | httpbin.org | ✅ | 1.0s |
| 2 | ScienceDirect | ✅ | 1.3s |
| 3 | G2.com | ✅ | 1.0s |
| 4 | Google Search | ✅ | 1.3s |
| 5 | PubMed | ✅ | 1.9s |
| 6 | Sci-Hub | ❌ (blokeeritud) | – |
| 7 | Google Scholar | ✅ | 1.5s |
| 8 | JSTOR | ✅ | 2.7s |
| 9 | ResearchGate | ✅ (Challenge solved!) | 20.3s |
8/9 testi läbitud.