This is a fork

Forked from badlogic/fasttext-lid-server, this repository uses a newer debian version (bookworm), uses the bigger model (126MB) and gives you the top 3 results instead of only 1.

FastText Language ID Server

HTTP server that provides language identification using FastText models.

Quick Start

# Pull and run
docker run -p 8080:8080 suspiciousactivity/fasttext-lid-server

# Test
curl "http://localhost:8080/detect?text=Hello%20world"

API

GET /detect

Parameter: text (required)
Response: JSON with language code and probability

[
	{
		"language": "en",
		"probability": 0.987
	},
	{
		"language": "de",
		"probability": 0.234
	},
	{
		"language": "nl",
		"probability": 0.123
	}
]

Errors:
- 400: Missing text parameter
- 500: Prediction failed

Docker

Using docker run

# Default settings
docker run -p 8080:8080 suspiciousactivity/fasttext-lid-server

# Custom settings
docker run -p 9000:9000 \
  -e PORT=9000 \
  -e MODEL_FILE=/models/custom.ftz \
  -v $(pwd)/models:/models \
  suspiciousactivity/fasttext-lid-server

Using docker compose

version: '3.8'

services:
  lid-server:
    image: suspiciousactivity/fasttext-lid-server
    ports:
        - "${PORT:-8080}:${PORT:-8080}"
    environment:
        - MODEL_FILE=${MODEL_FILE:-lid.176.ftz}
        - PORT=${PORT:-8080}
    volumes:
        - ./models:/models
    healthcheck:
        test: ["CMD", "curl", "-f", "http://localhost:${PORT:-8080}/detect?text=Hello%20world"]
        interval: 5s
        timeout: 5s
        retries: 3

Run with:

# Default settings
docker compose up

# Custom port
PORT=9000 docker compose up

# Custom model
MODEL_FILE=/models/custom.ftz docker compose up

Building from Source

Prerequisites: CMake 3.14+, C++17 compiler

mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make

Run:

./fasttext_lid_server [model_path] [port]

2.0 KiB Raw Blame History