SayIt/server

🎙️ SayIt

Just say it, and write well

Self-hosted speech-to-text server with real-time ASR and AI cleanup.

Quick Start · Architecture · Configuration · Deployment · License

---

What is SayIt?

SayIt is a self-hosted speech-to-text service that combines real-time ASR (Qwen3-ASR) with AI cleanup. It provides:

• Browser Demo — Record and transcribe directly in the browser, no install needed
• Windows Desktop App — Push-to-talk with automatic paste into any application
• Admin Dashboard — Monitor sessions, performance metrics, GPU/CPU usage
• REST & WebSocket API — Integrate speech-to-text into your own applications

Everything runs on a single server with one GPU.

Features

• 🎯 Real-time ASR — Qwen3-ASR-1.7B with vLLM acceleration, <2s latency
• ✨ AI Cleanup — Converts spoken language to written text (Azure OpenAI / Groq / Ollama)
• 🔥 Hotword Boosting — Custom vocabulary for domain-specific terms
• 📊 Admin Dashboard — Session analytics, performance percentiles, system monitoring
• 🐳 Docker Ready — docker compose up with GPU support, model baked into image
• 🔒 Security — Rate limiting, WebSocket connection limits, admin path isolation
• 📦 SQLite / PostgreSQL — SQLite for single-node, PostgreSQL for cluster deployment

Architecture

                    ┌─────────────────────────────────────┐
                    │            ALB (HTTPS 443)           │
                    │         ACM TLS termination          │
                    └──────────────┬──────────────────────┘
                                   │
                    ┌──────────────▼──────────────────────┐
                    │         FastAPI Backend (:8000)       │
                    │                                      │
                    │  /              Landing page + demo   │
                    │  /api/*         REST APIs             │
                    │  /ws/transcribe WebSocket streaming   │
                    │  /admin/*       Dashboard (blocked    │
                    │                 on public, SSH only)  │
                    │                                      │
                    │  ┌────────┐  ┌─────┐  ┌──────────┐  │
                    │  │Qwen3   │  │ LLM │  │Telemetry │  │
                    │  │ASR+vLLM│  │proxy│  │ SQLite/  │  │
                    │  │ (GPU)  │  │     │  │   PG     │  │
                    │  └────────┘  └─────┘  └──────────┘  │
                    └─────────────────────────────────────┘

Quick Start

Prerequisites

• NVIDIA GPU with ≥16GB VRAM (e.g., A10G, L4, RTX 4090)
• Docker with NVIDIA Container Toolkit
• An LLM API key (Azure OpenAI, OpenAI, Groq, or local Ollama)

1. Clone and configure

git clone https://github.com/crosswk/SayIt.git
cd SayIt
cp .env.example .env
cp config.example.yaml config.yaml

Edit .env — fill in your API keys:

SAYIT_OPENAI_API_KEY=sk-...        # or SAYIT_AZURE_API_KEY
SAYIT_ADMIN_PASSWORD=your-password

Edit config.yaml — adjust LLM provider if needed (default: Azure OpenAI).

2. Start

docker compose up -d --build

First build takes ~15 minutes (downloads model + builds image). Subsequent starts take ~60 seconds (model loading).

3. Access

URL	Description
https://your-server/	Landing page + browser demo
https://your-server/healthz	Health check
http://localhost:8000/admin	Admin dashboard (via SSH tunnel)

Admin access: ssh -L 8000:localhost:8000 your-server then open http://localhost:8000/admin

Project Layout

SayIt/
├── backend/              # FastAPI backend
│   ├── app/
│   │   ├── main.py       # Routes, WebSocket handler
│   │   ├── config.py     # Nested config dataclasses
│   │   ├── asr.py        # Qwen3-ASR engine + VAD
│   │   ├── llm.py        # AI cleanup (multi-provider)
│   │   ├── admin.py      # Admin API endpoints
│   │   ├── telemetry.py  # Usage tracking + analytics
│   │   ├── db.py         # SQLite / PostgreSQL abstraction
│   │   └── ratelimit.py  # Token-bucket rate limiter
│   ├── tests/
│   ├── Dockerfile
│   └── requirements.txt
├── gateway/              # HTTPS reverse proxy (Node.js)
├── web/                  # Landing page, browser demo, admin UI
├── prompts/              # System prompt + hotword files
├── config.example.yaml   # Configuration template
├── .env.example          # Secrets template
├── docker-compose.yml
└── LICENSE               # AGPL-3.0

Configuration

Configuration is split into two files:

File	Contains	Committed to git?
config.yaml	All settings (ASR, LLM, ports, logging, etc.)	Yes (template)
.env	Secrets only (API keys, passwords)	No

All environment variables use the SAYIT_ prefix. See config.example.yaml for full documentation.

Key settings

asr:
  engine: "qwen3"                    # or "firered"
  model: "Qwen/Qwen3-ASR-1.7B"
  device: "cuda:0"

llm:
  enabled: true
  provider: "openai"                 # azure / openai / groq / ollama

web_demo:
  enabled: true
  max_duration_sec: 600              # Max recording per session
  max_concurrency_per_ip: 3

admin:
  enabled: true

Database

Default: SQLite (zero config). For multi-node deployment:

telemetry:
  db_backend: "postgresql"
  db_path: "postgresql://user:pass@host:5432/sayit"

Or via environment variable: SAYIT_DB_URL=postgresql://...

Deployment

Single server (recommended for most users)

docker compose up -d --build

With ALB (AWS)

1. Request ACM certificate for your domain
2. Create ALB with HTTPS listener + ACM certificate
3. Target group → EC2:8000, health check /healthz
4. Block /admin/* at ALB level (fixed 404 response)
5. Access admin via SSH tunnel

See docs/deployment.md for details.

Development

# Create virtualenv
python3 -m venv .venv
source .venv/bin/activate
pip install -r backend/requirements.txt

# Run backend directly
cd backend
uvicorn app.main:app --host 0.0.0.0 --port 8000

# Run tests
python -m pytest backend/tests/ -v

API

WebSocket /ws/transcribe

// → Client sends
{"cmd": "start", "client_meta": {"user_id": "..."}, "app_context": {"process_name": "..."}}
// → Client sends PCM audio frames (16kHz, 16-bit, mono)
{"cmd": "stop", "usage_meta": {"ptt_hold_ms": 1500}}

// ← Server responds
{"type": "asr", "text": "你好世界", "asr_ms": 450}
{"type": "final", "asr_text": "你好世界", "llm_text": "你好，世界。", "asr_ms": 450, "llm_ms": 320}
{"type": "done"}

REST

Method	Path	Description
GET	/healthz	Health check
GET	/api/public/config	Public site configuration
GET	/api/hotwords	Current hotword list
PUT	/api/hotwords	Update hotwords
GET	/admin/api/overview	Dashboard metrics (auth required)
GET	/admin/api/sessions	Session list (auth required)

See docs/api.md for full API reference.

License

This project is licensed under the GNU Affero General Public License v3.0.

You are free to self-host and modify SayIt. If you distribute a modified version or run it as a network service, you must make your source code available under the same license.

For commercial licensing inquiries, please contact the maintainers.