藍+85CD
24432f881c
* refactor(test): use `granite4:350m-h` instead of `granite3:2b`, `qwen3:0.6b` * refactor(test): add 1b-h model, update test snapshot * refactor: simplify embed test * chore: revert changes * fix: test * fix: test * fix: try another seed * fix: test
2.3 KiB
Contributing
Before
Before creating a Pull Request, please contact us at issues or discussions to confirm that the feature in question is one we need.
When creating a Pull Request, please make sure you have read and agreed to it:
- Allow us (xsAI Maintainers) to modify your code at any time.
- Usually, this is to make it simpler, but there may be exceptions.
- Allow us (xsAI Maintainers) to create new Pull Request with your code and close the old ones while adding the
Co-authored-bytrailers.
We will do our best to try to explain and discuss the possible implementation approaches as much detailed as possible before modifications and Pull Request state changes, but still, sometimes it may be much more efficient for us to directly use actual code to address our opinions.
Testing
We weren't supposed to connect to OpenAI's paid APIs while testing, so we chose some local services to use for test:
embed, {generate,stream}Text, {generate,stream}Object, tool
It uses Ollama.
The model used may change as needed, currently granite3.3:2b (chat completion, tool calling), qwen3:0.6b (reasoning), and all-minilm (embedding).
ollama pull all-minilm
ollama pull granite4:350m-h
ollama pull granite4:1b-h
ollama pull qwen3:0.6b
ollama serve
generateSpeech
It uses openai-edge-tts.
You can quickly start one via docker or docker-compose:
docker run --rm -p 5050:5050 -e PORT=5050 travisvn/openai-edge-tts:latest
services:
openai-edge-tts:
image: travisvn/openai-edge-tts:latest
ports:
- 5050:5050
generateTranscription
It uses whisper.cpp and the ggml-large-v3-turbo-q5_0 model.
If you're using Nix, you can do this:
{ pkgs, ... }: {
home.packages = with pkgs; [ openai-whisper-cpp ];
}
Then store the model files in the directory you want and start the server:
mkdir models
cd models
whisper-cpp-download-ggml-model large-v3-turbo-q5_0
cd ..
whisper-cpp-server --host 127.0.0.1 --port 9010 -nt -m models/ggml-large-v3-turbo-q5_0.bin --request-path /audio/transcriptions --inference-path ""