npm

text-embeddings-inference

Blazing fast inference solution for text embedding models

brewmacoslinux

Try with needOr install directly

Source

About

Blazing fast inference solution for text embeddings models

Commands

text-embeddings-inference

Examples

Start the inference server on default port 3000$ text-embeddings-inference --model-id sentence-transformers/all-MiniLM-L6-v2

Start server on custom port with GPU acceleration$ text-embeddings-inference --model-id sentence-transformers/all-MiniLM-L6-v2 --port 8080

Query the running server with a text embedding request$ curl http://localhost:3000/embed -X POST -H 'Content-Type: application/json' -d '{"inputs": ["Hello world"]}'