text-embeddings-inference

Fast inference server for text embedding models

brewmacoslinux

Try with needOr install directly

About

Blazing fast inference solution for text embeddings models

text-embeddings-inference

start embedding server on default port$ text-embeddings-inference

run embedding server on custom port$ text-embeddings-inference --port 8080

load specific embedding model from hugging face$ text-embeddings-inference --model-id sentence-transformers/all-MiniLM-L6-v2

enable GPU acceleration for faster embeddings$ text-embeddings-inference --cuda-cores 8

check version and available options$ text-embeddings-inference --help