npm

mlx-lm

Run and interact with large language models using MLX framework

brewmacoslinux

Try with needOr install directly

Source

About

Run LLMs with MLX

Commands

mlx-lm

Examples

Generate text from a prompt using a default model$ mlx-lm generate --prompt 'Hello, how are you?'

Run inference with a specific quantized model$ mlx-lm generate --model mlx-community/Mistral-7B-Instruct-v0.1-4bit --prompt 'Explain quantum computing'

Run an interactive chat session with a model$ mlx-lm chat --model mlx-community/Llama-2-7b-chat-4bit