Llama.cpp now supports tool calling (OpenAI-compatible) (opens in new tab)

(github.com)

3 pointsochafik1y ago1 comments

1 comments

Supports 8+ models’ native formats (Llama, Functionary, Hermes, Mistral, Firefunction, DeepSeek) + generic fallback for all other models.

Runs anywhere; on a Mac:

  brew install llama.cpp
  llama-server --jinja -fa -hf bartowski/Qwen2.5-7B-Instruct-GGUF:Q4_K_M

Still fresh / lots of bugs to discover, feedback welcome!

j / k navigate · click thread line to collapse

ochafikOP1y ago

Supports 8+ models’ native formats (Llama, Functionary, Hermes, Mistral, Firefunction, DeepSeek) + generic fallback for all other models.

Runs anywhere; on a Mac:

  brew install llama.cpp
  llama-server --jinja -fa -hf bartowski/Qwen2.5-7B-Instruct-GGUF:Q4_K_M

Still fresh / lots of bugs to discover, feedback welcome!

j / k navigate · click thread line to collapse