Practical Llama 3 inference in Java (opens in new tab)

(github.com)

4 pointsmukel1y ago1 comments

1 comments

mukelOP1y ago

Llama3.java: featuring .GGUF file format support, Q8_0 and Q4_0 quantizations, fast matrix/vector multiplication routines using Java's Vector API; served by a simple CLI with a --chat mode to interact with the Llama 3 models.

j / k navigate · click thread line to collapse

Practical Llama 3 inference in Java (opens in new tab)

(github.com)

4 pointsmukel1y ago1 comments

1 comments

mukelOP1y ago

Llama3.java: featuring .GGUF file format support, Q8_0 and Q4_0 quantizations, fast matrix/vector multiplication routines using Java's Vector API; served by a simple CLI with a --chat mode to interact with the Llama 3 models.

j / k navigate · click thread line to collapse