Best Open source LLM and embeddings

Hi, I have been asked to create a "chat with your documents" kind of product, but using open source LLM's, locally stored on a users desktop. Kind of like privategpt.io.

Embeddings have also to be created/queried on a user's desktop.

Is there any LLM and embeddings comparable to Open AI's products? My use case is for lawyers, so cant use Open AI, as they cant upload their documents on any server at all, everything has to be in house.

Is privategpt good enough, or is it just garbage, full of hallucinations? Will there be something for my use case available in a few months?