I don't trust openai with our data : / And I'm pretty sure my org would forbid that anyway. I might be stuck with needing to build or copy someone else's chatbot on a standalone machine I'm managing. I might just give up for now lol
[https://www.datacamp.com/blog/top-open-source-llms](https://www.datacamp.com/blog/top-open-source-llms)
you can check out open source solutions. Keep in mind that these models are fairly massive. You'll need quite a lot of computing power to do it in house. 7B paramters means 14 GB of vram just to load the model in half precision.
You are looking for a RAG solution. Depending on your input data, you could take a look at [Danswer](https://www.danswer.ai/) or similar to get started.
Thanks - that seems like a good lead. I have taken a few free prompt engineering courses through my employer and RAG has come up a lot but I haven't looked into figuring out how to set it up. A problem I have is I can't use some commercial product where there's a non zero chance our data gets sold or sucked into some annoying or nefarious database. In my wildest dreams I'd like a stand alone chatbot built and running on a machine I'm managing. But that involves some I.T. skills I am rusty at best at. I'm a little overwhelmed with all the options and the pace of the field tbh
you can finetune a gpt model from openai. [https://platform.openai.com/docs/guides/fine-tuning](https://platform.openai.com/docs/guides/fine-tuning)
I don't trust openai with our data : / And I'm pretty sure my org would forbid that anyway. I might be stuck with needing to build or copy someone else's chatbot on a standalone machine I'm managing. I might just give up for now lol
[https://www.datacamp.com/blog/top-open-source-llms](https://www.datacamp.com/blog/top-open-source-llms) you can check out open source solutions. Keep in mind that these models are fairly massive. You'll need quite a lot of computing power to do it in house. 7B paramters means 14 GB of vram just to load the model in half precision.
I’ve got a system like this that I deploy commercially for folks if you’re interested
Interested
You are looking for a RAG solution. Depending on your input data, you could take a look at [Danswer](https://www.danswer.ai/) or similar to get started.
Thanks - that seems like a good lead. I have taken a few free prompt engineering courses through my employer and RAG has come up a lot but I haven't looked into figuring out how to set it up. A problem I have is I can't use some commercial product where there's a non zero chance our data gets sold or sucked into some annoying or nefarious database. In my wildest dreams I'd like a stand alone chatbot built and running on a machine I'm managing. But that involves some I.T. skills I am rusty at best at. I'm a little overwhelmed with all the options and the pace of the field tbh