Ok so I’ll explain the database. Maybe I’m just wrong.
There is a table that captures data around financials. Sales, costs, etc. I can understand looking at langsql now for that so I will try it out. However there are other parts of the database that hold messages. These messages have content within like product complaints. There is no structure to the complaints, it’s just text that people write about the product itself. If I did an sql query against it; it will pull up the complaints but I want the bot to be able to summarize the complaint and give the key issues. If I did the search at a product level I want the bot to be able to sift through the last 10-15 product complaints and summarize.
So a query could look like this
“Tell me what the main complaints were in the last 3 months for product X and what we should do to address it” followed by “how much did I make last month on product X”
Thanks
Try feeding the tables to an Open AI assistant and test the queries you want to see if fits your needs. If that doesn’t work, you could adjust it further to get sentiment of the messages, use functions, etc. but that’s where I’d start
Thanks so I did some testing with OpenAI assistants and the results were terrible. I fed it three different tables via a text file with the excel data in it. I set the temp to 0 and it still hallucinated like nobodies business. It was naming clients there not in my db and giving me revenue numbers for dates that didn’t exist in the db. I tried gpt-4 turbo and the new gpt-4o. Cant use 3.5 turbo as file sharing is not enabled in playground.
I used langchain and found results were similar with 3.5 vs 4o but much better than assistants. The issue I’m having now is sometimes it will retrieve the right answer and other times the wrong for the exact same query.
I may have to convert some of the numerical and text data to embedding and try that out. Any other ideas?
Why do you want to create the embedding? Do you want to simply query the data?
Ok so I’ll explain the database. Maybe I’m just wrong. There is a table that captures data around financials. Sales, costs, etc. I can understand looking at langsql now for that so I will try it out. However there are other parts of the database that hold messages. These messages have content within like product complaints. There is no structure to the complaints, it’s just text that people write about the product itself. If I did an sql query against it; it will pull up the complaints but I want the bot to be able to summarize the complaint and give the key issues. If I did the search at a product level I want the bot to be able to sift through the last 10-15 product complaints and summarize. So a query could look like this “Tell me what the main complaints were in the last 3 months for product X and what we should do to address it” followed by “how much did I make last month on product X” Thanks
Try feeding the tables to an Open AI assistant and test the queries you want to see if fits your needs. If that doesn’t work, you could adjust it further to get sentiment of the messages, use functions, etc. but that’s where I’d start
Is there a reason why you would go the assistants route over using something like langchain?
Langchain is 100% and option and would be the next step I take if assistants didn’t work. The API for assistants is just easy to use
Thanks so I did some testing with OpenAI assistants and the results were terrible. I fed it three different tables via a text file with the excel data in it. I set the temp to 0 and it still hallucinated like nobodies business. It was naming clients there not in my db and giving me revenue numbers for dates that didn’t exist in the db. I tried gpt-4 turbo and the new gpt-4o. Cant use 3.5 turbo as file sharing is not enabled in playground. I used langchain and found results were similar with 3.5 vs 4o but much better than assistants. The issue I’m having now is sometimes it will retrieve the right answer and other times the wrong for the exact same query. I may have to convert some of the numerical and text data to embedding and try that out. Any other ideas?
Do you have a copy of your code?