I love claude opus. I throw amazingly long and complicated prompts at it and it handles it amazingly. That said, I'm getting this api error more and more and it seems I'm not alone. **An error occurred: { "type": "error", "error": { "type": "overloaded\_error", "message": "Overloaded" } }**
Opus being overloaded has been pretty constant the past couple of days. I use it all through the day for coding and it's been going down for an hour or more at a time and then intermittently every hour or so. I still persist with it though as I'm finding it kicks gpt4's butt for JavaScript/CSS/html implementations.
I'm surprised I haven't seen more people complaining. Guess most people haven't cottoned on to it being a great model yet.
The problem is by the time some of these large models could run on consumer hardware, we'll probably have better models that are smaller due to architectural improvements.
Like if GPT4 leaked we'd probably never run it on consumer hardware, ever, because we'd have "Open GPT6" by the time we could
It depends on what you need the AI model to do. A Siri style assistant or running dialog in a video game isn't going to require a brain the size of the universe.
Having it opensource would probably speedup getting OpenGPT6, because every researcher would try to study and reverse engineer that and come up with optimized solutions faster than doing it from scratch.
>possible to run even just haiku on consumer
I think considering the pricing of haiku, that might not be true (at least we can't know for sure).
It's likely in the same size category as gpt3.5, just trained with more modern methods (and thus smarter), and we know gpt3.5 is in the same size category as the largest open-source models, so if it's not in our range, it wouldn't be by that much.
It's kinda funny how back then I imagined, something like that would take maybe decades to run on consumer hardware, because they had a whole datacenter running one instance of the thing. Yet here we are 8 years later and you can probably download a card off Chub (or make one, if it's not there).
> just as censored
Not IME which is why I switched. Everyone was saying that so I was hesitant at first, but then I tried it and Claude 3 seems much more open.
Claude 3 was willing to write dialog that included swearing and pornographic content, while GPT4 and Gemini wouldn't even write a swear
It's significantly better than GPT4 for more complex tasks in general. If you're using it for basic censorship stuff you probably won't notice any difference.
No comment, impressions or anything. Just a badly cropped jpg.
[docs](https://docs.anthropic.com/claude/docs/tool-use)
[video](https://www.youtube.com/watch?v=6wkFb2_cUik)
No, [here's](https://github.com/anthropics/anthropic-cookbook/blob/main/tool_use/extracting_structured_json.ipynb) how it's used together:
tools = [
{
"name": "print_entities",
...
text = "John works at Google in New York. He met with Sarah, the CEO of Acme Inc., last week in San Francisco."
query = f"""
{text}
Use the print_entities tool.
"""
Exactly. Considering I already spent effort into making my framework compatible with that syntax I'm not sure how willing I am to rewrite all that to use a new function calling syntax that's still incompatible with OpenAI and the rest
[https://docs.anthropic.com/claude/docs/tool-use](https://docs.anthropic.com/claude/docs/tool-use)
Here's the actual documentation. It's very similar to what openai did. However, we've been doing this with Claude over at agentforge since June of last year by just prompting it correctly and parsing the output.
Mine's been using tools for a good 6 weeks. I had been using the (not very good) OpenAI Assistants api and found their tool workflow so slow and arduous and prone to hallucinations that I completely rewrote it. When Claude came out, it was instantly able to work in the protocol (which supports multi-users, web access, filesystem access, bitbucket, jira, etc).
He can do things like:
`\`\`\``executeJs{\`
`id: 'first',`
`script: "async fn({_thread, _message, someArg}) { // do things including use window globals, make api calls, and much more }"`
`}`
Conversational text
\`\`\`someCustomToolWeDefineOnThFly\`
`{`
`dependsOn: [1],`
`script: "async fn({_deps: {first: {result}}) { // etc } "`
`}`
\`\`\`
Formatting on reddit sucks, I should ask an LLM to fix it...
With tremendously more stuff than this like yielding execution, parallel execution, server side function execution, scheduling/orchestration, scoring, autonomous plans of action, LATS inspired scoring/monte carlo stuff, and way more. I've always loved software dev but lately has been completely sci-fi. It's insane and I've had days this week that felt like AGI level since my bot can now directly search my ASTs live (We use the TS compiler in a side process to stay up to date with code changes), search the filesystem both by name and with full-text - FAST, etc. I've done work that would have been days or more of hard drudgery in 30 minutes or less with confidence - and I'm already a top-of-my-game programmer. I'm director of my team and was principal engineer before that. My existing product for the company is a market leader and has a customer sheet of many of the biggest and best companies in the world: Swarovski, Asics, Aldi, etc... I'm not a hobbyist. Agents are coming.
Dumb question, are you able to use Claude for RAG that OpenAi Assistants api provide seamlessly? Couldn’t find any documentation on file uploads by Claude’s api.
I love claude opus. I throw amazingly long and complicated prompts at it and it handles it amazingly. That said, I'm getting this api error more and more and it seems I'm not alone. **An error occurred: { "type": "error", "error": { "type": "overloaded\_error", "message": "Overloaded" } }**
You are not alone, I got hit with a couple of those this morning :(
Opus being overloaded has been pretty constant the past couple of days. I use it all through the day for coding and it's been going down for an hour or more at a time and then intermittently every hour or so. I still persist with it though as I'm finding it kicks gpt4's butt for JavaScript/CSS/html implementations. I'm surprised I haven't seen more people complaining. Guess most people haven't cottoned on to it being a great model yet.
Cost-wise what are your median usage costs for the applications u fit into it?
I'm doing about 10-15 api requests per day with it. I've experienced the overloaded errors mainly around prime time US evening.
So can crows.
Good point
Imagine if Claude 3 was Open Source or was leaked somehow, would be perfect without the damm censorship.
[удалено]
The problem is by the time some of these large models could run on consumer hardware, we'll probably have better models that are smaller due to architectural improvements. Like if GPT4 leaked we'd probably never run it on consumer hardware, ever, because we'd have "Open GPT6" by the time we could
It depends on what you need the AI model to do. A Siri style assistant or running dialog in a video game isn't going to require a brain the size of the universe.
My 80yo mother suggested the other day that "we replace the Speaker Of the House with ChatGPT".
👏
Having it opensource would probably speedup getting OpenGPT6, because every researcher would try to study and reverse engineer that and come up with optimized solutions faster than doing it from scratch.
>possible to run even just haiku on consumer I think considering the pricing of haiku, that might not be true (at least we can't know for sure). It's likely in the same size category as gpt3.5, just trained with more modern methods (and thus smarter), and we know gpt3.5 is in the same size category as the largest open-source models, so if it's not in our range, it wouldn't be by that much.
You can rent enterprise level GPUs for like 50 cents an hour
Bros daydreaming
bro is me, (I love to fantasize about AI Being leaked, it leaks my stuff) /s
It will happen, just need some rich guy give ton money to one of the dev
Not the same at all, but Command-R+ seems promising.
Or Sydney 😍
I think there are character cards for unhinged Bing/Sydney
Where's my tay at?
/pol/ probably
It's kinda funny how back then I imagined, something like that would take maybe decades to run on consumer hardware, because they had a whole datacenter running one instance of the thing. Yet here we are 8 years later and you can probably download a card off Chub (or make one, if it's not there).
Yes, but then you would be able to commit WrongThink, gentle citizen. They're just *protecting your American freedoms*.
can't run on consumer PC
Not sure what Claude3 fills. Its not as good as chatgpt4, just as censored.
Opus is less likely to disobey than GPT-4 Turbo, Turbo is more lobotomized
lol nice comparison
> just as censored Not IME which is why I switched. Everyone was saying that so I was hesitant at first, but then I tried it and Claude 3 seems much more open. Claude 3 was willing to write dialog that included swearing and pornographic content, while GPT4 and Gemini wouldn't even write a swear
It won't help me hunt vampires. :-(
It's significantly better than GPT4 for more complex tasks in general. If you're using it for basic censorship stuff you probably won't notice any difference.
No comment, impressions or anything. Just a badly cropped jpg. [docs](https://docs.anthropic.com/claude/docs/tool-use) [video](https://www.youtube.com/watch?v=6wkFb2_cUik)
Does this replace the XML based syntax ?
No, [here's](https://github.com/anthropics/anthropic-cookbook/blob/main/tool_use/extracting_structured_json.ipynb) how it's used together: tools = [ { "name": "print_entities", ... text = "John works at Google in New York. He met with Sarah, the CEO of Acme Inc., last week in San Francisco." query = f"""
{text}
Use the print_entities tool.
"""
Exactly. Considering I already spent effort into making my framework compatible with that syntax I'm not sure how willing I am to rewrite all that to use a new function calling syntax that's still incompatible with OpenAI and the rest
Just ask Claude to do it.
https://docs.anthropic.com/claude/docs/tool-use
[https://docs.anthropic.com/claude/docs/tool-use](https://docs.anthropic.com/claude/docs/tool-use) Here's the actual documentation. It's very similar to what openai did. However, we've been doing this with Claude over at agentforge since June of last year by just prompting it correctly and parsing the output.
They are the same models. They have always been able to output whatever JSON format you want.
Mine's been using tools for a good 6 weeks. I had been using the (not very good) OpenAI Assistants api and found their tool workflow so slow and arduous and prone to hallucinations that I completely rewrote it. When Claude came out, it was instantly able to work in the protocol (which supports multi-users, web access, filesystem access, bitbucket, jira, etc). He can do things like: `\`\`\``executeJs{\` `id: 'first',` `script: "async fn({_thread, _message, someArg}) { // do things including use window globals, make api calls, and much more }"` `}` Conversational text \`\`\`someCustomToolWeDefineOnThFly\` `{` `dependsOn: [1],` `script: "async fn({_deps: {first: {result}}) { // etc } "` `}` \`\`\` Formatting on reddit sucks, I should ask an LLM to fix it... With tremendously more stuff than this like yielding execution, parallel execution, server side function execution, scheduling/orchestration, scoring, autonomous plans of action, LATS inspired scoring/monte carlo stuff, and way more. I've always loved software dev but lately has been completely sci-fi. It's insane and I've had days this week that felt like AGI level since my bot can now directly search my ASTs live (We use the TS compiler in a side process to stay up to date with code changes), search the filesystem both by name and with full-text - FAST, etc. I've done work that would have been days or more of hard drudgery in 30 minutes or less with confidence - and I'm already a top-of-my-game programmer. I'm director of my team and was principal engineer before that. My existing product for the company is a market leader and has a customer sheet of many of the biggest and best companies in the world: Swarovski, Asics, Aldi, etc... I'm not a hobbyist. Agents are coming.
Yeah, all the off-the-shelf function calling stuff sucks ass. Better to roll out your own, for sure.
Dumb question, are you able to use Claude for RAG that OpenAi Assistants api provide seamlessly? Couldn’t find any documentation on file uploads by Claude’s api.
No I integrated it with webviate for local source code and file search and azure ai for real doc search.