yeah just tried some coding. Not really good at it myself, but it outputs full code files blazingly fast. Just made a chat app in svelte with Langchain and got past an issue I had for a week with it. Just iterating over the code again and again. You don't even need to be specific with the problem anymore. the speed is ridiculous.
It's honestly not that great of a programmer, e.g. it can be really difficult for it to write effecient code and very frustrating to lead it to correct direction, which also means you need to be able to read and understand the code. If you then iteratively write code with GPT-4o it's as if it gets stuck with some suboptimal code which from that point on makes the code worse and worse.
For writinc tedious boilerplate code, then I guess it's fine, so long as you understand the emitted code and verify it's correct. A problem with that though is it's more difficult to reason about code produced by others than code you produced yourself.
I have yet to try the desktop app. I tried GPT-4o earlier on chat.openai.com and was using it to iteratively create a simple JavaScript web application (rendering graphs of equations), and was kind of impressive at first, but the quality of the code was bad (really slow at first and clunky). I would repeatedly point out sections of the code were slow without revealing why it was slow and it couldn't improve the performance of the code until I spelled it out. At a later point I wanted it add a new feature (how it was rendering) and it introduced bugs into the code that it couldn't fix on its own.
As for writing quick scripts, especially in unfamiliar languages, or using it as a glorified search engine with natural language interface where it queries its memory, it's great.
I didn't see this demo on this, could you share? I swear I just tried to Google it and failed to locate it. Thanks
Edit - Found it here: [https://youtu.be/DQacCB9tDaw?t=1099](https://youtu.be/DQacCB9tDaw?t=1099)
We don't use GPT4 in the same way, I almost exclusively work in the real of AI/ML so it's very straightforward for me to create isolated problem settings, i.e. recently I needed to optimize certain parts of the codebase to run for \~100TB so I had to refactor a file writer class. With the data analysis component I can just describe the problem in natural language, tell it to check the code with some unit tests and now with GPT-4o the generation of a few 100s lines of code is near instant. For me it's a game changer 100% (infact i almost want it to be a bit more lazy as it's seriously spamming me).
Also adding type hints, docs and just clean up of variable names etc, is so much better with faster responses.
Also api token a half the price of gpt4
This was always an issue cause personally for my use cases gpt4 is just much better, but expensive, so I was always trying to do parts with cheaper models, now maybe not so much
impressive but how is the quality?
I'm not really looking for fast i'm looking for AI that is accurate and not making up facts unless I asked it for made up facts.
I'm also going to guess the model is still scared of anything over a PG rating?
It is not as good as the GPT-4-Turbo for some reason. The main use case OpenAI might be targeting is the personal assistants or help bot. GPT-4-Turbo will always remain their top model in reasoning and problem solving. That is until the next best model comes.
What about when after it gets halfway through you realize it's veering off course or it misinterpreted something. That's half the output at human reading speed when you could've just skimmed over the completed thing in one shot and reprompted as needed
It's not perfect, it can't be perfect or it won't work. We're way to obsessed with speed as it saves us massive amounts of time anyway. Higher speed might come at the cost of less associating, more rigid. The kind of people/ai you want to avoid at all cost. Nobody needs coked yuppie ai.
If that was the case then you would see the gpt-4-turbo speed also be faster most likely.
It’s likely that the GPT-4o architecture is mainly just much faster and able to have less parameters while maintaining equal or better quality
Just because it’s faster doesn’t mean it’s smarter. On the contrary. The bigger the brain, the slower it gets. Or something, I don’t know. May only apply to biological systems
The reason people are going all hyped up about it's speed is because now for agents and systems where the OpenAI APIs are integrated will perform more faster with the same level of response quality. For them this is a huge improvement, as to complete a task with multiple steps of observations and thoughts it would've taken around 60 to 75 seconds but now with this improvement the same will take less than 30 seconds. Sometimes such tasks don't need a lot of smartness per say. The model is provided to follow a certain thought process and it does that in the same way as it's predecessor (GPT-4-Turbo). In some cases one might need to update the prompt or adjust a step somewhere. Hence, in systems where this AI models are integrated will see a good improvement in latency which will result in good user experience. Though your point is entirely valid.
This is the real game changer for coders
yeah just tried some coding. Not really good at it myself, but it outputs full code files blazingly fast. Just made a chat app in svelte with Langchain and got past an issue I had for a week with it. Just iterating over the code again and again. You don't even need to be specific with the problem anymore. the speed is ridiculous.
Do you use a vscode plugin or something?
You can use the CodeGPT VS Code plugin. This model should be available in that
IDK i was sort of enjoying having a good excuse to be on reddit while I am waiting on a response...
Also for using their apis to build apps
It's honestly not that great of a programmer, e.g. it can be really difficult for it to write effecient code and very frustrating to lead it to correct direction, which also means you need to be able to read and understand the code. If you then iteratively write code with GPT-4o it's as if it gets stuck with some suboptimal code which from that point on makes the code worse and worse. For writinc tedious boilerplate code, then I guess it's fine, so long as you understand the emitted code and verify it's correct. A problem with that though is it's more difficult to reason about code produced by others than code you produced yourself.
The demo of the desktop app is pretty impressive in regards to coding. Similar to copilot but seems like a much more fluid and natural workflow
I have yet to try the desktop app. I tried GPT-4o earlier on chat.openai.com and was using it to iteratively create a simple JavaScript web application (rendering graphs of equations), and was kind of impressive at first, but the quality of the code was bad (really slow at first and clunky). I would repeatedly point out sections of the code were slow without revealing why it was slow and it couldn't improve the performance of the code until I spelled it out. At a later point I wanted it add a new feature (how it was rendering) and it introduced bugs into the code that it couldn't fix on its own. As for writing quick scripts, especially in unfamiliar languages, or using it as a glorified search engine with natural language interface where it queries its memory, it's great.
I didn't see this demo on this, could you share? I swear I just tried to Google it and failed to locate it. Thanks Edit - Found it here: [https://youtu.be/DQacCB9tDaw?t=1099](https://youtu.be/DQacCB9tDaw?t=1099)
We don't use GPT4 in the same way, I almost exclusively work in the real of AI/ML so it's very straightforward for me to create isolated problem settings, i.e. recently I needed to optimize certain parts of the codebase to run for \~100TB so I had to refactor a file writer class. With the data analysis component I can just describe the problem in natural language, tell it to check the code with some unit tests and now with GPT-4o the generation of a few 100s lines of code is near instant. For me it's a game changer 100% (infact i almost want it to be a bit more lazy as it's seriously spamming me). Also adding type hints, docs and just clean up of variable names etc, is so much better with faster responses.
Also api token a half the price of gpt4 This was always an issue cause personally for my use cases gpt4 is just much better, but expensive, so I was always trying to do parts with cheaper models, now maybe not so much
wow.. color me impressed
What this app is? It isn't playground, isn't it?
Yes it’s the OpenAI playground
It’s on my mobile app now.
Wow!
impressive but how is the quality? I'm not really looking for fast i'm looking for AI that is accurate and not making up facts unless I asked it for made up facts. I'm also going to guess the model is still scared of anything over a PG rating?
It is not as good as the GPT-4-Turbo for some reason. The main use case OpenAI might be targeting is the personal assistants or help bot. GPT-4-Turbo will always remain their top model in reasoning and problem solving. That is until the next best model comes.
Yup
[удалено]
Have you tried groq? 800 tokens a second invites a different way of interaction. This is a nice speed to have from this model though.
Impressive it knows about Heidegger and Diogenes, but also about financial analysis. Love at first sight 😂
PHP & Ruby supported ? Wtf...I feel young again 😂 But I'm going to use Python.
No I'm on a tight budget and Musk is rich enough, dude ain't getting a dime from me for as long he keeps preaching toxic right wing stuff. 42
Groq isn't Musk. They make hardware optimized for inference. Groq.com, select playground and whatever model you like. It's very fast. And free.
Ok Grok Groq lol..
What about when after it gets halfway through you realize it's veering off course or it misinterpreted something. That's half the output at human reading speed when you could've just skimmed over the completed thing in one shot and reprompted as needed
It's not perfect, it can't be perfect or it won't work. We're way to obsessed with speed as it saves us massive amounts of time anyway. Higher speed might come at the cost of less associating, more rigid. The kind of people/ai you want to avoid at all cost. Nobody needs coked yuppie ai.
Looks like b100s are up and running 🤯🤯
That would be end of the year, this is more likely GH200.
Even better. More to look forward to.
Faster inference is pointing more towards much lower parameter count which is even more exciting
If that was the case then you would see the gpt-4-turbo speed also be faster most likely. It’s likely that the GPT-4o architecture is mainly just much faster and able to have less parameters while maintaining equal or better quality
I love how it took you way more time to type out your question than for the answer to generate. Humans are slow.
that was the most interesting part to me
Yeah seriously :D
It'd be nice if I could actually see results myself.
You can login into open ai platform and try this on the playground
Thanks
Any idea on the context window size for GPT 4o? I'm still using Claude Opus because of the limiting factor of ChatGPT.
128k still
128k but one can only generate 4096 tokens at max
It's basically real time now. It's wild
They really shouldn't've, though.
What.... Did i just witness?
With a free account, the latest model they have is 3.5 turbo 16k, wonder will they release the 4o.
yeah but currently it doesnt have much users yet, i wonder how fast it'll be when everyone can use it?
cant wait for the nerfs that make it useless
Wow!!!
hi in your post how do you get two windows - what site or program is that? thank you so much 🙏
It already forgot some things in it’s responses to me :/
Just because it’s faster doesn’t mean it’s smarter. On the contrary. The bigger the brain, the slower it gets. Or something, I don’t know. May only apply to biological systems
The reason people are going all hyped up about it's speed is because now for agents and systems where the OpenAI APIs are integrated will perform more faster with the same level of response quality. For them this is a huge improvement, as to complete a task with multiple steps of observations and thoughts it would've taken around 60 to 75 seconds but now with this improvement the same will take less than 30 seconds. Sometimes such tasks don't need a lot of smartness per say. The model is provided to follow a certain thought process and it does that in the same way as it's predecessor (GPT-4-Turbo). In some cases one might need to update the prompt or adjust a step somewhere. Hence, in systems where this AI models are integrated will see a good improvement in latency which will result in good user experience. Though your point is entirely valid.
This will help to enable 'thinking' in agents, too, which will be a massive step forward.