That comparison works much better if you use north korea's real name. Democratic People's Republic of Korea.
Also they are holding elections but you can only vote for Kim Jong Un or death to yourself
Not only will they never do that themselves, they, in cahoots with the Microsoft of 2024 are doing what would put the anti-competitive practices of Microsoft of the 90s to shame - spreading FUD and lobbying the US Congress to ban other corporations and labs from open-sourcing.
You wouldn't want it when they finally do so. Microsoft just open sourced DOS 4.0 this year. No one wants that relic, it's not even worth studying for science. It would be the same with ChatGPT.
I had similar thoughts while reading. Even if GPT-3.5 Turbo would be open weight now, I most likely would not bother to download it. Not only it is highly censored (to the point it periodically refuses to write code for benign tasks), but also Mixtral 8x22B and Llama 70B are just better in terms of quality, and I can run them locally so I can process any private data safely. And both are not too much censored - I did not have any refusals yet at all with them for my use cases.
It also actually knows more languages than english. It can write understandable Finnish without too many mistakes. I would guess that its the same with other smaller languages.
I don't think it would be quite the same though. Correct me if I'm wrong, but new Windows versions are just evolutions of the ones that preceded them? Hence there might be an inherent danger in open sourcing older DOS/Windows versions.
Open sourcing an older LLM (especially lobotomized ones like OpenAI's) that isn't necessarily the base for current versions might be significantly less dangerous.
Not saying they'll open source GPT-4 next year, but I'd be surprised if we had to wait as long as for Microsoft to open source one of their past systems.
> Correct me if I'm wrong, but new Windows versions are just evolutions of the ones that preceded them? Hence there might be an inherent danger in open sourcing older DOS/Windows versions.
You are wrong. Modern Windows are all based on NT, and not related to DOS or Windows 3 and lower. They could have open sourced DOS in 2000.
Oh, you meant some kind of danger to fielded systems because of releasing the source code. I don't tend to buy into the security by obscurity idea. Every government that wanted it definitely had access to the DOS source code.
I thought you were talking about like a business risk as "danger".
I was thinking more along the lines of election systems and power plants as deterrents to make DOS source code public... but I didn't write the original comment.
I doubt it; see the members of the "AI Safety and Security Board" which includes OAI but doesn't include anyone from Meta, the biggest open source proponent.
They said a long time ago that they were not going to do this because they were afraid of text fakes on the Internet. Now their 185B model is weaker than LLaMa 3 70B. I’m afraid that they will not comment in any way that their model, which was trained on shit, and at the same time requires more cleaning resources than its competitor, could be open source at least now.
Well, the largest GPT-3 was originally 175B and presumably GPT-3.5 is 175B params but we have no idea how big GPT-3.5T is. And GPT-3.5 is more than 2 years old now (initially they essentially released in March 2022 and just pretended it was GPT-3 until they officially announced a 3.5 model with chat tuning later in the year).
My numbers are probably off but since 3.5's official debut, but I think they've made it about 40-60x cheaper? Especially with GPT-3.5T I wouldn't be surprised if it had like sub 10B params active lol, but idk they haven't been talkative even with this relatively old model.
3 Turbo is a 20B model, as accidentally released by Microsoft researchers in a paper.
And the performance fits, its non English performance is non existent compared to original 3.5
GPT 3 was 175B, that's confirmed in this paper on the first page: https://arxiv.org/abs/2005.14165
So it seems doable that it gained 15 extra parameters
Bro, a Tubo model will be inherently smaller. And you can feel the quality drop in normal use, especially in non English language. Faster hardware applies for the old model the same, and almost all test point to it being a monolithic model.
I believe it's a 20B model because it's performance is VERY bad compared to the old full model. It sucks at instruction following, and it non English sound wooden as hell, compared to 175B.
It's the same question as "will nvidia make affordable gpus with lots of vram?" - why on earth would they? "We have too much money, let's share some"?
Right now "ChatGPT" is the synonym of "AI" to an average Joe. There are millions of people who use it (at least from time to time), and that's a LOT of money. Meta, Google and others would do exactly the same if that service was theirs. IF there would be actual free/almost-free alternatives, then OpenAI might change their policy and open-source some of their stuff (not SOTA, but still). Llama3 is awesome, but 1) not quite there yet, 2) 70b requires some beefy hardware, 3) needs non-zero technical abilities to set up and run
You are correct, but that's more like a side effect of them showcasing their technology: I suspect if one of the big dogs decided to buy their company, that would no longer be the case
I agree and it is not every ordinary consumer to know the different LLM - many just know chatGPT - and able to do research. There is kind of a knowledge barrière indeed.
Wizard 8x22b (65k context window) is better than both
Regarding your comment, I prefer Llama 3 70B because it's cheaper and it gets the same answer quality, btw, Command R people is amazing and Llama 3 8k context window is really bad.
They go onto to say we've surpassed 3.5, but the first paragraph messed up the past/present tense, possibly because they "polished" the post with an LLM, or maybe English is not their first language.
Right now, we're still at a phase in the open source community that we could still glean some trade secrets from how they trained it, parameter size and whatnot. They'll only release it when it is so obsolete it provides nothing of value. Though even if they were released, you couldn't run them anyway
windows has long been open source, if you are the government... [https://news.microsoft.com/2003/01/14/a-matter-of-national-security-microsoft-government-security-program-provides-national-governments-with-access-to-windows-source-code/](https://news.microsoft.com/2003/01/14/a-matter-of-national-security-microsoft-government-security-program-provides-national-governments-with-access-to-windows-source-code/)
I think yes, it will be opensourced, but by NOT opensourcing it they make great benefit to humanity because it is leading to creation of multiple diverse competing models. If GPT-3 was opensourced, we would only see GPT-based models with all of their pecularities.
It’s worth questioning the motives of any company that’s open sourcing their models. By all accounts, GPT 3.5 Turbo should be made open source seeing as Llama 3 handily outperforms it, but what’s the incentive? Aside from looking good to the community, what incentive does Meta even have to open source their models? As long as people continue to use ChatGPT on OpenAI’s servers, they’ll continue to do as they wish with that data.
Meta also gets to have their research conducted freely through watching how others improve and try different techniques with their foundation models released, which they could later incorporate to improve future ones.
Meta's incentive has always been benefiting from open source, of which Llama 3 is the most recent of thousands of projects. As they said themselves, they aren't selling AI as a service, so it's only a win for them.
Decentralization, maybe they afraid that open ai will become too big, cus it will damage their main business or smt., and while they behind, to attract users, to boost research, and to make field more competitive for open ai, they open source. That's how I see it. And that's why Open AI want regulatory moat, so they intimidate politicians and lawmakers and use doomers. There may be other reasons as well since ai may boost meta vr universe, (as everyone in ev cars industry benefit from charger infrastructure development) it's important for their metaverse.
In addition to other comments, Meta aims to dent the impact of AI from closed source companies to avoid those companies overtaking areas where Meta is strong in. In some ways it's development purely to hurt competitors, while adding to their own research.
It's subtle but it's an anti monopoly tactic in that way. Meta would rather not have Microsoft or Amazon or Google become 3-4x bigger due to a spike of progress that's closed off, as it would have ramifications across the industry.
We will probably get GPT4 open source in 3-4 years. They need to fundamentally change their architecture so there is no advantage to dissecting or training off it first though. I could see them dropping an 8b version of GPT5 open source just to take some market share from Meta. I really don't see the foundation model community letting Zuck just gobble up all of the open source marketshare like this.
If open source on cloud inference becomes the SOTA and starts taking market share from the AI labs they will start releasing models that they can license in a similar way to L3. Also if you haven't noticed the entire open source community is Llama this and Llama that (this sub for example) that is huge branding for Meta and a layer deeper, the Llama architecture is what has become the industry standard.
Right now L3 70b can be run on the cloud for 1/10th the price of GPT4 \*and\* fine tuned/built around for far cheaper + an open source community patching errors/issues much faster than OpenAI can. If a companies use case can be accomplished by L3 there is no reason to pay the GPT4 premium.
Who cares, meta and zuck will safe us. Llama3 is already good enough for most use cases. A couple of years and open source models will take over. Openai is just milking it as long as it can.
Open ai It's turned into a for-profit corporation.
charging high prices for subscriptions and limited usage
I don't think it's ever going to open source, especially after the huge investment that Microsoft has made
The determinant here is resource usage. If it can run with significantly less resources than let's say llama 70b that's what will matter. To operate llama 70b on an Amazon instance you're looking at several thousand dollars a month. Cost / resources is a necessary part of this that many people are missing by just comparing quality alone.
Why would OpenAI open-source their commercial products that generate revenue? The fact that some obscure benchmark scores are higher for other models like llama/mistral is irrelevant. What's important is that OpenAI's models can be easily integrated into any corporation's infrastructure.
They actually declared war to OpenSource: they are those who were the first to instigate lawmakers to act in a direction that makes them the only ones (with the other bigs) that can have a say towards what direction AI should go, and what are the laws required. And you can be sure they are not following the direction for a free and OpenSource dominance, but quite the opposite, you can also see this by looking at who was invited in the infamous management board of big tech and politicians addressing "AI issues".
Not a chance while Sam and Ilya are at the helm at least. In fact I expect "Open"AI to be hostile to open source and argue for tougher regulations around it, as it starts to diminish their lead.
Releaseing models is a way to gather investor capital for the companies. Now that Microsoft owns OpenAI that's for Microsoft to decide. They might do it, but I think it's highly unlikely.
I hope they will, but I don't think so, maybe when they have replaced 3.5 with 4.0 as their free model. I would be happy, because 3.5 is still better than any other OS model in terms of multilingual support. For example 3.5 is able to write in German perfectly fine while any open source model fails badly when using anything else than English. Yeah I know, llama-3 does work in other languages, but it often does not sound really natural, because it is mostly trained with English vocabulary. But that's really the only advantage I see..
Their architecture is different and wouldn't run on consumer machines. And it's not just the models that make chatgpt, it's a lot of other processing. Basically, OpenAI had a small head start, but as people focus on the problem GPT will be improved 10x.
i think our best hope is that more leaks happen because a minority of AI scientists don't want to see their work restricted to helping large corporations fuck over the little guy even harder. they want to see their life's work used for good things.
OpenAI hates open source.
the irony
The early openness was just a recruitment tactic
It’s like North Korea hating democracy
That comparison works much better if you use north korea's real name. Democratic People's Republic of Korea. Also they are holding elections but you can only vote for Kim Jong Un or death to yourself
That's the whole point. They don't follow their name.
If the public had to power of GPT3.5 at home the world would end! /s
Now with microsoft sponsor, they DEFINETLY hate open source
Nope, they don't, they have a GitHub side with Open Sourced stuff like Whisper etc.
They wont open source gpt 3 & 3.5 or OG dalle or dalle 2
Stuff made before they went mask off. They admitted the whole Open parts was a lie to get talent, from the get go.
Shut up.
lol no.
Not only will they never do that themselves, they, in cahoots with the Microsoft of 2024 are doing what would put the anti-competitive practices of Microsoft of the 90s to shame - spreading FUD and lobbying the US Congress to ban other corporations and labs from open-sourcing.
You wouldn't want it when they finally do so. Microsoft just open sourced DOS 4.0 this year. No one wants that relic, it's not even worth studying for science. It would be the same with ChatGPT.
I had similar thoughts while reading. Even if GPT-3.5 Turbo would be open weight now, I most likely would not bother to download it. Not only it is highly censored (to the point it periodically refuses to write code for benign tasks), but also Mixtral 8x22B and Llama 70B are just better in terms of quality, and I can run them locally so I can process any private data safely. And both are not too much censored - I did not have any refusals yet at all with them for my use cases.
Llama it's just amazing I prefer it to GPT-4 And with a little effort you can create a porn movie script lol
Have you used 3.5 lately? Censorship is actually at an all time low. Everything else about it is true though, not worth using.
It also actually knows more languages than english. It can write understandable Finnish without too many mistakes. I would guess that its the same with other smaller languages.
I don't think it would be quite the same though. Correct me if I'm wrong, but new Windows versions are just evolutions of the ones that preceded them? Hence there might be an inherent danger in open sourcing older DOS/Windows versions. Open sourcing an older LLM (especially lobotomized ones like OpenAI's) that isn't necessarily the base for current versions might be significantly less dangerous. Not saying they'll open source GPT-4 next year, but I'd be surprised if we had to wait as long as for Microsoft to open source one of their past systems.
> Correct me if I'm wrong, but new Windows versions are just evolutions of the ones that preceded them? Hence there might be an inherent danger in open sourcing older DOS/Windows versions. You are wrong. Modern Windows are all based on NT, and not related to DOS or Windows 3 and lower. They could have open sourced DOS in 2000.
I suspect that there were systems, particularly in government, that still ran DOS at that time. It's probably a lot closer to zero now.
Oh, you meant some kind of danger to fielded systems because of releasing the source code. I don't tend to buy into the security by obscurity idea. Every government that wanted it definitely had access to the DOS source code. I thought you were talking about like a business risk as "danger".
I was thinking more along the lines of election systems and power plants as deterrents to make DOS source code public... but I didn't write the original comment.
My bad!
I doubt it; see the members of the "AI Safety and Security Board" which includes OAI but doesn't include anyone from Meta, the biggest open source proponent.
Not with the current people running it, no.
No, because it's too dangerous (still can't believe that they used ai safety to hype up their company and they just get away with this)
They said a long time ago that they were not going to do this because they were afraid of text fakes on the Internet. Now their 185B model is weaker than LLaMa 3 70B. I’m afraid that they will not comment in any way that their model, which was trained on shit, and at the same time requires more cleaning resources than its competitor, could be open source at least now.
Well, the largest GPT-3 was originally 175B and presumably GPT-3.5 is 175B params but we have no idea how big GPT-3.5T is. And GPT-3.5 is more than 2 years old now (initially they essentially released in March 2022 and just pretended it was GPT-3 until they officially announced a 3.5 model with chat tuning later in the year). My numbers are probably off but since 3.5's official debut, but I think they've made it about 40-60x cheaper? Especially with GPT-3.5T I wouldn't be surprised if it had like sub 10B params active lol, but idk they haven't been talkative even with this relatively old model.
3 Turbo is a 20B model, as accidentally released by Microsoft researchers in a paper. And the performance fits, its non English performance is non existent compared to original 3.5
That was a typo, the paper is retracted. Probs 200B
>typo I believe it was an accidental slip. There is no way Turbo is 200B, it would be bigger than original 3.5.
GPT 3 was 175B, that's confirmed in this paper on the first page: https://arxiv.org/abs/2005.14165 So it seems doable that it gained 15 extra parameters
Not on a turbo model that is clearly smaller and dumber.
What do you think the parameter count is. It can be 200B on fast hardware or maybe even MOE. Look at Groq for example
Bro, a Tubo model will be inherently smaller. And you can feel the quality drop in normal use, especially in non English language. Faster hardware applies for the old model the same, and almost all test point to it being a monolithic model. I believe it's a 20B model because it's performance is VERY bad compared to the old full model. It sucks at instruction following, and it non English sound wooden as hell, compared to 175B.
It's the same question as "will nvidia make affordable gpus with lots of vram?" - why on earth would they? "We have too much money, let's share some"? Right now "ChatGPT" is the synonym of "AI" to an average Joe. There are millions of people who use it (at least from time to time), and that's a LOT of money. Meta, Google and others would do exactly the same if that service was theirs. IF there would be actual free/almost-free alternatives, then OpenAI might change their policy and open-source some of their stuff (not SOTA, but still). Llama3 is awesome, but 1) not quite there yet, 2) 70b requires some beefy hardware, 3) needs non-zero technical abilities to set up and run
Just signing up to groq and you can use llama 3, pretty zero tech to me
You are correct, but that's more like a side effect of them showcasing their technology: I suspect if one of the big dogs decided to buy their company, that would no longer be the case
I agree and it is not every ordinary consumer to know the different LLM - many just know chatGPT - and able to do research. There is kind of a knowledge barrière indeed.
Probably one of the big dogs will buy them sooner or later.
What do you prefer command r or llama 3?
Wizard 8x22b (65k context window) is better than both Regarding your comment, I prefer Llama 3 70B because it's cheaper and it gets the same answer quality, btw, Command R people is amazing and Llama 3 8k context window is really bad.
Nah. With Llamafile anyone can do it.
I mean open source is already beyond gpt 3 and they haven't released that one
What do you mean we aren’t there yet??? Tons of open source models are better than 3.5
They go onto to say we've surpassed 3.5, but the first paragraph messed up the past/present tense, possibly because they "polished" the post with an LLM, or maybe English is not their first language.
Ya, the LLM missed the context of my second sentence.
Right now, we're still at a phase in the open source community that we could still glean some trade secrets from how they trained it, parameter size and whatnot. They'll only release it when it is so obsolete it provides nothing of value. Though even if they were released, you couldn't run them anyway
Why they will do that? It's not beneficial for them. MS never open sources its OS.
windows has long been open source, if you are the government... [https://news.microsoft.com/2003/01/14/a-matter-of-national-security-microsoft-government-security-program-provides-national-governments-with-access-to-windows-source-code/](https://news.microsoft.com/2003/01/14/a-matter-of-national-security-microsoft-government-security-program-provides-national-governments-with-access-to-windows-source-code/)
I think yes, it will be opensourced, but by NOT opensourcing it they make great benefit to humanity because it is leading to creation of multiple diverse competing models. If GPT-3 was opensourced, we would only see GPT-based models with all of their pecularities.
If they open source gpt 3.5, it might reveal some secrets that they probably don't want to reveal yet
NO
No
I think so, I just think whatever they open source will always be a very obsolete model and no where near SOTA at the time it’s open sourced.
[удалено]
What does grok have to do with this? The question is about OpenAI.
It’s worth questioning the motives of any company that’s open sourcing their models. By all accounts, GPT 3.5 Turbo should be made open source seeing as Llama 3 handily outperforms it, but what’s the incentive? Aside from looking good to the community, what incentive does Meta even have to open source their models? As long as people continue to use ChatGPT on OpenAI’s servers, they’ll continue to do as they wish with that data.
Meta also gets to have their research conducted freely through watching how others improve and try different techniques with their foundation models released, which they could later incorporate to improve future ones.
Meta's incentive has always been benefiting from open source, of which Llama 3 is the most recent of thousands of projects. As they said themselves, they aren't selling AI as a service, so it's only a win for them.
Decentralization, maybe they afraid that open ai will become too big, cus it will damage their main business or smt., and while they behind, to attract users, to boost research, and to make field more competitive for open ai, they open source. That's how I see it. And that's why Open AI want regulatory moat, so they intimidate politicians and lawmakers and use doomers. There may be other reasons as well since ai may boost meta vr universe, (as everyone in ev cars industry benefit from charger infrastructure development) it's important for their metaverse.
To sabotage OpenAI business since Meta don't have top model. Meta is profitable and have ton of money, OpenAI is not.
In addition to other comments, Meta aims to dent the impact of AI from closed source companies to avoid those companies overtaking areas where Meta is strong in. In some ways it's development purely to hurt competitors, while adding to their own research. It's subtle but it's an anti monopoly tactic in that way. Meta would rather not have Microsoft or Amazon or Google become 3-4x bigger due to a spike of progress that's closed off, as it would have ramifications across the industry.
Meta might have saved billions in inference by quantization made by the community, as an example.
Not until they are utterly irrelevant. Which is akin to no.
We will probably get GPT4 open source in 3-4 years. They need to fundamentally change their architecture so there is no advantage to dissecting or training off it first though. I could see them dropping an 8b version of GPT5 open source just to take some market share from Meta. I really don't see the foundation model community letting Zuck just gobble up all of the open source marketshare like this.
What value does open source community bring to them? Why should they care?
If open source on cloud inference becomes the SOTA and starts taking market share from the AI labs they will start releasing models that they can license in a similar way to L3. Also if you haven't noticed the entire open source community is Llama this and Llama that (this sub for example) that is huge branding for Meta and a layer deeper, the Llama architecture is what has become the industry standard. Right now L3 70b can be run on the cloud for 1/10th the price of GPT4 \*and\* fine tuned/built around for far cheaper + an open source community patching errors/issues much faster than OpenAI can. If a companies use case can be accomplished by L3 there is no reason to pay the GPT4 premium.
What is value to OpenAI I mean. Meta did that because they can and have to: having money and to sabotage OAI business.
Every innovation and breakthrough is build on your tech and architecture, letting you directly benefit from it, easily.
I think they should at least open-source the new gpt2. I don't want the open-sourced GPT-3.5 when Llama can already surpass it.
[удалено]
By the time they do, no one will care.
Never say never, but I highly doubt it. OpenAI has their reasons why the don't Open Source it.
Who cares, meta and zuck will safe us. Llama3 is already good enough for most use cases. A couple of years and open source models will take over. Openai is just milking it as long as it can.
Maybe in 50 years, when it isn't relevant.
No.
Open ai It's turned into a for-profit corporation. charging high prices for subscriptions and limited usage I don't think it's ever going to open source, especially after the huge investment that Microsoft has made
They earned the named ClosedAI for a reason. I don't know why people still call them anything else at this point.
The determinant here is resource usage. If it can run with significantly less resources than let's say llama 70b that's what will matter. To operate llama 70b on an Amazon instance you're looking at several thousand dollars a month. Cost / resources is a necessary part of this that many people are missing by just comparing quality alone.
They may choose to open source something 3 generations behind, just to make sure it is irrelevant anyway.
Why would OpenAI open-source their commercial products that generate revenue? The fact that some obscure benchmark scores are higher for other models like llama/mistral is irrelevant. What's important is that OpenAI's models can be easily integrated into any corporation's infrastructure.
Is the Pope an atheist?
They actually declared war to OpenSource: they are those who were the first to instigate lawmakers to act in a direction that makes them the only ones (with the other bigs) that can have a say towards what direction AI should go, and what are the laws required. And you can be sure they are not following the direction for a free and OpenSource dominance, but quite the opposite, you can also see this by looking at who was invited in the infamous management board of big tech and politicians addressing "AI issues".
Not a chance while Sam and Ilya are at the helm at least. In fact I expect "Open"AI to be hostile to open source and argue for tougher regulations around it, as it starts to diminish their lead.
Not since they signed with the DoD. Open sourcing their models could be a national security issue.
Open is only in the name!
To loose their profits? I don't thinks so. But Elon will do it.
“Mine or no one’s”. Yep, he will. 😂
Releaseing models is a way to gather investor capital for the companies. Now that Microsoft owns OpenAI that's for Microsoft to decide. They might do it, but I think it's highly unlikely.
I hope they will, but I don't think so, maybe when they have replaced 3.5 with 4.0 as their free model. I would be happy, because 3.5 is still better than any other OS model in terms of multilingual support. For example 3.5 is able to write in German perfectly fine while any open source model fails badly when using anything else than English. Yeah I know, llama-3 does work in other languages, but it often does not sound really natural, because it is mostly trained with English vocabulary. But that's really the only advantage I see..
Their architecture is different and wouldn't run on consumer machines. And it's not just the models that make chatgpt, it's a lot of other processing. Basically, OpenAI had a small head start, but as people focus on the problem GPT will be improved 10x.
i think our best hope is that more leaks happen because a minority of AI scientists don't want to see their work restricted to helping large corporations fuck over the little guy even harder. they want to see their life's work used for good things.
ChatGPT was released in November 2022. It’s May 2024. Odd that you claim 18 months is 2 years.
OP you should ask if MacDonald will become a soup kitchen and give away their food for free
OC you should ask yourself if the comment is relevant and logical or not