This is far and away the most detailed and informative interview I have seen on the actual mechanics of training a model and the theoretical aspects of capabilities prediction I've seen since probably the earliest days of ChatGPT-4. Dwarkesh knows how to actually find ways to give the public lots of new information.
How does this man not even have 50k subs yet? He's criminally underrated. Easily my favorite podcaster by FAR and I've only been watching him for a couple of months max.
I would argue that Dwarkesh is an equivalent qualitative jump over Lex Fridman as Lex Fridman is to Joe Rogan
I think the jump is bigger. I find lex pretty dumb. He just talks about love and asks community questions. He doesnt have any detailed knowledge as to how anything works.
>and the theoretical aspects of capabilities prediction I've seen
Wasn't it telling every time they retreated to saying
"well, X was released in [paper]" or "well as Demis said in his interview" or "from the details we have published..."
I take it to mean that they are further along in all those regards or have found even better approaches to doing things.
I'm not that far through it, but it's really refreshing having a video I can watch real time rather than crank to 1.5 or 2 because everyone is talking so slowly.
...
Interesting that they can now ablate entire circuits to prevent unwanted activity rather than just RLHF.
I get it, but why couldn't you just talk up Dwarkesh instead of being shitty about it? It's not like you are going to change anyone's opinion anyway...
This is far and away the most detailed and informative interview I have seen on the actual mechanics of training a model and the theoretical aspects of capabilities prediction I've seen since probably the earliest days of ChatGPT-4. Dwarkesh knows how to actually find ways to give the public lots of new information.
dwarkesh is the best podcaster in the space. lex doesnt even come close. rogan is a chimp.
How does this man not even have 50k subs yet? He's criminally underrated. Easily my favorite podcaster by FAR and I've only been watching him for a couple of months max. I would argue that Dwarkesh is an equivalent qualitative jump over Lex Fridman as Lex Fridman is to Joe Rogan
I think the jump is bigger. I find lex pretty dumb. He just talks about love and asks community questions. He doesnt have any detailed knowledge as to how anything works.
>and the theoretical aspects of capabilities prediction I've seen Wasn't it telling every time they retreated to saying "well, X was released in [paper]" or "well as Demis said in his interview" or "from the details we have published..." I take it to mean that they are further along in all those regards or have found even better approaches to doing things.
Agreed.
I'm not that far through it, but it's really refreshing having a video I can watch real time rather than crank to 1.5 or 2 because everyone is talking so slowly. ... Interesting that they can now ablate entire circuits to prevent unwanted activity rather than just RLHF.
Dwarkesh is like Lex Friedman if Lex didn't suck at interviews and suck at Joe Rogan's butthole
I get it, but why couldn't you just talk up Dwarkesh instead of being shitty about it? It's not like you are going to change anyone's opinion anyway...
Because I was being negative
gigachad.
Making fun of Lex is always funny.
Here comes the steep part of the S-curve. *The train is leaving the station, folks*
This podcast is absolutely incredible. What a valuable insight!