Unsupervised Learning with Jacob Effron
00:37:21 · Diogo, Jacob Effron
Like, I would be truly embarrassed to say that this is prod ready because there's so much jaggedness left in it that I'm I'm jazzed if people find stuff to actually deploy, but I will say that this does not have, like, my seal of approval yet. We need to do so many rounds. Because, like, it it... Like like like, the... It's it's not quite as bad as, like, other people just shipping a shape of something, but it's it's, like, in the grand scheme of it, like, image... Our image model is, like, maybe 10% of the way that our, like, text model is. [00:37:51] S0: As you move from, like, 10% by your characterization to, like, 90, 95%, what are the first kinds of things that you're excited about gem models doing on the image side?
00:38:00 · Diogo
[00:38:00] S1: On image land, I have a soft spot for video games and computer use. Video games because I have, like, a video game background. I... And... Oh, man. I I I... Confession. I've also, like, played around with botting them, not for competitive reasons, but just because it's cool to automate stuff. And Yeah. I think, like, being able to play around with that and, like... I I feel like software is a little bit like warping technology to your will, and being able to, like, expand what you can warp to your will is just very cool in general. So I think that is really cool in video games. Computer use, I think, is gonna be super duper valuable, unbelievably valuable. I I think it's it's... Like, especially combined with, like, lots of other stuff, I think that AI should be so much better for interfaces. It's actually kind of confusing to me how it is so bad. You know, like, why... There's a lot of people who don't like using computers, and they still have to even if they're, like, old and not experts despite that, despite AI being probably better than an assistant that they could hire to do a lot of computer stuff for them. And I would love for that to be reliable enough that people could, like, trust it as an interface. And, you know, I I guess an additional thing that is, like, I have a soft spot for, and this is why I'm wearing my, like, unsloppable shirt, is I would love ways to just stop slop. For me, personally, it might be bad for the Internet. Like like, the Internet is funded by ads, so, like, maybe it's a good thing. I don't know. But I would love if I could have some... Or someone would make me, like, a slop stopper where I just don't need to see it anymore. That would be lovely.
00:39:36 · Jacob Effron
[00:39:36] S0: The one thing I've been curious about from the outside is, like, the extent to which, you know, the advances that you've had at Jev were enabled by, like, all the advances we've seen in in, you know, AI in the last two, three years? Or could you have just done this five years ago? Like, obviously, you have a different approach that that is, in some ways orthogonal to the rest of the field. Like, could you have done this five years ago, or how much of what you built, like, does leverage some of the stuff that's been done, you know, on the LM side?
00:40:01 · Diogo
[00:40:01] S1: Five years ago, it would have been harder but doable. Back when I was more naive... Oh, man. I sound like an old dude. When I was more naive and I thought that this would take a week of work, what convinced me... I I love writing documents. I love... Like, I think it's a great way to think. I wrote a document at OpenAI that was titled, we could have had AGI a year ago, at least by OpenAI's definition of AGI, which I don't think that people really listen to the letter of the definition, but I think it's actually a really great letter. I think that models from the... Starting from GPT 3.5 era, possibly even GPT three era, could have met the definition of AGI at the time, which is still the same, like, doing most of the world's economically valuable work by volume. It doesn't say by volume, but I assume the spirit of that is by volume. Because the models are so unbelievably smart. Like, GPT 3.5 level models surpassed median level intelligence at all sorts of things. And, like, shouldn't that be used for tons of automation, at least for the easy work? And so, yes, I think that something like this would have been harder, but it could have been done five years ago. Absolutely. And, you know, like, on the people who are, like, obsessed with, like, for example, like, the the benchmarks of the day and they see, like, what tasks AI is doing. These tests are, like, really hard. They're, like, really, really, really hard. GPQA was saturated twenty twenty three ish. Maybe not totally saturated, but it was, like, surpassing PhD student level at 2023, I believe. Give me Google proof question answering for, like, PhD level intelligence at these extremely hard multiple choice tasks. It was passing that level twenty twenty three. Like, how is it that that level of intelligence was not being put to work? That is super tragic. So I think we've had a lot of, like, raw material for intelligence for a very long time, and we're gonna see a whole lot of craziness. Yeah. I mean, speaking of
00:42:02 · Jacob Effron
[00:42:02] S0: that craziness, I think part of... I imagine there's a lot a lot of noise. You know, I feel like there's been a lot of noise in the ecosystem, and I bet many of our listeners will be familiar with, you know, after you guys came with these these models, you had OpenAI and Cloudflare and Microsoft, and everyone kind of released these these models. I I know you'll you'll say that the benchmarks aren't aren't a good way to test them. They obviously... You know, I've I've tried to show that that they're near equivalent. I guess, how do you think about... Like, you obviously have now invited a bunch of competition into the space. How do you think about, like, what, you know, keeps Jev... You know, I guess everyone likes to ask about moats, so I guess I'll ask it on on behalf of our listeners. Like, how do you think about that going forward for the business?
00:42:39 · Diogo
[00:42:39] S1: Oh, boy. It feels like other people are digging our moat for us, which is kind of not how I thought it would go, and I think that that's kind of bad for the field. I think that releasing models that are, like, kind of more benchmark optimized than real world optimized actually burns that goodwill for people to try things out, which is the only thing that matters. I I truly think that this is not great for the field, and my hope is that just giving unbelievable facts about reliability is going to be the north star. I think that will be immensely, immensely valuable for the world. There's just so much more to do there that I think that it's a little bit early to worry about competition. It's it's kind of like OpenAI worrying about a moat at at G P T 3.5. Like, who cares? Like, G P T 3 Point Five's market is nothing, or it was nothing, right, in comparison. It's still, I think, going to be nothing in the grand scheme of things. Like Yeah. We truly believe in Jevons paradox, obviously, hence the name Jev. Like, Jev and Paradox... Like like, are we seeing rebound, you know, more demand because it's so cheap? Absolutely. I believe every good sees rebound. Are we seeing backfire? Like, the total market size increases. This is this is when Jev and Paradox holds. Absolutely not. Like, it it... Like like, a key part of it has to be like, wow. This is so good and so cheap. I'm gonna be throwing it all over the place because it makes value in all of these places, and my budget for AI will actually increase.