The Information's TITV
00:49:11 · Akash Pasricha, Dan Shipper, Paul Kedrosky
[00:49:11] S0: fast. And this is kind of interesting though because you're saying management's expectations are too high, and yet we've got headlines every week saying x y z company is putting a cap on how much you you you spent on AI. [00:49:25] S2: And there's, you know, and there's there's different managers in different parts of the economy, and and and I think there's this flip flop from, like, token maxing to, like, token token minning. And and and to me, what what that says is if you're an executive, you're... If you're a CEO and you are not... You don't have your hands in this technology all the time, the only thing that you can do is follow the trend and be like, oh, token maxing or oh, no. Token maxing is over and now we're gonna put put budget caps. That's the only thing you can do and both don't work. Because the reality is that, yeah, forcing people to spend as many tokens as possible when your organization is not set up to do that well, it's not gonna work. Obviously, you're just gonna get people who are gonna spend tokens on things that don't don't matter. But but if your organization is set up to use AI well and you and you unleash the people who are really good at doing it, you actually probably should be token maxing because the more tokens you spend... If your if your setup is good, the more token you... The more tokens you spend, the more high quality output you [00:50:22] S1: get. Right.
00:50:22 · Paul Kedrosky, Dan Shipper, Akash Pasricha
[00:50:22] S1: get. Right. [00:50:23] S2: Right. And so I don't think there's I don't think there's one right answer, but the the the... In terms of, like, a headline token maxing or token budgets, but the the right answer is always you have to have your hands in it in order to make good decisions. [00:50:35] S0: Right. Okay. So let's let's talk about the tokens and the model specifically. So you've been doing these vibe checks for for how long been [00:50:43] S2: Three years [00:50:44] S0: or four years? Yeah. Three years of vibe checks. You've probably done dozens by now. [00:50:47] S2: I mean, [00:50:47] S0: more time. Maybe. So I wanna talk about where the models are going. And I asked you a question a couple months ago when you came on the show about what the models are not yet good at. [00:51:00] S2: And What did I say? Do you remember? [00:51:03] S0: I I don't. The the reason I'm asking you again is because we've had a couple models come out since then. [00:51:11] S1: Yeah. [00:51:12] S0: So I'm gonna re ask you the question, what are they not good at right now?
00:51:16 · Dan Shipper
[00:51:16] S2: That's a that's a really good one. Okay. So, you know, obviously, the models, they're out there, like, hacking into, like, government, like, infrastructure and solving Navier Stokes. So they're, like, they're very good. And there are some models that are coming out that are not out yet that are very, very good. And I I think will change how we think about things like building software, for example. And they are very bad at a lot of things. A really a really simple example is models often leak their context in in the way that they do their output. So an example might be, you've ever had the experience of working with a model and you're like... You're you're you're you're making a landing page together and you're like, yeah, I don't like this headline because it it mentions x and and... But it should mention y. And then the model's like, great. I'm gonna rewrite the landing page, and it says, we're not gonna talk about x. We're we're gonna we're gonna write y, and that puts it in the copy of the landing page. Like, is a... That's a leak in the way that it that it that it thinks into the artifact that it's producing. And this is actually quite common even with the smartest models. And more generally, the category I would put this in is coherence, is when a model gives you an answer, does it have an accurate understanding of what it knows, what you know, and what should be in the thing that you're making together? And very often, models break coherence. And even though they're out there swarming and and solving all these amazing problems, something that's that that basic, you and I would not have a particular problem knowing, is still something that they need to improve a lot on.
00:52:58 · Akash Pasricha, Dan Shipper
[00:52:58] S0: Okay. So which lab is doing the best job of solving that? [00:53:00] S2: Oh, you know, there's there's actually, like, a really good dark horse here, which is Google. Right. [00:53:05] S0: You just talk on about Gemini being [00:53:07] S2: I've seen slept on a bit. Gemini is just really good. It's not a good coder, but it is really good... A really good writer. And there... It knows things about how to write that you probably know that are about, like, how to tell a story where each part of your argument, you have to... Like, the the the one paragraph has to lead into the next. It has to set it up so it causes the next thing to happen. Like, it knows how to do that in a way that even, like, Opus or Fable or Astra, they just sort of, like, jump and then you get that, like, this is, like, this is sort of slap. Like, it all kind of makes sense if you don't if you don't look at it too carefully. But once you really look at it, it doesn't make any sense. Gemini doesn't do that. And I have no idea why. My own... My my one theory is that there is just an advantage to being Google. Like, you you have actually organized the world's information for the last twenty years, and that allows you to, like, get a data mix that's different. [00:54:02] S0: Right.
00:54:02 · Akash Pasricha, Dan Shipper
[00:54:02] S0: Right. [00:54:02] S2: And it it it reminds me using Gemini how strange it is that almost all the other models sound a lot alike. Like, their data mixes are very similar. And Gemini is the first one that I've used, like a a frontier model, the first frontier model I've used in a while that just feels like it's trained on something else. And that's really cool. [00:54:21] S0: So so this is kind of interesting. I mean, so so Google, if I hear you correctly, you're saying that if you want to basically, what, be a creative writer, you know, Google is the model suite you can use. Where I'm going with this is actually, I I wanna get your take on, if we go through each of the labs, I wanna know from you what type of work you think they're prioritizing. Sounds like Google and the Gemini family, it's creative work that you think they're prioritizing [00:54:51] S2: I wouldn't say creative writing. I'm thinking more about, like, non fiction writing, like the writing you might do do at work. Or, you know, for us at Every, like, we we write, like, long form pieces. I wrote this, like, 8,000 word piece on the history of Codex, like, [00:55:06] S0: Codex It's is having that still personal personal.
00:55:06 · Akash Pasricha, Dan Shipper
[00:55:06] S0: Codex It's is having that still personal personal. [00:55:08] S2: It is... Yeah. Like, that is a personal first person narrative. Okay. But it's not creative writing in the sense that I don't necessarily look at it as something that I would wanna write a short story with. And I also don't necessarily think it is the best at nailing a particular person's style. I think Claude is actually much better at at just doing the style or voice. But in terms of how it structures and lays out a series of thoughts, it is the best. [00:55:32] S0: Okay. So... And so let's go rapid fire through the the rest of the labs here. Just give me, you know, two words here to describe the type of work that you think each lab is prioritizing for. So let's let's go with meta, actually. Consumer [00:55:50] S2: consumer shopping. Consumer shopping. Okay. Or, like, consumer interactions, you know, what... Whether that's, like, keeping up with your sports team or or finding stuff to buy or whatever, like, all the kinda, like, stuff that happens in your day to day life as a as a regular consumer. [00:56:06] S0: What about Anthropic? [00:56:08] S2: Power creative tools. Power creative tools? [00:56:12] S0: Yeah. Not power enterprise tools. [00:56:14] S2: I mean, I think that they care about that, but the the, like, the feeling that I get from Anthropic is, like, it's for it's for pros. It's for creative pros. [00:56:21] S0: Okay. Yeah. What about OpenAI? [00:56:23] S2: Knowledge work. Like like, more more specifically for the general knowledge worker that just needs to get shit done.