"The Cognitive Revolution"
01:18:34 · Nathan Labenz, Prakash Narayanan
So you you would need multiple kinds of expertise. Like, that data is not that accessible. There are these, like, you know, moves to try to rein in that data and not let it, you know, just hang out all over the Internet. Prakash had heard a claim that morning that Claude was refusing to help clone video games. [01:18:56] S6: I I think I think what I fear is that they're pissing off the gamers. Right? And gaming. And you say Claude is not gonna help you, like, make these games? You're just asking for trouble. You're asking for the most security competent group, the most technically competent, and the people with the most time to burn, and it's a passion for them. NVIDIA was built on gamers. Like, John Carmack was a gamer. Right? You tell John Carmack that he can't do something. Like, you are in trouble. Right?
01:19:30 · Nathan Labenz
[01:19:30] S0: I I think that's apt. I mean, you... I think it is definitely the case that, like, we should be pretty cautious around where we deploy these things because we don't want to burn them on stupid things that don't really matter that much. It'll be like, you know, of all the ways to fail in this, it'll be really dumb if it's like, well, we were interested in protecting, you know, Disney IP. And so because of that, we, like, put in these guardrails, but then, you know, the fan fic community was so strong that they came over the top and, like, broke all of our safeguards. And now everything's, like, running wild, and it all was because, like, we were too draconian on things that didn't really matter. I mean, that's a bit of a caricature story, but it's... You know, that's basically the same story as the gaming one you're telling. And I can see that happening in different domains, and it really would be incredibly stupid. So I I do think we need to be, you know, pretty disciplined about, like, just how far we try to create these, like, knowledge gaps or refusals because everyone does potentially mobilize another community to try to undo them. And I agreed too that you're not gonna convince gamers that, like, well, hey. Because you're doing this, you know, you're creating this other downstream problem. Well, good luck having that. You know, you're not gonna win that fight in the Discord. So better to just probably let them have their fun and then, you know, keep our key techniques where we, like, really, really care and really have to have them. One more from Friday's closing right after Sagar left. Mistral's technical report and a Claude four system card story I told from memory. The card itself says, Anthropic had mistakenly omitted the only fine tuning dataset that included harmful system prompts, and that this was not the sole cause. So I'll be very interested to read the more open report on exactly, and hopefully, you know, we'll see how how open they are about all of the technical details. There have been these, like, footnotes in multiple system card reports over time. One that I really was kind of obsessed with for a minute was when... So Anthropic had forgotten to include a particular dataset called sysprompt harmful, which was meant to teach the model that it should not respond...
01:22:05 · Nathan Labenz, S7
You know, it should not directly follow a harmful system prompt. So it did follow the directions of a harmful system prompt, and this made it all the way through to the model card. We don't roll back and, you know, train again. Nope. We just kinda figure out a way to patch. I've really been very curious as to, like, how that stuff actually works and, like, how reliably we should think of it as working. That's the week. Thanks for listening to the AI and the AM weekly highlights. The narration was my cloned voice. Please tell us what worked and what didn't. See you in the morning. [01:23:31] S7: Aimed out and let her go. Ain't no calling her back. Water flat as a plate. All we do now is wait. This is the one we've got. The one we've got. Halfway out, something knocking, something wrong down below. Can't turn around, can't stop or grease up to the elbow. All hands all night. Nobody's saying it's fine. Come on in one or two ways. Everybody happy or everybody sad. No in between or halfway. Hold on. Hold on. Hold on. Hold on. This is the one we've got. The one we've Waterline keeps rising, call goes out for one more hand. There's a faith in fixing.
01:23:31 · S7, Nathan Labenz
[01:23:31] S7: Aimed out and let her go. Ain't no calling her back. Water flat as a plate. All we do now is wait. This is the one we've got. The one we've got. Halfway out, something knocking, something wrong down below. Can't turn around, can't stop or grease up to the elbow. All hands all night. Nobody's saying it's fine. Come on in one or two ways. Everybody happy or everybody sad. No in between or halfway. Hold on. Hold on. Hold on. Hold on. This is the one we've got. The one we've Waterline keeps rising, call goes out for one more hand. There's a faith in fixing. [01:25:53] S0: If you're finding value in the show, we'd appreciate it if you'd take a moment to share it with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries, either via our website, cognitive revolution dot a I, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts which is now part of a sixteen z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at aipodcast.ing. And thank you to everyone who listens for being part of the Cognitive Revolution.