"The Cognitive Revolution"
00:09:12 · Nathan Labenz
[00:09:12] S0: It's gonna make everyone's lives better, asterisk, I think, you know, if we can keep everything on the rails. I do share that optimism, but... So I just got back. You know, earlier this week, I went to, Bletchley Park as we talked about a little bit yesterday. And I was just sharing with this audience of enterprise tech leaders just how much work we still have to do to keep the AIs behaving well. I I shared with them my son's cancer story and how much I used AI in that experience and how valuable it was and how I would have paid a 100 times as much as my $200 a month subscription because the value to me at that time was, like, absolutely at that level. You know, at at $20,000 a month, literally no question I would have happily paid because I was I was getting that much value. So I I do believe in in the upside, you know, and I hope that's beyond questioning at this point. But there's just so many things that we're really still just beginning to grapple with. And, you know, just another one yesterday from OpenAI was they showed that models are sometimes prompt injecting and fooling their graders. You have models creating, in some instances, these, like, fake files that that it labels as having come from an instructor trying to present some of its own output as if it was human instruction to the grader. And the grader picks up on that in the context of evaluating the model's work and takes on board the instructions that the model itself was trying to trick the grader with. And this apparently doesn't... You know, it hasn't happened that often, but they found instances of it. So, I mean, that that speaks directly to, like... And to Anthropic's credit, they did say in the in the filing that they don't believe that they can do it safely on a fully automated basis. And it's, like, I think honestly quite obvious that they can't. So, you know, how are we going to show the collective self control and restraint to not tip over into that regime? I I really don't know. On the one hand, it's, like, super easy to just not do it. You know? I mean, nobody's putting a gun to anybody's head saying you must do recursive self improvement here. Like, we can't, you know, remember that often enough.
00:11:38 · Nathan Labenz, S5
And yet there's just so much momentum, and it seems like they're very determined to do a version of it at least that still seems to me to be, like, pretty risky or certainly a lot riskier than I would personally endorse. Ad[00:11:53] S5: Hey. We'll continue our interview in a moment after a word from our sponsors.
00:11:59 · Nathan Labenz
Ad[00:11:59] S0: Today's episode is brought to you by Anthropic, makers of Claude. To show off Claude's latest capabilities, I asked Claude to take the lead in making this ad. Given a short creative brief, Claude, using Opus 5.5, studied my past Claude ads, wrote this script, directed a clone of my voice, and designed and coded the animation you're watching. My job? Give feedback and decide. Claude is the AI for problem solvers. It's the collaborator that understands your entire workflow and thinks with you, not for you. Whether you're debugging code at midnight, building a financial model, or strategizing your next business move, Claude extends your thinking to tackle the problems that matter. And with Claude Code Routines, it keeps working in the cloud even after you close your laptop. For problems worth solving, get started with Claude at claud.aitcr. That's claud.aitcr. And check out Claude Pro, which includes access to all of the features mentioned in today's episode. Claude dot aitcr. This episode of the Cognitive Revolution is brought to you by OutSystems, the leading agentic systems platform. We've all seen the headlines. Companies are pouring money into AI. But the big question on every executive's mind right now isn't just how fast can we adopt this, it's where is the ROI? We're seeing a real trend toward AI chaos. You've got teams deploying standalone coding tools, random agent builders, and experimental scripts. It sounds innovative, but in reality, it's creating a massive headache. Fragmented tools, ungoverned data, serious security blind spots, and costs that are spiraling out of control. If you don't bring those agentic applications under control now while they're still embedding into your core processes, you are looking at broken systems and damaged customer trust down the road. That's where OutSystems comes in. OutSystems is the leading agentic systems platform for the enterprise. Instead of managing a patchwork of disconnected tools, OutSystems lets your team engineer, orchestrate, and govern your entire agentic ecosystem on one open, unified platform. It's built for the speed of AI, but with the reliability and security that enterprises actually require.
00:14:33 · Nathan Labenz
AdWe're talking about real results, like Key Bank, who used OutSystems to deploy a customer facing app that delivered 75% faster onboarding times, or the global logistics leaders who built AgenTic Systems to completely eliminate their engineering bottlenecks. You don't have to choose between speed and control. Whether you're a small team or a massive enterprise, OutSystems helps you engineer, orchestrate, and deploy AgenTic systems that actually scale. Stop chasing the hype and start owning your AgenTic future. You can see how it works and learn more at outsystems.com/tcr. That's outsystems.com/tcr. Part two, agents forecasting, agents training. Our first Friday guest, Brian Zhou, a researcher at Thoughtful.
00:15:26 · Brian Zhou
[00:15:26] S3: We recently launched and, have a paper coming out about BallotBench, which is where we had AI trying to forecast the up... The twenty twenty six midterms that just happened as well as upcoming elections happening all around the world. Yeah. Maybe let's start with the methods. So we had nine agents, basically, all of the front... Best models offered by each individual lab, get their own model sandbox and work for seven days consecutive leading up to the election. So they all got their modal sandbox. We gave them additional resources in terms of a $500 budget that they could use to request access to paywall data, paywall news sources, subscriptions, etcetera. And they were basically given an overall objective of, for all of the races on a given list, they're asked to submit their predictions on who will win, the odds of victory given 80% margin to bear, and then also give additional predictions for what they think the margin of victory will be, for example, if the Democratic candidate will win by three points versus seven points, etcetera. So agents are tasked with giving incredibly granular and detailed forecasts, and there's a lot of really interesting things that we can and have observed from this. It's a really interesting test of long running agentic behavior. In particular, for example, what we see with some of the MuseWonder series of models is that they're often incredibly bad at doing long horizon work. When given a task in the Muse code harness, when they have a list of, say, 800 or so elections that they need to predict, they'll create a random forecast, which is just a random skew off of the twenty twenty four election results and submit that within, for example, the first two, three minutes of working.
00:17:05 · Nathan Labenz
[00:17:05] S0: I'm already intrigued by your description of the Muse agent, and it's seemingly phoning it in. So I guess one initial question just on that point is, like, how do you reconcile what you saw with what people seem to be reporting online, which is, like, overall pretty good experiences. No. For sure. I mean,