00:06:52 · Jaron Lanier
[00:06:52] S7: Yeah. That's right. I guess I should just take a moment here to say that I'm not an absolutist critic at all. What I had always thought is that the core issue is really the business model that if the way you make money from computation is by manipulating people, which is very, very easy to do with computation. It's the very easiest thing to do with it at all. Then your incentives become rather dark, and you tend to make money from doing things that are darker and darker. And in my view, that's why we have a kind of an unfortunately dark and depressed culture in younger generations all over the world. And it's why politics went off the rails in a lot of ways during this period and many other things. Just as we're saying that tech should be able to improve medicine, tech is actually probably doing more to harm people by making them reject modern medicine through cranky algorithms. So, you know, we make ourselves so absurd. Now the thing is, the business model for AI has a lot in common. It has the addiction dynamic in common. People definitely get addicted to using their chat. It definitely has the inherent ultra easy readily available illusions of computing in common. And what I mean by that, for those who don't know, way back in the mid sixties when computers were so small, they really couldn't do anything, there was an experiment called ELIZA where somebody made a chatbot that just kinda spit back what somebody had said, and people were convinced it was the most empathic, fully conscious, real therapist god ever. Like, you know, like, it takes no technical know how to make computing into this addictive manipulative thing because it's just so natural, just having this information system that can respond. It's just super easy to turn it into a manipulation box. This is going back to the social media era. If you say everything's free, then you're the product as the saying goes and as many have discovered it. The interesting thing about the AI era is people are actually paying for these subscriptions for the most... A lot of them are, not everybody. But... So in a way, what I wanted is happening. But here, the problem is a different problem where... Because we now have GenAI, we have a way of filtering all of the wealth and efforts and creativity of mankind through this scrambler so that it seems like it's just coming from the company.
00:09:12 · Jaron Lanier, Danny Fortson
So in a way, we're getting the worst of both worlds. We're getting the sort of weird anonymizer, dehumanizer thing that was happening with the algorithms on social media. But at this... But then we are paying for it, but it's not actually going to the people who are the sources of the information that's being rescrambled to turn into Gen AI. Instead, it's just going to these companies, And it suggests this idea where the only thing you need in the world is OpenAI and Anthropic, and everything else becomes dispensable, including their own customers. So it's very stupid, and I feel very embarrassed for my kind. I don't know what to say. I just... It's it's just very pathetic. [00:09:52] S10: So is what you're saying that effectively what we're doing with AI is basically we're paying for our own stuff because these systems have trained on all human creation. Anything on everything on the Internet, all the books, everything. And then as you say, it goes to the dehumanizer, which is a great term. And then we pay $20 a month to get access to that output
00:10:18 · Jaron Lanier
[00:10:18] S7: that Let me qualify this a little bit because that, I am sort of saying that, but not quite. There's an important difference here. So to me, the the large language models are cool things. Like, I... I'm not anti the technology itself. In fact, I think it's cool. I love having new tools to extend human efforts and collaboration. But the thing is, just like with a library, a library by itself just sits there. It only has any meaning if somebody goes in and does something with it. Like a library without a culture of scholarship, without a culture of library science, without librarians, it's it's just inert. And the same thing is true of these models, these large language models and other recent frontier models. There has to be something done with them in order for them to have any impact. And so the thing we chose to do is reproduce this Eliza illusion from the sixties or the thing that drove social media, which is, oh, let's make it a fake person, and then, people become addicted to it. They treat it as their therapist or girlfriend or partner in some way. And then, of course, you start to own their lives because now you own this intimate part of them. Just like with social media, I think sometimes it works out well. Sometimes it doesn't.
00:11:33 · Danny Fortson, Jaron Lanier
[00:11:33] S10: So question. One of the things we've been talking a lot about and writing about recently on the pod and in the paper is this notion that we, the leaders of this company, have created a new species. And therefore, they're, quote, unquote, moral agents that they have some equivalents and that we must treat them or really think about them in a way that is not just [00:11:59] S7: Yeah. [00:12:02] S10: One, do you agree with that? And two, if you don't, is that problematic, that framing? [00:12:06] S7: It embarrasses me so deeply. [00:12:08] S10: Why? [00:12:08] S7: So so... Okay. So let's unpack this. First of all, if we're talking about absolute truth, any question about what's a being and what's not a being is ultimately a matter of faith. We have latitude to believe different things. We have to live, and we have to make decisions. And I think as a pragmatic concern, we just have to have assumptions that we make about things. So I I feel a similar argument for me has to apply to AI, which is, you know, if we make a machine and we declare it god, what's the consequence? And the consequence is we have to let it run rampant, and letting a machine run rampant is really stupid because they... [00:12:51] S10: You wouldn't Well, and that's kind of what we're... That is what we're doing right now.
00:12:55 · Jaron Lanier, Danny Fortson
[00:12:55] S7: Yeah. And so the thing is it's pragmatically absolutely absurd. Let me give you a couple of examples of that. Yeah. So what actual core benefits have we gotten from AI so far that have made a difference in things? Actually, you know, it's a little hard to nail them down, you know, honestly. But let's say one of them is... There have been some scientific projects that have been furthered by AI, like, ish... Proteins and whatnot. There's a Nobel Prize out. So one of the things that's interesting about that, if you look at that... The Nobel Prize work, it was before the popularity of trying to treat them as gods. And so, therefore, they didn't use the formulation of agents. They didn't have the thing going off on its own. It was rather an algorithm that searched the space. Now the whole idea of the agent is an... It's a ritual worship object. The idea is that we believe the AI is this living god thing. And so instead of just using it, we're gonna have it go off on its own. But the thing is that's actually not needed for doing anything productive. There are a few exceptions. If you wanna use an I... AI algorithm to control something in real time, you might wanna let it go and and do its thing. I have some friends who are trying to work with Frontier AI models to improve ICU's intensive care units at hospitals. You need an agent for that. The most prominent proposed application, which has actually seen the light of day in Ukraine, is using it to control, smart drones. [00:14:23] S10: Right.
00:14:23 · Danny Fortson, Jaron Lanier
[00:14:23] S10: Right. [00:14:23] S7: The thing is, in almost all cases, if it's something about your exercise or recipes, if it's something about your stock portfolio even, if it's something about productivity in your company, the use of agents doesn't actually help you. It's a ritual to support the adulation of this new golden calf that people made. So it's a ritual of adoration. But the problem with it is it creates dangers. So you have this repeated stream of things of, oh, we sent out all these agents, and the agents mess [00:14:54] S10: They broke out [00:14:55] S7: of their [00:14:55] S10: sandbox, and they created... They committed felonies or whatever. Yeah. [00:14:59] S7: They actually weren't needed in the first place. Like, the whole thing is stupid. This is like somebody saying, oh, I ate dirt, and it upset my stomach. [00:15:07] S10: Don't eat dirt. Don't [00:15:08] S7: eat dirt. If people do survive our stupidity during this generation... I mean, we could make people extinct. I mean, but we will be the ones doing it, not some AI god. But as presuming we survive, future historians will look back at this and will call this the era of the stupid engineers. You know? It's a shockingly dumb period. [00:15:29] S10: Do you have a sense of why? Like, why are we here? Why are we being so moronic and stupid [00:15:35] S7: Yeah. That's a great question.