We might become grey goo, but before that, what about our spirits? "The world it envisions for all of us is small, flat, grey, leveled of all exception… a world without heroes or glory… without myths, without men who rise above the rest…"
That quote is from a Marco Rubio speech about communism, but as
@GregoryConti19 points out, it may be more immediately pertinent to AI.
compactmag.com/article/big-t…
I find the language in this essay quite moving, both as it celebrates the spirit of human excellence and worries for its undoing:
" [A] world in which there is no opportunity for individuals to distinguish themselves via skill and effort, because these things will have been rendered irrelevant … A world of boredom, idleness and mere consumption awaits us, for it is the machines that will henceforth solve all the problems we face."
"The channels for the human mind to swim in will dry up, one-by-one… The opportunities for a man to feel the satisfaction of a "job well done" will disappear. … The awe that we feel in reflecting on an Einstein or a Dostoevsky, we will feel toward no one again… The whole concept of a "life's work" that belongs to a great progenitor of ideas is under threat."
"Who cares if AI…generates reams of new historical writing but in doing so saps our ability to cherish and devote ourselves to history?"
"Human beings weren't made to only have hobbies. We need, even more, to have vocations. The nature of the latter is that the pleasure comes not merely from the doing itself but from the idea that one is making a real contribution—in theological language, that God called you to it because your work there was necessary. But as soon as the machines can do what we do better than we ourselves can, the conditions for our having a vocation cease to exist. Instead, we will have only play, mere amusement"
I don't draw conclusions quite so strong as Gregory's.
I think he's right that many of these channels will be destroyed, and that this is a deep harm that AI advocates are not adequately considering. I think we'll now have the forced obligation (and, through new capacities, the opportunity) to create new channels for excellence—and that we'll succeed, though I'm not sure to what degree. And we've already retired some scholarly feats which would have once seemed inspiring and central, like accurately memorizing long texts, or rapid mental arithmetic.
I don't think that machines which solve all our material problems would necessarily rob us entirely of "all that is good and noble". I like
@Meaningness's definition of nobility as "the aspiration to manifest glory for the benefit of others". There are lots of opportunities to do that in, say, the relational sphere, though as
@clarabcollier has recently highlighted, that might feel awfully hollow if it's all we have.
I'm unsure, too, about the boundary between hobbies and vocations. I generally think of design and research as my vocation, and a world where my ingenuity can contribute nothing there does seem dimmer. But I also spend a lot of time on my music hobbies. I'm not extraordinary by any means—many others have played anything I'm playing better. But when I play with others, it does feel like I'm "making a real contribution", to that moment together. Maybe the kinds of contributions I make will become mostly like that—situated, particular. If my life were nothing but that kind of contribution, it would be less rich, but not quite, I think, "small, flat, grey", etc.
Meanwhile, in many ways, I expect life to become much more vivid. I already experience this in small ways: I have an idea for a prototype, and coding agents can help me explore the possibility space with much less meaningless grind; I understand more of what I'm looking at when I'm traveling; I'm able to spot more interesting connections across my journals over time.
With the last few paragraphs, I'm qualifying Gregory's claims a bit, but that doesn't mean I think the situation is okay, at all. Leaving beside all globe-striding concerns about bio-risk, x-risk, etc, these are deep problems that are being imposed on me, and you, by others, without democratic participation.
I think this is immoral, but I'm confused about that assessment, because of course I am myself a technologist—isn't my whole telos to create transformative change? Where's the line? Or, maybe more importantly, what's the acceptable rate of change of the line? And who gets to decide?