I’ve decided to swing the pendulum all the way from “masterpiece which takes ten years to produce a single sentence,” to “thoughts fresh off the shower.” We’ll see how it goes. The following is a summary of what I was thinking about yesterday, i.e., whether or not it makes sense to think of evolution as optimizing.
In general I’m pretty interested in why living things seem to not fall into some of the failure modes that we associate with software/AI, and also how they achieve their goals. Like, life's code may appear spaghetti-like, but it has none of the downsides (e.g., is extremely robust). All of the cells in your body seem pretty “aligned,” (e.g., when your brain sends a message to your muscles it just does it). Some crazy amount of goal preservation is happening when bodies go from a single cell to an adult form (like somehow a “human body” reliably manages to happen when scaling from one cell to 40 trillion, that’s pretty wild yo). And finally, although evolution is talked about as an optimization process, it doesn’t feel like it suffers from the failure modes (Goodharting, wireheading, optimizer’s curse, etc.)
As usual, when I first started wondering about this topic I Googled “lesswrong optimization evolution” and poked around. I stumbled onto some things Demski said which I really liked. He’s like, look, everyone talks about everything as optimization processes but in its breadth it fails to describe some meaningful bits of what’s actually going on. Like, logic? Not really an optimization process. Bayes? You’re trying to have true beliefs, but the underlying thing you’re doing is about proportional updates, which doesn’t feel very “maximize-y.” And even though ML is the classic case of optimization, there are actually things in place which try to mitigate the failures of over-optimizing, such as regularization and dropout.
In particular, the thing ML seems to be doing with regularization and dropout is more like “finding some general core” rather than a single solution to a problem. Demski thinks evolution is also doing more of this type of thing, the “finding a general core” rather than single, optimized solutions. This seems right to me, but I don’t know that I understand why.
When I think about evolution as an optimization process I’m like, “okay it’s simple, things either survive or don’t, iteratively, and then eventually you get things that are ‘optimized’ for surviving.” And in some ways this feels similar to a supervised learning algorithm training on “cat or not cat”—whatever signals made it more likely to say the right answer are reinforced, whatever signals made it less likely are punished. So too with evolution.
But… something feels a bit weird, also. Like “survive or not” is such a wide target that it stops feeling like a “target” at all. So many things can survive! Like you can just see it, they’re everywhere. And they all look pretty different. Is optimizing towards something with that large of a target really “optimization”?
I think it probably isn’t. Or like, the notion needs some revision. Maybe this is what people mean when they say that evolution “weakly optimizes,” I don’t know. To me, the “weakness” comes from where the “power” is in the optimization process. When someone has a very concrete goal that they’re optimizing for, e.g., “finish this essay by the end of the week,” it seems like the “power” derives mostly from the end state. What I mean is something like the way actions flow is from the end state to the present, e.g., by planning or back-chaining your actions from the goal.
But when you’re considering things with such huge targets that it’s unclear whether or not they’re even usefully construed as targets, it feels like the “power” of the optimization stems more from the process you use to get there. Like, maybe evolution is awesome not because the selective force is optimizing super hard, but because whatever is doing the thing that makes the stuff in the first place is powerful. (If you’re like, wtf, the “creative” part is just random mutations, how could that be powerful? I hear you, and I disagree, but that is a whole thing for a whole other post).
Or to make it more concrete, when I have some very “broad” target like “I want to understand what life is,” my actions do not flow through back-chains and I don’t really plan, instead I have a sense of what it means to do this thing, and I have some process which reliably seems to uncover new interesting questions, or make forward steps. But it’s not because I’m deriving them from a concrete goal, it’s more like, this process is directionally good, although the “direction” is not quite pinned down. My actions feel like they flow forward through the “process” more than backwards through the “goal.” There isn’t even really a “goal.”
What does any of this have to do with why evolution doesn’t seem to have as many of the optimization problems, or why it is “general,” in the sense that Demski means?
I have this hunch that it’s about processes versus outcomes, somehow. Like, when I think about Goodharting, it seems to me to mostly be about the fact that often you want to select for a process but can only select on outcomes. For example, when you’re hiring for an engineer what you want is for that engineer to have good processes, e.g., the ability to think well about new problems. But a lot of what you have to go off of are outcomes, e.g., “4.0 gpa from a good school,” or “won this award.” Outcomes are proxies for good processes, but they are not the same thing and many bad processes (e.g., cheating) can achieve the same outcome when optimization pressure is applied.
Processes are more like this “general” thing. You don’t want someone who found a hacky way to solve this particular problem (cheating to get a 4.0 gpa), you want someone who is generally smart and competent. If the process is good, it can usually be generally applied well. That’s also what we want in ML, and what regularization and what not is trying to do—you want the model to perform well on the training data and also everything else, you want a process which generalizes, not a single, over-optimized solution.
Does evolution somehow make the process thing happen? This is currently where I’m at. It sure does seem like it selects on outcome, e.g., survive or not, but then it also seems obvious that it has more of this “general” thing. What’s going on? Perhaps it is the “weak optimization” thing, where selecting on such a broad target actually pushes more of the power towards the creation part, and that part is somehow more process based. Or is it just that because the target is so broad, it’s kind of impossible to “overfit”? I’m not even sure what overfitting would mean here, like you survive so much? Or like you overfit to your environment and then the environment changes and you’re fucked? Okay maybe that’s what it is—that even though it only selects on one outcome (survive or not) the rules of the game (environments) change so much that you necessarily get out general looking structures? Processes (i.e., strategies) which perform well in many environments.
It seems like I currently think it’s some combination of these things. Evolution has a broad target it “optimizes” for: survive. Because this target is so broad, there isn’t much “top-down” pressure on the systems, i.e., it’s not selecting for a particular configuration, very far from it. Also, when the environment changes, the fitness landscape changes and so organisms have to adapt strategies that apply to multiple environments. These two things—not much pressure towards specifics and adapting to novel environments—seems like it will end up with something pretty general, something more like a “process” than a particular solution.
These thoughts don’t feel all that put together, still have some ways to go. Am I even thinking about this on the right level of abstraction? What the fuck does it even mean for evolution to optimize when evolution isn’t an entity? Like the thing that’s actually happening is that genes stick around if they’re good for surviving. But genes aren’t doing the optimization in the same way we expect AI to do the optimization (i.e., execute plans). Is what’s actually important about evolution that the optimization power is separate from what is being optimized…? Or something…?
fascinating. makes sense to me that evolution rewards more "general" solutions rather than over-optimization because "survival" can be achieved in so many different ways and the environment is constantly changing. (also interesting that the things being selected – genes or organisms – are all interacting and co-evolving with each other). I like the connection to process-oriented vs outcome-oriented.
your point about "although evolution is talked about as an optimization process, it doesn’t feel like it suffers from the failure modes (Goodharting, wireheading, optimizer’s curse, etc.)" made me think: aren't there a number of big failure modes in evolution like...cancer? maybe that's at the wrong level of analysis though (within organisms rather than at the level of organisms), idk
also, I coincidentally ran into this paper about _degeneracy_ recently which might be relevant: "we point out that degeneracy is a ubiquitous biological property and ... it is both necessary for, and an inevitable outcome of, natural selection." https://www.pnas.org/doi/10.1073/pnas.231499798