fascinating. makes sense to me that evolution rewards more "general" solutions rather than over-optimization because "survival" can be achieved in so many different ways and the environment is constantly changing. (also interesting that the things being selected – genes or organisms – are all interacting and co-evolving with each other). I like the connection to process-oriented vs outcome-oriented.
your point about "although evolution is talked about as an optimization process, it doesn’t feel like it suffers from the failure modes (Goodharting, wireheading, optimizer’s curse, etc.)" made me think: aren't there a number of big failure modes in evolution like...cancer? maybe that's at the wrong level of analysis though (within organisms rather than at the level of organisms), idk
also, I coincidentally ran into this paper about _degeneracy_ recently which might be relevant: "we point out that degeneracy is a ubiquitous biological property and ... it is both necessary for, and an inevitable outcome of, natural selection." https://www.pnas.org/doi/10.1073/pnas.231499798
fascinating. makes sense to me that evolution rewards more "general" solutions rather than over-optimization because "survival" can be achieved in so many different ways and the environment is constantly changing. (also interesting that the things being selected – genes or organisms – are all interacting and co-evolving with each other). I like the connection to process-oriented vs outcome-oriented.
your point about "although evolution is talked about as an optimization process, it doesn’t feel like it suffers from the failure modes (Goodharting, wireheading, optimizer’s curse, etc.)" made me think: aren't there a number of big failure modes in evolution like...cancer? maybe that's at the wrong level of analysis though (within organisms rather than at the level of organisms), idk
also, I coincidentally ran into this paper about _degeneracy_ recently which might be relevant: "we point out that degeneracy is a ubiquitous biological property and ... it is both necessary for, and an inevitable outcome of, natural selection." https://www.pnas.org/doi/10.1073/pnas.231499798