Intelligent Mutations in Genetic Programming: OpenAI Proposes Evolution Through Large Models

Large-scale language models (LLMs) have achieved impressive performance in automated code generation by bootstrapping human knowledge and learning from extremely large datasets. Might it be possible to combine deep learning-based LLMs with genetic programming (GP) — a generative technique that can evolve in low resource spaces — to leverage the benefits of both methods?

In the new paper Evolution Through Large Models, an OpenAI research team shows that LLMs trained to generate modern programming language can suggest mutations that are intelligent; and that this ability can be leveraged to realize dramatically improved mutation operators for GP.

The team summarizes their study’s main contributions as follows:

1. The ELM method for efficiently evolving programs through LLMs.
2. A technique for improving ELM’s ability to search over time by fine-tuning its LLM-based mutation operator.
3. A demonstration of ELM in a domain not included in the training data of the LLM.
4. Validation that data generated through ELM can bootstrap enhanced LLMs that offer a novel path towards open-endedness.

The ELM approach is centred on rethinking the mutation operator for code by exploiting the capabilities of LLMs, and comprises three core components: 1) a novel mutation operator driven by an LLM, 2) an evolutionary outer loop that calls this mutation operator, and 3) a method for updating and improving the LLM based on preceding performance.

In a conventional GP approach, a mutation range is specified for the operator to yield a reasonable chance that the perturbations will lead to useful functional changes in the code. This differs from LLMs, which use a more humanlike approach wherein the operator aims at “interpreting” the code and “understanding” how it changes. The proposed ELM method borrows from this, employing a mutation (diff) operator that leverages commit messages that carry useful mutation information for calibration.

The team implements ELM within a quality diversity (QD) algorithm (MAP-Elites, Mouret et al., 2015) for evolution in the outer loop. This algorithm was chosen for its ability to search intelligently for arbitrarily complex programs, which advances the process toward open-endedness, i.e. a paradigm of never-ending innovation such as that evidenced in nature.

To enable the self-adaptation capabilities of the proposed ELM, the team fine-tunes their pretrained diff model with accepted diffs by MAP-Elites from initial ELM iterations/runs, so that ELM naturally improves itself through iterations.

In their empirical study, the team implemented ELM combined with MAP-Elites to generate hundreds of thousands of Python codes outputting working ambulating robots in the Sodarace creature-creation domain. The results show that the proposed intelligent LLM-based mutation operators can successfully bootstrap new models that output appropriate artifacts in a given domain under a zero-shot setting.

The team hopes their work can motivate research on additional applications that can benefit from combining GP and Deep Learning.

The paper Evolution through Large Models is on arXiv.

---

Author: Hecate He | Editor: Michael Sarazen

---

We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.