The NEAT Algorithm is Neat

NEAT’s core promise is structural learning: instead of keeping a fixed neural-network shape and only tuning weights, it evolves the network topology itself—adding and removing nodes and connections—while also adjusting weights and biases. That shift matters because it lets a relatively lightweight, CPU-friendly evolutionary search discover working controllers without the heavy training pipelines associated with modern GPU reinforcement learning. In practice, the walkthrough uses NEAT to solve classic OpenAI Gym control tasks by repeatedly evaluating candidate networks, scoring them by episode performance, and letting the population evolve until a fitness threshold is met.

The setup starts with the NEAT Python library and the NEAT algorithm’s defining idea: begin with a simple network and let evolution grow it toward better performance. The guide emphasizes that NEAT was published in 2002, predating today’s GPU-centric deep learning era, and argues that many problems NEAT can handle don’t require long training runs. For experimentation, OpenAI Gym environments are used because they provide straightforward observation vectors and discrete or bounded action spaces.

For CartPole, the process is concrete. The environment produces an observation vector (cart position, cart velocity, pole angle, pole angular velocity), while the action space is discrete—effectively choosing left or right. The NEAT evaluation loop resets the environment, feeds the current observation through the network via `net.activate`, converts the network outputs into an action (using `np.argmax` in the discrete case), steps the environment, and accumulates fitness based on reward (reward per frame alive). A NEAT configuration file defines network input/output sizes, activation function (the walkthrough favors `clamped`, which outputs in a range like -1 to 1), population size, and the stopping criterion. After some debugging around import/config naming and fitness thresholds, the system reaches the target performance quickly—balancing the pole for the environment’s maximum episode length.

The same training scaffold is then adapted to BipedalWalker. Here the observation is larger (a flattened vector of 24 values), and the action space is continuous with multiple action dimensions (four outputs). The evaluation loop changes accordingly: instead of taking an argmax, the network’s outputs are used directly as actions, and fitness is tied to forward progress (with large penalties if the agent falls). The walkthrough notes that this task takes longer and may require tuning NEAT hyperparameters like population size to keep iteration times reasonable.

Finally, the guide extends NEAT beyond control tasks into Conway’s Game of Life. The “environment” is a grid of cells governed by birth/death rules, with no direct movement—only evolving patterns. Fitness is based on how long a structure survives, with additional logic to penalize trivial oscillations (including repeated frame sequences). The evolved networks learn to place initial live cells that generate stable or long-lived configurations, including recognizable motifs like gliders and a specific long-surviving structure nicknamed the “176er.” The result is a demonstration that NEAT can discover nontrivial, rule-driven behaviors—often by evolving surprisingly compact structures—so long as the task’s search space isn’t dominated by extreme complexity.