Lunar Landing RL: learning REINFORCE with a rocket

Separate playing, demonstrating, and learning

Manual controls let a person try the task. The fixed-rule demo shows a hand-written controller, while neural mode can load an untrained or trained network. Watching a successful demo does not establish that training succeeded.

Gymnasium supplies LunarLander’s simulation and rewards. This repository adds the learning workflow, controls, explanations, and charts rather than claiming authorship of the underlying environment.

A small policy with visible updates

The documented network maps eight observations through two 128-unit ReLU layers to four action probabilities. A PyTorch REINFORCE trainer uses sampled episodes and discounted returns to update the policy.

Parallel workers collect complete episodes with the current weights, followed by a batch update. Saved models retain their architecture, while CSV files and charts show the training trajectory. Each training command starts a fresh network.

Exploratory curves need context

The repository includes seven 5,000-episode charts varying learning rate and gamma. The 0.003 learning-rate and 0.99 gamma run has the highest final smoothed training reward among those charts.

Those historical images do not record every setting or separate evaluation scores. They cannot establish a reliable ranking or landing success rate. The trainer saves the final policy rather than automatically selecting the best earlier one.

Use the learning guide with the code

The guide explains discounted returns, policy-gradient loss, and evaluation. Compare training reward with a separate greedy evaluation before interpreting a chart, and keep new output names so experiments remain distinguishable.