A 16 July arXiv preprint reports that a reinforcement learning agent, given the Lyapunov characteristic exponent as its reward signal, rediscovered the Kapitza pendulum effect and then went past it, leaving an inverted pendulum standing strictly upright with no oscillation [S1]. The Lyapunov exponent, a number that tracks how quickly a dynamical system falls into chaos or settles into order, has been a staple of control theory for decades. Feeding it directly to an RL agent as a reward signal is a small idea with large consequences for how machines learn physical rules, and the strictly upright result is something nobody asked the agent to find.

A number that measures chaos

The Lyapunov characteristic exponent, or LCE, is a single number that tells you whether a system is stable or chaotic. Positive LCE means nearby trajectories diverge, the system is sensitive to initial conditions, the classic butterfly effect. Negative LCE means trajectories converge, the system settles down. Zero means you are on the edge [S1].

For a pendulum, the LCE tells you whether the motion is tumbling into chaos or damping toward a stable state. The authors of the preprint, which is listed under cs.AI and cs.LG on arXiv and has not been peer-reviewed [S1], propose using this number as what RL researchers call a dense reward signal.

Dense reward means the agent gets feedback at every step, rather than only at the end. Most RL tasks use sparse rewards: win the game, get a point; lose, get nothing. Sparse rewards are easy to define but hard to learn from, because the agent spends most of its time with no signal at all. Dense rewards give constant guidance, but designing one that captures what you actually want is notoriously difficult. Hand-craft a reward poorly and the agent will exploit it, finding shortcuts that satisfy the letter of the reward while violating its spirit.

The LCE reward is physics-informed: it measures something real about the system's stability at every timestep, and it cannot be gamed without actually making the system more stable.

What the agent found

The task was classic: stabilise an inverted pendulum, a rod balanced on its end, by moving its pivot point up and down [S1]. This is the setup of the Kapitza pendulum, a well-known physics result in which rapid vertical vibration of the pivot holds an inverted pendulum upside down. The effect is counterintuitive: the same rapid shaking that would knock a pendulum sideways pins it upright.

The RL agent, guided by the LCE reward, found the Kapitza oscillation on its own, according to the authors [S1]. That alone is notable. The agent was not told the answer, only told to minimise the Lyapunov exponent, and it arrived at a known physics result through exploration.

Then it went further. The authors report that the agent damped the pendulum's pivoting entirely, leaving it standing strictly upright with no oscillation [S1]. This is beyond the classical Kapitza regime, where the pendulum stays upright but still wobbles. A perfectly still inverted pendulum stabilised by vertical motion alone is not a standard textbook result.

What it means

The core idea, using a physical stability measure as an RL reward, is part of a growing wave of physics-informed machine learning. When you let an AI system learn freely, it often cheats the physics. When you wire the physics into the learning signal itself, the agent has to play by the rules.

The LCE reward is a clean example of this principle. The agent cannot fake stability. The Lyapunov exponent either goes down or it does not.

Related work shows the approach is gaining traction. A 2024 preprint called SuPLE applied Lyapunov-based rewards to robot learning, though it has not yet accumulated citations [P2]. At ICML 2025, researchers presented an RL-based method for discovering Lyapunov functions analytically, showing that the combination of reinforcement learning and stability theory is an active research frontier [P5]. On the tooling side, a CUDA kernel for computing Lyapunov exponent spectra is available on HuggingFace, meaning the computational building blocks for this kind of work are increasingly accessible [P3].

The strictly upright result, if it holds up, suggests that RL agents guided by physical principles can find control strategies that humans have not. The Kapitza pendulum took decades of physics to understand. The beyond-Kapitza result took an agent and a well-chosen reward signal.

What it means for business

For robotics companies and control engineers, the idea of replacing hand-tuned reward functions with a physics-based measure like the LCE could simplify a painful part of the development pipeline. Designing rewards for robotic control is one of the most time-consuming tasks in applied RL. A reward grounded in a physical property of the system, rather than a heuristic, could reduce the trial-and-error cycle.

A two-person robotics startup building a balancing controller for a drone or a bipedal robot could, in principle, use the LCE of the system's dynamics as a training signal instead of crafting a custom reward for each new platform. The same reward function would apply to any system where stability is the goal.

For simulation software vendors, the result points toward a product category: physics-informed reward libraries that ship with simulation environments, so users can train agents without becoming reward-design experts.

The caveat is that this is a single preprint, self-reported, on a well-studied toy problem. The inverted pendulum is the "hello world" of control theory. Whether the LCE reward scales to systems with many degrees of freedom, such as a humanoid robot or a fluid dynamics controller, is an open question that the paper does not address.

What we don't know yet

The preprint has not been peer-reviewed, and every claim about the agent's performance comes from the authors' own reporting [S1]. No third party has replicated the strictly upright result.

The paper is categorised under computer science, not physics, which means the physics community has not yet weighed in on whether the beyond-Kapitza result is a genuine new regime or a consequence of the simulation setup. The Kapitza pendulum is traditionally an oscillatory stabilisation: the pendulum stays upright because it is wobbling. A strictly upright, non-oscillating pendulum stabilised by vertical motion alone would need a physical explanation that the preprint does not appear to provide.

The broader question is whether LCE-based rewards generalise. The inverted pendulum has one degree of freedom. Real systems, from robot arms to power grids, have many. Computing the full Lyapunov spectrum for a high-dimensional system is expensive, and whether the reward remains informative as dimensionality grows is unknown.

The next concrete signal to watch for is either peer review of this preprint or an independent replication from a physics or robotics lab. NeurIPS 2026, which has already received at least one physics-reasoning benchmark submission [P4], could surface related work that tests whether physics-informed rewards hold up on harder problems.

If you want to catch that next result the moment it lands, subscribe and we will have it in your inbox before the coffee gets cold.

Sources

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.