Search results
Results From The WOW.Com Content Network
Backward induction is the process of determining a sequence of optimal choices by reasoning from the endpoint of a problem or situation back to its beginning using individual events or actions. [1] Backward induction involves examining the final point in a series of decisions and identifying the optimal process or action required to arrive at ...
Bellman's contribution is remembered in the name of the Bellman equation, a central result of dynamic programming which restates an optimization problem in recursive form. Bellman explains the reasoning behind the term dynamic programming in his autobiography, Eye of the Hurricane: An Autobiography: I spent the Fall quarter (of 1950) at RAND ...
Bellman showed that a dynamic optimization problem in discrete time can be stated in a recursive, step-by-step form known as backward induction by writing down the relationship between the value function in one period and the value function in the next period. The relationship between these two value functions is called the "Bellman equation".
Stochastic dynamic programming deals with problems in which the current period reward and/or the next period state are random, i.e. with multi-stage stochastic systems. The decision maker's goal is to maximise expected (discounted) reward over a given planning horizon.
The one-shot deviation principle is very important for infinite horizon games, in which the backward induction method typically doesn't work to find SPE. In an infinite horizon game where the discount factor is less than 1, a strategy profile is a subgame perfect equilibrium if and only if it satisfies the one-shot deviation principle.
For example, the dynamic programming algorithms described in the next section require an explicit model, and Monte Carlo tree search requires a generative model (or an episodic simulator that can be copied at any state), whereas most reinforcement learning algorithms require only an episodic simulator.
The standard solution to the centipede game is determined by backward induction. According to this method, if Bob reaches his final decision, he will prefer to keep a larger share of a smaller pot to the smaller share of a larger pot, so he will end the game instead of expanding the pot.
The meaning of "backward induction" in game theory is closely related to its meaning in dynamic programming. I see no reason this page couldn't include sections on both topics. Moving that discussion to the Bellman equation page, section "solution methods" would likely make the Bellman equation page far too long (at least once the Bellman ...