IMPORTANT:This material is provided since some find it useful. It represents course material from the 1990s. I no longer keep this material up to date. Please DO NOT EMAIL ME ON THIS MATERIAL. In particular, I regret that I have no time to clarify material or provide solutions to any of the questions or projects. I am tempted to just remove this material. Please don't abuse my decision to keep this online.
Next: Uncertain Payoffs
Michael A. Trick
Mini II, 1995
In deterministic dynamic programming, given a state and a decision, both the immediate payoff and next state are known. If we know either of these only as a probability function, then we have a stochastic dynamic program. The basic ideas of determining stages, states, decisions, and recursive formulae still hold: they simply take on a slightly different form.