Discrete-Time Markov Chains

Part I. Prologue and Preliminaries

1. Introduction, Overview, and Examples

2. Mathematical Preliminaries

Part II. Asymptotic Properties

3. Asymptotic Expansions

4. Occupation Measures

5. Exponential Bounds

6. Interim Summary and Extensions

Part III. Applications

7. Stability of Dynamic Systems

8. Filtering

9. Markov Decision Processes

10. LQ Controls

11. Mean-Variance Controls

12. Production Planning

13. Stochastic Approximation

DRM-restrictions

Printing: not available
Clipboard copying: not available

Keywords: MATHEMATICS / General MAT000000