Markov Decision Processes in Practice

Part I. General Theory

1. One-Step Improvement Ideas and Computational Aspects
Henk Tijms

2. Value Function Approximation in Complex Queueing Systems
Sandjai Bhulai

3. Approximate Dynamic Programming by Practical Examples
Martijn R. K. Mes, Arturo Pérez Rivera

4. Server Optimization of Infinite Queueing Systems
András Mészáros, Miklós Telek

5. Structures of Optimal Policies in MDPs with Unbounded Jumps: The State of Our Art
H. Blok, F. M. Spieksma

Part II. Healthcare

6. Markov Decision Processes for Screening and Treatment of Chronic Diseases
Lauren N. Steimle, Brian T. Denton

7. Stratified Breast Cancer Follow-Up Using a Partially Observable MDP
J. W. M. Otten, A. Witteveen, I. M. H. Vliegen, S. Siesling, J. B. Timmer, M. J. IJzerman

8. Advance Patient Appointment Scheduling
Antoine Sauré, Martin L. Puterman

9. Optimal Ambulance Dispatching
C. J. Jagtenberg, S. Bhulai, R. D. Mei

10. Blood Platelet Inventory Management
Rene Haijema, Nico M. van Dijk, Jan Wal

Part III. Transportation

11. Stochastic Dynamic Programming for Noise Load Management
T. R. Meerburg, Richard J. Boucherie, M. J. A. L. Kraaij

12. Allocation in a Vertical Rotary Car Park
M. Fackrell, P. Taylor

13. Dynamic Control of Traffic Lights
Rene Haijema, Eligius M. T. Hendrix, Jan Wal

14. Smart Charging of Electric Vehicles
Pia L. Kempker, Nico M. van Dijk, Werner Scheinhardt, Hans Berg, Johann Hurink

Part IV. Production

15. Analysis of a Stochastic Lot Scheduling Problem with Strict Due-Dates
Nicky D. Foreest, Jacob Wijngaard

16. Optimal Fishery Policies
Eligius M. T. Hendrix, Rene Haijema, Diana Dijk

17. Near-Optimal Switching Strategies for a Tandem Queue
Daphne Leeuwen, Rudesindo Núñez-Queija

Part V. Communications

18. Wireless Channel Selection with Restless Bandits
Julia Kuhn, Yoni Nazarathy

19. Flexible Staffing for Call Centers with Non-stationary Arrival Rates
Alex Roubos, Sandjai Bhulai, Ger Koole

20. MDP for Query-Based Wireless Sensor Networks
Mihaela Mitici

Part VI. Financial Modeling

21. Optimal Portfolios and Pricing of Financial Derivatives Under Proportional Transaction Costs
Jörn Sass, Manfred Schäl

Nyckelord: Business and Management, Operation Research/Decision Theory, Operations Research, Management Science, Probability Theory and Stochastic Processes