Nonzero-Sum Markov Games with Finite Random Horizon

  • Authors

    • R. Israel Ortega-Gutiérrez Benemérita Universidad Autónoma de Puebla
    • Hugo Cruz-Suárez Benemérita Universidad Autónoma de Puebla
    • Adriana González-Quiroz Benemérita Universidad Autónoma de Puebla
    https://doi.org/10.14419/etx28e49
  • Dynamic programming, Nonzero-Sum stochastic games, Random horizon

  • Abstract

    This manuscript aims to establish the existence of a Nash equilibrium in nonzero-sum Markov games with a random horizon of finite support. The proof relies on dynamic programming techniques adapted to the stochastic nature of the horizon and the interaction between players. Introducing a random horizon with finite support allows for a more realistic modeling of scenarios in which the duration of the game is uncertain and influenced by exogenous random events. To illustrate the applicability of the theoretical results, we examine a dynamic game version of the Great Fish War, which models competition over a renewable resource under uncertainty about the duration of exploitation. This framework enhances the applicability of Markov game theory to decision-making contexts where time horizons are unpredictable.

  • References

    1. Bhabak, A., & Saha, S. (2022). “Risk-sensitive semi-Markov decision problems with discounted cost and general utilities”. Statistics & Probability Letters, 184, 109408. https://doi.org/10.1016/j.spl.2022.109408
    2. Camilo-Garay, C., Ortega-Gutiérrez, R. I., & Cruz-Suárez, H. (2020). “Optimal strategies for a fishery model applied to utility functions”. Mathematical Biosciences and Engineering, 18(1), 518–529. https://doi.org/10.3934/mbe.2021028
    3. Crespo-Guerrero, J. M., & Casado Izquierdo, J. M. (2023). “Pesca y comercialización del pulpo en Yucatán: ¿un proceso extractivista impulsado por la Unión Europea?” Geográfica Venezolana, 64(2), 301–319. https://doi.org/10.53766/RGV/2024.64.2.06
    4. Cruz-Suárez, H., Ilhuicatzi-Roldán, R., & Montes-de Oca, R. (2013). “Markov decision processes on borel spaces with total cost and random horizon”. Journal of Optimization Theory and Applications, 162, 329–346. https://doi.org/10.1007/s10957-012-0262-8
    5. Ekström, E., & Wang, Y. (2024). “Stopping problems with an unknown state”. Journal of Applied Probability, 61(2), 515–528. https://doi.org/10.1017/jpr.2023.52
    6. Gao, X., & Zhou, X. Y. (2024). “Logarithmic regret bounds for continuous-time average-reward Markov decision processes”. SIAM Journal on Control and Optimization, 62(5), 2529–2556. https://doi.org/10.48550/arXiv.2205.11168
    7. González-Sánchez, D., Luque-Vásquez, F., & Minjárez-Sosa, J. A. (2019). “Zero-sum markov games with random state-actions dependent discount factors: existence of optimal strategies”. Dynamic Games and Applications, 9, 103–121. https://doi.org/10.1007/s13235-018-0248-8
    8. Guo, X., & Wen, X. (2025). “Nonstationary nonzero-sum Markov games under a probability criterion”. arXiv preprint arXiv:2505.10126. https://arxiv.org/abs/2505.10126
    9. Hernández-Lerma, O., & Lasserre, J. B. (1996). “Discrete-time Markov control processes: basic optimality criteria”, volumen 30. Springer Science & Business Media. https://doi.org/10.1007/978-1-4612-0729-0
    10. Jáskiewicz, A., & Nowak, A. S. (2016). “Non-zero-sum stochastic games”. In Handbook of dynamic game theory, pages 281–344. Springer. https://doi.org/10.1007/978-3-319-27335-8_33-1
    11. Levhari, D., & Mirman, L. J. (1977). “Savings and consumption with an uncertain horizon”. Journal of Political Economy, 85(2), 265–281. https://doi.org/10.1086/260562
    12. Levhari, D., & Mirman, L. J. (1980). “The great fish war: an example using a dynamic Cournot-Nash solution”. The Bell Journal of Economics, pages 322–334. https://doi.org/10.2307/3003416
    13. Miller, K. A. (2003). “North American Pacific salmon: A case of fragile cooperation”. FAO Fisheries Report, pages 105–122.
    14. Minjárez-Sosa, J. A. (2020). “Zero-Sum discrete-time Markov games with unknown disturbance distribution: discounted and average criteria”. Springer Nature. https://doi.org/10.1007/978-3-030-35720-7
    15. Missios, P. C., & Plourde, C. (1996). “The Canada-European Union turbot war: A brief game theoretic analysis”. Canadian Public Policy/Analyse de Politiques, pages 144–150. https://doi.org/10.2307/3551905
    16. Nowak, A. (2006). “A note on an equilibrium in the great fish war game”. Economics Bulletin, 17(2), 1–10.
    17. Ortega-Gutiérrez, R. I., Montes-de Oca, R., & Cruz-Suárez, H. (2024). “Characterization of a Cournot–Nash equilibrium for a fishery model with fuzzy utilities”. Journal of Mathematics, 2024(1), 6885051. https://doi.org/10.1155/2024/6885051
    18. Shapley, L. S. (1953). “Stochastic games”. Proceedings of the national academy of sciences, 39(10), 1095–1100. https://doi.org/10.1073/pnas.39.10.1095
    19. Wang, Y. (2023). “Optimal stopping, incomplete information, and stochastic games”. Doctoral dissertation, Department of Mathematics, Uppsala University.

  • Downloads

  • How to Cite

    Ortega-Gutiérrez, R. I., Cruz-Suárez, H., & González-Quiroz, A. (2025). Nonzero-Sum Markov Games with Finite Random Horizon. International Journal of Advanced Mathematical Sciences, 11(2), 17-24. https://doi.org/10.14419/etx28e49