Informatics and Applications
2020, Volume 14, Issue 3, pp 101-108
ON THE CONCEPT OF A STOCHASTIC MODEL WITH CONTROL AT THE MOMENTS OF THE PROCESS AT THE BORDER OF A PRESENTED SUBSET OF MULTIPLE STATES
- P. V. Shnurkov
- D. A. Novikov
Abstract
The work is devoted to the creation and analysis of the general concept of a special stochastic model with controls. The main feature of the model is that the control actions are carried out at times when a stochastic process describing the system under research reaches the boundary of a given subset of the set of states. The control action itself consists in transferring the process from the boundary to one of the internal states of a given subset. In this case, the internal states are interpreted as acceptable and the boundary ones as unacceptable. Control actions are described by a set of discrete probability distributions depending on the boundary state number. Such a set defines a control strategy. The problem of optimal control is formalized as the problem of finding a control strategy that delivers a global extremum to a certain stationary cost-effectiveness indicator, which in terms of its economic content represents the average specific profit arising from a long evolution of the system. The posed problem of optimal control is proposed to be alled the tuning problem. The paper notes that this stochastic model and the corresponding setup problem can be used to study many real phenomena occurring in economic and technical systems. As an example of such a real phenomenon, interventions in the foreign exchange market of the Russian Federation are considered.
[+] References (12)
- Lopatnikov, L.I. 2003. Ekonomiko-matematicheskiy slovar': slovar' sovremennoy ekonomicheskoy nauki [Eco-nomic and mathematical dictionary: Modern economic science dictionary]. Moscow: Delo. 520 p.
- OECD - Organisation for Economic Cooperation and Development. 2008. OECD glossary of statistical terms. OECD Publishing. 288 p. Available at: https://stats.oecd.org/glossary/ (accessed July 30, 2020).
- 264-FZ. 2015. O razvitii sel'skogo khozyaystva: federal'nyy zakon [About the development of agriculture: The Russian Federal Law]. Available at: http://www.kremlin. ru/acts/bank/24837 (accessed July 30, 2020).
- Kemeny, J., and J. Snell. 1976. Finite Markov chains. Prentice-Hall. 232 p.
- Korolyuk, V. S., and A. F. Turbin. 1976. Polumarkovskie protsessy i ikh prilozheniya [Semi-Markov processes and their applications]. Kiev: Naukova dumka. 184 p.
- Janssen, J., and R. Manca. 2006. Applied semi-Markov processes. New York, NY: Springer. 309 p.
- Jewell, W. S. 1963. Markov-renewal programming. I, II. Oper. Res. 11(6):938-971.
- Mine, H., and S. Osaki. 1970. Markovian decision process-es. New York, NY: Elsevier. 142 p.
- Shnurkov, PV. 2017. Optimal control problem in a stochastic model with periodic hits on the boundary of a given subset of the state set (tuning problem). arXiv.org.
16 p. Available at: https://arxiv.org/abs/1709.03442v1 (accessed July 30, 2020).
- Shnurkov, P V., and D. A. Novikov. 2018. Analysis of the problem of intervention control in the economy on the basis of solving the problem of tuning. arXiv.org. 15 p. Available at: https://arxiv.org/abs/1811.10993 (accessed July 30, 2020).
- Shnurkov, P.V. 2016. Solution of the unconditional extremum problem for a linear fractional integral functional on a set of probability measures. Dokl. Math. 94(2):550- 554.
- Shnurkov, P.V., A. K. Gorshenin, and V.V. Belousov.
2016. Analiticheskoe reshenie zadachi optimal'nogo upravleniya polumarkovskim protsessom s konechnym mnozhestvom sostoyaniy [An analytic solution of the optimal control problem for a semi-Markov process with a finite set of states]. Informatika i ee Primeneniya - Inform. Appl. 10(4):72-88.
[+] About this article
Title
ON THE CONCEPT OF A STOCHASTIC MODEL WITH CONTROL AT THE MOMENTS OF THE PROCESS AT THE BORDER OF A PRESENTED SUBSET OF MULTIPLE STATES
Journal
Informatics and Applications
2020, Volume 14, Issue 3, pp 101-108
Cover Date
2020-09-30
DOI
10.14357/19922264200315
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
control in stochastic systems; Markov controlled processes; semi-Markov controlled processes; stochastic tuning problem
Authors
P. V. Shnurkov  and D. A. Novikov
Author Affiliations
National Research University Higher School of Economics, 34 Tallinskaya Str., Moscow 123458, Russian Federation
|