Informatics and Applications
2022, Volume 16, Issue 2, pp 75-84
ON THE ANALYTICAL STRUCTURE OF SOME KINDS OF TARGET FUNCTIONALS ASSOCIATED WITH THE CONTROL PROBLEMS OF SEMI-MARKOV STOCHASTIC PROCESSES
Abstract
The present author investigates the analytical structure of three kinds of functionals from a controllable semi-Markov process with a finite set of states. It is proved that all these mathematical objects can be represented in the form of a fractional-linear integral functional defined on a finite set of probability measures that determine the control strategy of the corresponding semi-Markov process. For each of these functionals, explicit representations for the integrand functions of the numerator and denominator through the initial probabilistic characteristics of the controlled semi-Markov process are obtained. This result allows one to reduce the problem of optimal control of a semi-Markov process with a particular target functional to the problem of investigation on the global extremum of a given function of a finite number of variables.
[+] References (13)
- Mine, H., and S. Osaki. 1970. Markovian decision processes. New York, NY: Elsevier. 142 p.
- Jewell, W. S. 1963. Markov-renewal programming. I, II. Oper. Res. 11(6):938-971.
- Luque-Vasquez, F, and Î. Herndndez-Lerma. 1999. Semi-Markov control models with average costs. Appl. Math. 26(3):315-331.
- Vega-Amaya, O., and F. Luque-Vasquez. 2000. Sample- path average cost optimality for semi-Markov control processes on Borel spaces: Unbounded costs and mean holding times. Appl. Math. 27(3):343-367.
- Gnedenko, B. V., ed. 1983. Voprosy matematicheskoy teorii nadezhnosti [Questions of mathematics reliability theory]. Moscow: Radio i svyaz'. 376 p.
- Barzilovich, E. Yu., and V. A. Kashtanov. 1971. Nekotorye matematicheskie voprosy teorii obsluzhivaniya slozhnykh sistem [Some mathematical questions in theory of complex systems maintenance]. Moscow: Sovetskoe radio. 272 p.
- Snurkov, P. V., and A. V. Ivanov. 2015. Analysis of a discrete semi-Markov model of continuous inventory control with periodic interruptions of consumption. Discrete Math. 25(1):59-67.
- Halmos, P. 1950. Measure theory. Princeton, NJ: Van Nostrand. 304 p.
- Korolyuk, V. S., and A. F Turbin. 1976. Polumarkovskie protsessy i ikh prilozheniya [Semi-Markov processes and their applications]. Kiev: Naukova Dumka. 184 p.
- Janssen, J., and R. Manca. 2006. Applied semi-Markov processes. New York, NY: Springer. 309 p.
- Shnurkov, P.V. 2016. Solution of the unconditional extremum problem for a linear-fractional integral functional on a set of probability measures. Dokl. Math. 94(2):550- 554.
- Shnurkov, P.V., A. K. Gorshenin, and V.V. Belousov
2016. Analiticheskoe reshenie zadachi optimal'nogo upravleniya polumarkovskim protsessom s konechnym mnozhestvom sostoyaniy [An analytic solution of the optimal control problem for a semi-Markov process with
a finite set of states]. Informatika i ee Primeneniya - Inform. Appl. 10(4):72-88.
- Shnurkov, P. 2020. Solution of the unconditional extremum problem for a linear-fractional integral functional on a set of probability measures and its applications in the theory of optimal control of semi-Markov processes. arXiv.org. 26 p. Available at: https://arxiv.org/ abs/2001.06424 (accessed April29, 2022).
[+] About this article
Title
ON THE ANALYTICAL STRUCTURE OF SOME KINDS OF TARGET FUNCTIONALS ASSOCIATED WITH THE CONTROL PROBLEMS OF SEMI-MARKOV STOCHASTIC PROCESSES
Journal
Informatics and Applications
2022, Volume 16, Issue 2, pp 75-84
Cover Date
2022-07-25
DOI
10.14357/19922264220210
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
stochastic control models; optimal control of semi-Markovian processes; partial-linear integral functional; basic function of partial-linear integral functional
Authors
P. V. Shnurkov 
Author Affiliations
National Research University Higher School of Economics, 34 Tallinskaya Str., Moscow 123458, Russian Federation
|