Raúl Montes-de-Oca

Universidad Autónoma Metropolitana

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Rolando Cavazos–Cadena

Universidad Autónoma Agraria Antonio Narro

Hugo Cruz−Suárez

Benemérita Universidad Autónoma de Puebla

Enrique Lemus-Rodríguez

Universidad Anáhuac

Patricia Saavedra

Universidad Autónoma Metropolitana

Karel Sladký

Czech Academy of Sciences, Institute of Information Theory and Automation

Karla Carrero-Vera

Benemérita Universidad Autónoma de Puebla

Daniel Cruz-Suárez

Universidad Juárez Autónoma de Tabasco

Francisco Salem-Silva

Universidad Veracruzana

R. Israel Ortega-Gutiérrez

Benemérita Universidad Autónoma de Puebla

Rubén Becerril-Borja

Universidad Autónoma Metropolitana

Cooperative Institutions

Benemérita Universidad Autónoma de Puebla

Universidad Autónoma Metropolitana

Center for Research and Advanced Studies of the National Polytechnic Institute

Instituto Politécnico Nacional

Universidad Autónoma Agraria Antonio Narro

Autonomous University of Tlaxcala

ORCID

Universidad del Centro de México

National Tsing Hua University

Mathematics Research Center

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

Estimates for perturbations of general discounted Markov control chains

Applicationes Mathematicae (2003)

Raúl Montes-de-Oca A. I. Sakhanenko Francisco Salem-Silva

We extend previous results of the same authors ([11]) on the effects of perturbation in the transition probability of a Markov cost chain for discounted Markov control processes. Supposing valid, for each stationary policy, conditions of Lyapunov and Harr

Markov kernel

Markov renewal process

10.4064/am30-3-4

Cite

Citations (8)

Sample-Path Optimal Stationary Policies in Stable Markov Decision Chains with the Average Reward Criterion

Journal of Applied Probability (2015)

Rolando Cavazos–Cadena Raúl Montes-de-Oca Karel Sladký

This paper concerns discrete-time Markov decision chains with denumerable state and compact action sets. Besides standard continuity requirements, the main assumption on the model is that it admits a Lyapunov function ℓ. In this context the average reward criterion is analyzed from the sample-path point of view. The main conclusion is that if the expected average reward associated to ℓ 2 is finite under any policy then a stationary policy obtained from the optimality equation in the standard way is sample-path average optimal in a strong sense.

Sample (material)

10.1017/s0021900200012559

Cite

Citations (1)

Value iteration and action Ɛ-approximation of optimal policies in discounted Markov decision processes

Raúl Montes-de-Oca Enrique Lemus-Rodríguez

It is well-known that in Markov Decision Processes, with a total discounted reward, for instance, it is not always possible to explicitly find the optimal stationary policy f*. But using the Value Iteration, a stationary policy fN such that the optimal discounted rewards of f* and fN are close, for the N-th iteration of the procedure, a question arises: are the actions f*(x) and fN(x) necessarily close for each state x? To our knowledge this question is still largely open. In this paper it is studied when it is possible to stop the value iteration algorithm so that the corresponding maximizer stationary policy fN approximates an optimal policy both in the total discounted reward and in the action space (uniformly over the state space). This kind of results will shed light on important computability issues of great practical interest. In this article the action space is assumed to be a compact set and the reward function bounded. An ergodicity condition on the transition probability law and a structural condition on the reward function are needed. Under these conditions, an upper bound on the number of steps needed in the value iteration algorithm, such that its corresponding maximizer is a uniform approximation of the optimal policy, is obtained.

Ergodicity

Q-learning

Cite

Citations (0)

Discounted Markov Decision Processes with Fuzzy Rewards Induced by Non-fuzzy Systems

Karla Carrero-Vera Hugo Cruz−Suárez Raúl Montes-de-Oca

10.5220/0010231400490059

Cite

Citations (1)

Procesos de Control de Markov Descontados Miopes

DOAJ (DOAJ: Directory of Open Access Journals) (2005)

Raúl Montes-de-Oca

Source

Cite

Citations (0)

Optimal Policies for Payment of Dividends through a Fixed Barrier at Discrete Time

Raúl Montes-de-Oca Patricia Saavedra Gabriel Zacarías-Espinoza Daniel Cruz-Suárez

10.5220/0006193700001482

Cite

Citations (0)

A consumption-investment problem modelled as a discounted Markov decision process

Kybernetika (2011)

Hugo Cruz−Suárez Raúl Montes-de-Oca Gabriel Zacarías

In this paper a problem of consumption and investment is presented as a model of a discounted Markov decision process with discrete-time. In this problem, it is assumed that the wealth is affected by a production function. This assumption gives the investor a chance to increase his wealth before the investment. For the solution of the problem there is established a suitable version of the Euler Equation (EE) which characterizes its optimal policy completely, that is, there are provided conditions which guarantee that a policy is optimal for the problem if and only if it satisfies the EE. The problem is exemplified in two particular cases: for a logarithmic utility and for a Cobb-Douglas utility. In both cases explicit formulas for the optimal policy and the optimal value function are supplied.

Investment

Consumption

Isoelastic utility

Decision problem

Source

Cite

Citations (6)

A Counterexample on Sample-Path Optimality in Stable Markov Decision Chains with the Average Reward Criterion

Journal of Optimization Theory and Applications (2013)

Rolando Cavazos–Cadena Raúl Montes-de-Oca Karel Sladký

Counterexample

Theory of computation

Sample space

Sample (material)

10.1007/s10957-013-0474-6

Cite

Citations (5)

Estimates for perturbations of average markov decision processes with a minimal state and upper bounded by stochastically ordered markov chains

Kybernetika (2005)

Raúl Montes-de-Oca Francisco Salem-Silva

This paper deals with Markov decision processes (MDPs) with real state space for which its minimum is attained, and that are upper bounded by (uncontrolled) stochastically ordered (SO) Markov chains. We consider MDPs with (possibly) unbounded costs, and to evaluate the quality of each policy, we use the objective function known as the average cost. For this objective function we consider two Markov control models IP and P 1 . P and P 1 have the same components except for the transition laws. The transition q of P is taken as unknown, and the transition q 1 of P 1 , as a known approximation of q. Under certain irreducibility, recurrence and ergodic conditions imposed on the bounding SO Markov chain (these conditions give the rate of convergence of the transition probability in t-steps, t = 1,2,... to the invariant measure), the difference between the optimal cost to drive P and the cost obtained to drive P using the optimal policy of P 1 is estimated. That difference is defined as the index of perturbations, and in this work upper bounds of it are provided. An example to illustrate the theory developed here is added.

Bounding overwatch

Markov kernel

Irreducibility

Source

Cite

Citations (9)

A Family of Models for Finite Sequential Games Without a Predetermined Order of Turns

Communications in computer and information science (2017)

Rubén Becerril-Borja Raúl Montes-de-Oca

Variation (astronomy)

10.1007/978-3-319-53982-9_3

Cite

Citations (2)