Verena Wolf

Saarland University

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Michael Backenköhler

Saarland University

Luca Bortolussi

University of Trieste

Gerrit Großmann

German Research Centre for Artificial Intelligence

Linar Mikeev

Saarland University

Charalampos Kyriakopoulos

Bristol-Myers Squibb (Germany)

Thomas A. Henzinger

Institute of Science and Technology Austria

Pascal Giehr

Ludwig-Maximilians-Universität München

David Spieler

Munich University of Applied Sciences

Alexander Lück

Saarland University

Maria Mateescu

Institute of Science and Technology Austria

Cooperative Institutions

Saarland University

RWTH Aachen University

Max Planck Society

Friedrich-Alexander-Universität Erlangen-Nürnberg

École Polytechnique Fédérale de Lausanne

Technical University of Munich

Centre National de la Recherche Scientifique

Max Planck Institute for Informatics

Max Planck Institute for Software Systems

University of Edinburgh

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

FORMALISMS FOR SPECIFYING MARKOVIAN POPULATION MODELS

International Journal of Foundations of Computer Science (2011)

Thomas A. Henzinger Barbara Jobstmann Verena Wolf

In this survey, we compare several languages for specifying Markovian population models such as queuing networks and chemical reaction networks. All these languages — matrix descriptions, stochastic Petri nets, stoichiometric equations, stochastic process algebras, and guarded command models — describe continuous-time Markov chains, but they differ according to important properties, such as compositionality, expressiveness and succinctness, executability, and ease of use. Moreover, they provide different support for checking the well-formedness of a model and for analyzing a model.

Principle of compositionality

Succinctness

Stochastic Petri net

Process calculus

10.1142/s0129054111008441

Cite

Citations (33)

Real-Time Decision Making for a Car Manufacturing Process Using Deep Reinforcement Learning

2018 Winter Simulation Conference (WSC) (2020)

Timo P. Gros Joschka Gros Verena Wolf

Computer simulations of manufacturing processes are in widespread use for optimizing production planning and order processing. If unforeseeable events are common, real-time decisions are necessary to maximize the performance of the manufacturing process. Pre-trained AI-based decision support offers promising opportunities for such time-critical production processes. Here, we explore the effectiveness of deep reinforcement learning for real-time decision making in a car manufacturing process. We combine a simulation model of a central production part, the line buffer, with deep reinforcement learning algorithms, in particular with deep Q-Learning and Monte Carlo tree search. We simulate two different versions of the buffer, a single-agent and a multi-agent one, to generate large amounts of data and train neural networks to represent near-optimal strategies. Our results show that deep reinforcement learning performs extremely well and the resulting strategies provide near-optimal decisions in real-time, while alternative approaches are either slow or give strategies of poor quality.

Production line

10.1109/wsc48552.2020.9383884

Cite

Citations (10)

A Stochastic Model for the Formation of Spatial Methylation Patterns

arXiv (Cornell University) (2017)

Alexander Lück Pascal Giehr Jörn Walter Verena Wolf

DNA methylation is an epigenetic mechanism whose important role in development has been widely recognized. This epigenetic modification results in heritable changes in gene expression not encoded by the DNA sequence. The underlying mechanisms controlling DNA methylation are only partly understood and recently different mechanistic models of enzyme activities responsible for DNA methylation have been proposed. Here we extend existing Hidden Markov Models (HMMs) for DNA methylation by describing the occurrence of spatial methylation patterns over time and propose several models with different neighborhood dependencies. We perform numerical analysis of the HMMs applied to bisulfite sequencing measurements and accurately predict wild-type data. In addition, we find evidence that the enzymes' activities depend on the left 5' neighborhood but not on the right 3' neighborhood.

Bisulfite sequencing

Source

Cite

Citations (1)

DSMC Evaluation Stages: Fostering Robust and Safe Behavior in Deep Reinforcement Learning – Extended Version

ACM Transactions on Modeling and Computer Simulation (2023)

Timo P. Gros Joschka Groß Daniel Höller Jörg Hoffmann Michaela Klauck

Neural networks (NN) are gaining importance in sequential decision-making. Deep reinforcement learning (DRL), in particular, is extremely successful in learning action policies in complex and dynamic environments. Despite this success, however, DRL technology is not without its failures, especially in safety-critical applications: (i) the training objective maximizes average rewards, which may disregard rare but critical situations and hence lack local robustness; (ii) optimization objectives targeting safety typically yield degenerated reward structures, which, for DRL to work, must be replaced with proxy objectives. Here, we introduce a methodology that can help to address both deficiencies. We incorporate evaluation stages (ES) into DRL, leveraging recent work on deep statistical model checking (DSMC), which verifies NN policies in Markov decision processes. Our ES apply DSMC at regular intervals to determine state space regions with weak performance. We adapt the subsequent DRL training priorities based on the outcome, (i) focusing DRL on critical situations and (ii) allowing to foster arbitrary objectives. We run case studies on two benchmarks. One of them is the Racetrack, an abstraction of autonomous driving that requires navigating a map without crashing into a wall. The other is MiniGrid, a widely used benchmark in the AI community. Our results show that DSMC-based ES can significantly improve both (i) and (ii).

Robustness

Benchmark (surveying)

10.1145/3607198

Cite

Citations (3)

H(O)TA: estimation of DNA methylation and hydroxylation levels and efficiencies from time course data

arXiv (Cornell University) (2016)

Charalampos Kyriakopoulos Pascal Giehr Verena Wolf

Methylation and hydroxylation of cytosines to form 5-methylcytosine (5mC) and 5-droxymethylcytosine (5hmC) belong to the most important epigenetic modifications and their vital role in the regulation of gene expression has been widely recognized. Recent experimental techniques allow to infer methylation and hydroxylation levels at CpG dinucleotides but require a sophisticated statistical analysis to achieve accurate estimates.

CpG site

Hydroxylation

10.48550/arxiv.1606.07807

Cite

Citations (0)

Tracking the Race Between Deep Reinforcement Learning and Imitation Learning -- Extended Version

arXiv (Cornell University) (2020)

Timo P. Gros Daniel Höller Jörg Hoffmann Verena Wolf

Learning-based approaches for solving large sequential decision making problems have become popular in recent years. The resulting agents perform differently and their characteristics depend on those of the underlying learning approach. Here, we consider a benchmark planning problem from the reinforcement learning domain, the Racetrack, to investigate the properties of agents derived from different deep (reinforcement) learning approaches. We compare the performance of deep supervised learning, in particular imitation learning, to reinforcement learning for the Racetrack model. We find that imitation learning yields agents that follow more risky paths. In contrast, the decisions of deep reinforcement learning are more foresighted, i.e., avoid states in which fatal decisions are more likely. Our evaluations show that for this sequential decision making problem, deep reinforcement learning performs best in many aspects even though for imitation learning optimal decisions are considered.

Tracking (education)

10.48550/arxiv.2008.00766

Cite

Citations (1)

SABRE: A Tool for Stochastic Analysis of Biochemical Reaction Networks

arXiv (Cornell University) (2010)

Frédéric Didier Thomas A. Henzinger Maria Mateescu Verena Wolf

The importance of stochasticity within biological systems has been shown repeatedly during the last years and has raised the need for efficient stochastic tools. We present SABRE, a tool for stochastic analysis of biochemical reaction networks. SABRE implements fast adaptive uniformization (FAU), a direct numerical approximation algorithm for computing transient solutions of biochemical reaction networks. Biochemical reactions networks represent biological systems studied at a molecular level and these reactions can be modeled as transitions of a Markov chain. SABRE accepts as input the formalism of guarded commands, which it interprets either as continuous-time or as discrete-time Markov chains. Besides operating in a stochastic mode, SABRE may also perform a deterministic analysis by directly computing a mean-field approximation of the system under study. We illustrate the different functionalities of SABRE by means of biological case studies.

Uniformization (probability theory)

Formalism (music)

Source

Cite

Citations (1)

Lumping the Approximate Master Equation for Multistate Processes on Complex Networks

arXiv (Cornell University) (2018)

Gerrit Großmann Charalampos Kyriakopoulos Luca Bortolussi Verena Wolf

Complex networks play an important role in human society and in nature. Stochastic multistate processes provide a powerful framework to model a variety of emerging phenomena such as the dynamics of an epidemic or the spreading of information on complex networks. In recent years, mean-field type approximations gained widespread attention as a tool to analyze and understand complex network dynamics. They reduce the model's complexity by assuming that all nodes with a similar local structure behave identically. Among these methods the approximate master equation (AME) provides the most accurate description of complex networks' dynamics by considering the whole neighborhood of a node. The size of a typical network though renders the numerical solution of multistate AME infeasible. Here, we propose an efficient approach for the numerical solution of the AME that exploits similarities between the differential equations of structurally similar groups of nodes. We cluster a large number of similar equations together and solve only a single lumped equation per cluster. Our method allows the application of the AME to real-world networks, while preserving its accuracy in computing estimates of global network properties, such as the fraction of nodes in a state at a given time.

Fraction (chemistry)

Complex system

Network Dynamics

10.48550/arxiv.1804.02981

Cite

Citations (0)

Equivalences on Phase Type Processes

Verena Wolf

In this thesis, we introduce Phase Type Processes (PTPs), a novel stochastic modeling approach that can express probabilistic and nondeterministic choices as well as random delays following phase type distributions, a generalization of exponential distributions. Action-labeled transitions are used to react on external stimuli and they are clearly separated from phase type transitions. The semantics of PTPs are defined in terms of path probabilities with respect to schedulers that resolve nondeterministic choices based on the timed process history. The main emphasis of this work is to analyze a variety of notions of equivalence for PTPs and classify them with respect to their distinguishing power. Amongst others, we define bisimulation, trace and testing equivalence as well as extensions of failure trace equivalence. Moreover, the contribution includes a discussion of parallel composition in the context of a partial memoryless property and the examination of a mapping from PTPs to the subclass of single phased processes in which all random delays are exponentially distributed.

Nondeterministic algorithm

TRACE (psycholinguistics)

Bisimulation

Source

Cite

Citations (3)

INFINITE LEVEL-DEPENDENT QBD PROCESSES AND MATRIX-ANALYTIC SOLUTIONS FOR

Tuǧrul Dayar Werner Sandmann David Spieler Verena Wolf

Systems of stochastic chemical kinetics are modeled as infinite level-dependent quasi birth-and-death (LDQBD) processes. For these systems, in contrast to many other applications, levels have an increasing number of states as the level number increases and the probability mass may reside arbitrarily far away from lower levels. Ideas from Lyapunov theory are combined with existing matrix-analytic formulations to obtain accurate approximations to the stationary probability distribution when the infinite LDQBD process is ergodic. Results of numerical experiments on a set of problems are provided.

Stationary ergodic process

Matrix (chemical analysis)

Stationary distribution

Source

Cite

Citations (0)