Design heuristic for parallel many server systems

Abstract We study a parallel queueing system with multiple types of servers and customers. A bipartite graph describes which pairs of customer-server types are compatible. We consider the service policy that always assigns servers to the first, longest waiting compatible customer, and that always assigns customers to the longest idle compatible server if on arrival multiple compatible servers are available. For a general renewal stream of arriving customers, general service time distributions that depend both on customer and on server types, and general customer patience distributions, the behavior of such systems is very complicated. Key quantities for their performance are the matching rates, the fraction of services for each pair of compatible customer-server. Calculation of these matching rates in general is intractable, it depends on the entire shape of service time distributions. We suggest through a heuristic argument that if the number of servers becomes large, the matching rates are well approximated by matching rates calculated from the tractable bipartite infinite matching model. We present simulation evidence to support this heuristic argument, and show how this can be used to design systems with desired performance requirements.