Success-Odds: An improved Win-Ratio

Multiple and combined endpoints involving also non-normal outcomes appear in many clinical trials in various areas in medicine. In some cases, the outcome can be observed only on an ordinal or dichotomous scale. Then the success of two therapies is assessed by comparing the outcome of two randomly selected patients from the two therapy groups by 'better', 'equal' or 'worse'. These outcomes can be described by the probabilities $p^-=P(X ~Y)$. For a clinician, however, these quantities are less intuitive. Therefore, Noether (1987) introduced the quantity $\lambda=p^+ / p^-$ assuming continuous distributions. The same quantity was used by Pocock et al. (2012) and by Wang and Pocock (2016) also for general non-normal outcomes and has been called 'win-ratio' $\lambda_{WR}$. Unlike Noether (1987), Wang and Pocock (2016) explicitly allowed for ties in the data. It is the aim of this manuscript to investigate the properties of $\lambda_{WR}$ in case of ties. It turns out that it has the strange property of becoming larger if the data are observed less accurately, i.e. include more ties. Thus, in case of ties, the win-ratio looses its appealing property to describe and quantify an intuitive and well interpretable treatment effect. Therefore, a slight modification of $\lambda_{WR} = \theta / (1-\theta)$ is suggested, namely the so-called 'success-odds' where $\theta=p^+ + \frac12 p_0$ is called a success of a therapy if $\theta>\frac12$. In the case of no ties, $\lambda_{SO}$ is identical to $\lambda_{WR}$. A test for the hypothesis $\lambda_{SO}=1$ and range preserving confidence intervals for $\lambda_{SO}$ are derived. By two counterexamples it is demonstrated that generalizations of both the win-ratio and the success-odds to more than two treatments or to stratified designs are not straightforward and need more detailed considerations.