Sources of systematic error in gravitational-wave measurements of the binary neutron star mass distribution

The binary neutron star (BNS) mass distribution measured with gravitational-wave observations has the potential to reveal information about the dense matter equation of state, supernova physics, the expansion rate of the universe, and tests of General Relativity. As most current gravitational-wave analyses measuring the BNS mass distribution do not simultaneously fit the spin distribution, the implied population-level spin distribution is the same as the spin prior applied when analyzing individual sources. In this work, we demonstrate that introducing a mismatch between the implied and true BNS spin distributions can lead to biases in the inferred mass distribution. This is due to the strong correlations between the measurements of the mass ratio and spin components aligned with the orbital angular momentum for individual sources. We find that applying a low-spin prior which excludes the true spin magnitudes of some sources in the population leads to significantly overestimating the maximum neutron star mass and underestimating the minimum neutron star mass at the population level with as few as six BNS detections. The safest choice of spin prior that does not lead to biases in the inferred mass distribution is one which allows for high spin magnitudes and tilts misaligned with the orbital angular momentum.