Solitary wave solutions and modulational instability in a system of coupled complex Newell–Segel–Whitehead equations

Abstract By virtue of the modulational instability (MI) and phase amplitude ansatz approach, a system of coupled complex Newell–Segel–Whitehead equations (NSWEs), which describes isotropic systems near a subcritical oscillatory instability, is investigated. The constraints that allow the MI procedure to transform the system under consideration into a study of the roots of a polynomial equation of the fourth degree are obtained. A number of examples are analyzed graphically, to overcome the complexity of the dispersion relation and its dependence on many parameters. The existence of a variety of MI gain spectrum is observed. The influence of the cubic-quintic nonlinearity and the magnitude of the plane wave solutions of the system on the MI are also analyzed. Various novel solitary-wave solutions of the system, such as bright-bright, dark-dark, and dark-bright wave solutions, are analytically obtained using direct approach under some constraint conditions.