Ground states for a class of critical quasilinear coupled superlinear elliptic systems

Abstract In this work we consider the following class of quasilinear coupled systems − Δ u + a ( x ) u − Δ ( u 2 ) u = g ( u ) + θ α λ ( x ) | u | α − 2 u | v | β , x ∈ R N , − Δ v + b ( x ) v − Δ ( v 2 ) v = h ( v ) + θ β λ ( x ) | v | β − 2 v | u | α , x ∈ R N , where N ≥ 3 and a , b : R N → R are positive potentials, λ : R N → R is a nonnegative continuous function, θ > 0 and α , β > 2 satisfying α + β 2 ⋅ 2 ∗ . On the nonlinear terms we assume that g , h are in C 1 class which are superlinear functions at infinity and at the origin. We deal with nonlinearities g and h being subcritical or critical. The coupling term is a subcritical function which is superlinear at infinity. Our main theorem is stated without the well known Ambrosetti–Rabinowitz condition at infinity. Using a change of variable, we turn the quasilinear coupled system into a nonlinear coupled system, where we establish a variational approach based on Nehari method.