THE NUMBER OF ZEROS OF ABELIAN INTEGRALS FOR A PERTURBATION OF HYPERELLIPTIC HAMILTONIAN SYSTEM WITH DEGENERATED POLYCYCLE

In this paper, we provide a complete study of the zeros of Abelian integrals obtained by integrating the 1-form (α + βx + x2)ydx over the compact level curves of the hyperelliptic Hamiltonian $H(x, y) = \frac{y^2}{2}-\frac{1}{5}x^3(x-1)^2$. Such a family of compact level curves is bounded by a polycycle passing through a nilpotent cusp and a hyperbolic saddle of this hyperelliptic Hamiltonian system, which is not the exceptional family of ovals proposed by Gavrilov and Iliev. It is shown that the least upper bound for the number of zeros of the related hyperelliptic Abelian integral is two, and this least upper bound can be achieved for some values of parameters (α, β). This implies that the Abelian integral still has Chebyshev property for this nonexceptional family of ovals. Moreover, we derive the asymptotic expansion of Abelian integrals near a polycycle passing through a nilpotent cusp and a hyperbolic saddle in a general case.