Abstract: Lyapunov’s first method, extended by V. V. Kozlov to nonlinear mechanical systems, is applied to the study of the instability of the position of equilibrium of a mechanical system moving in the field of conservative and dissipative forces. The motion of the system is limited by ideal nonlinear nonholonomic constraints. Five cases determined by the relationship between the degree of the first nontrivial polynomials in Maclaurin’s series for the potential energy and the functions that can be generated from the equations of nonlinear nonholonomic constraints are analyzed. In the three cases, the theorem on the instability of the position of equilibrium of nonholonomic systems with linear homogeneous constraints (V. V. Kozlov (1986)) is generalized to the case of nonlinear nonhomogeneous constraints. In the other two cases, new theorems are set extending the result from V. V. Kozlov (1994) to nonholonomic systems with nonlinear constraints.

Key words: Lyapunov first method, nonholonomic system, instability of equilibrium