Inspired by the demand of improving the riding comfort and meeting the lightweight design of the vehicle, an inerter-based X-structure nonlinear energy sink (IX-NES) is proposed and applied in the half-vehicle system to enhance the dynamic performance. The X-structure is used as a mechanism to realize the nonlinear stiffness characteristic of the NES, which can realize the flexibility, adjustability, high efficiency, and easy operation of nonlinear stiffness, and is convenient to apply in the vehicle suspension, and the inerter is applied to replacing the mass of the NES based on the mass amplification characteristic. The dynamic model of the half-vehicle system coupled with the IX-NES is established with the Lagrange theory, and the harmonic balance method (HBM) and the pseudo-arc-length method (PALM) are used to obtain the dynamic response under road harmonic excitation. The corresponding dynamic performance under road harmonic and random excitation is evaluated by six performance indices, and compared with that of the original half-vehicle system to show the benefits of the IX-NES. Furthermore, the structural parameters of the IX-NES are optimized with the genetic algorithm. The results show that for road harmonic and random excitation, using the IX-NES can greatly reduce the resonance peaks and root mean square (RMS) values of the front and rear suspension deflections and the front and rear dynamic tire loads, while the resonance peaks and RMS values of the vehicle body vertical and pitching accelerations are slightly larger. When the structural parameters of the IX-NES are optimized, the vehicle body vertical and pitching accelerations of the half-vehicle system could reduce by 2.41% and 1.16%, respectively, and the other dynamic performance indices are within the reasonable ranges. Thus, the IX-NES combines the advantages of the inerter, X-structure, and NES, which improves the dynamic performance of the half-vehicle system and provides an effective option for vibration attenuation in the vehicle engineering.