The mechanical response of the human arterial wall under the combined loading of inflation, axial extension, and torsion is examined within the framework of the large deformation hyper-elastic theory. The probability of the aneurysm formation is explained with the instability theory of structure, and the probability of its rupture is explained with the strength theory of material. Taking account of the residual stress and the smooth muscle activities, a two layer thick-walled circular cylindrical tube model with fiber-reinforced composite-based incompressible anisotropic hyper-elastic materials is employed to model the mechanical behavior of the arterial wall. The deformation curves and the stress distributions of the arterial wall are given under normal and abnormal conditions. The results of the deformation and the structure instability analysis show that the model can describe the uniform inflation deformation of the arterial wall under normal conditions, as well as formation and growth of an aneurysm under abnormal conditions such as the decreased stiffness of the elastic and collagen fibers. From the analysis of the stresses and the material strength, the rupture of an aneurysm may also be described by this model if the wall stress is larger than its strength.