A new approach for determining the orbital contribution to analytical derivatives of perturbation theory energies is presented. The usefulness of the new approach is demonstrated by application to closed-shell second-order Møller-Plesset perturbation theory (MP2). This new method is related to that used previously (Lee, T. J., and Rendell, A. P., 1991, J. Chem. Phys., 94, 6229) for analytical energy gradients of the closed-shell singles and doubles coupled-cluster method augmented with a perturbational estimate of the effects of connected triple excitations, CCSD(T), but it is more appealing in that potential singularities are rigorously eliminated. The computational savings for MP2 are modest (two n5 steps, where n is the number of molecular orbitals), but the savings for fourth-order perturbation theory (MP4) or CCSD(T) energy gradients (two n7 steps) are significant. Hence the new approach is less expensive than previous applications of analytical gradient theory to perturbation theory electron correlation energies, and it is numerically stable.