We implement Epstein–Nesbet perturbative corrections in the third-order for the initiator approximation of the configuration space quantum Monte Carlo. An improved sampling algorithm is proposed to address the stochastic noise of the corrections. The stochastic error for the perturbative corrections is considerably larger than that for the reference energy, and it fails to provide reasonable results unless a very long imaginary time integration is performed. The new sampling algorithm accumulates rejected walkers from multiple independent steps to cover a larger portion of the secondary space. The performance of the perturbative corrections is demonstrated for small molecules.