This pilot study successfully demonstrated that a retrospective, population-based, comparative design is a feasible method for evaluating the effectiveness of a community health worker program for women at risk for poor pregnancy and birth outcomes.