We present the first Green's function Monte Carlo calculations of light nuclei with nuclear interactions derived from chiral effective field theory up to next-to-next-to-leading order. Up to this order, the interactions can be constructed in a local form and are therefore amenable to quantum Monte Carlo calculations. We demonstrate a systematic improvement with each order for the binding energies of A=3 and A=4 systems. We also carry out the first few-body tests to study perturbative expansions of chiral potentials at different orders, finding that higher-order corrections are more perturbative for softer interactions. Our results confirm the necessity of a three-body force for correct reproduction of experimental binding energies and radii, and pave the way for studying few- and many-nucleon systems using quantum Monte Carlo methods with chiral interactions.