A field experiment was conducted under a wheat-maize rotation system from 1990 to 2006 in North China Plain (NCP) to determine the effects of N, P and K on yield and yield gap. There were five treatments: NPK, PK, NK, NP and a control. Average wheat and maize yields were the highest in the NPK treatment, followed by those in the NP plots among all treatments. For wheat and maize yield, a significant increasing trend over time was found in the NPK-treated plots and a decreasing trend in the NK-treated plots. In the absence of N or P, wheat and maize yields were significantly lower than those in the NPK treatment. For both crops, the increasing rate of the yield gap was the highest in the P omission plots, i.e., 189.1 kg ha(-1) yr(-1) for wheat and 560.6 kg ha(-1) yr(-1) for maize. The cumulative omission of P fertilizer induced a deficit in the soil available N and extractable P concentrations for maize. The P fertilizer was more pivotal in long-term wheat and maize growth and soil fertility conservation in NCP, although the N fertilizer input was important for both crops growth. The crop response to K fertilizers was much lower than that to N or P fertilizers, but for maize, the cumulative omission of K fertilizer decreased the yield by 26% and increased the yield gap at a rate of 322.7 kg ha(-1) yr(-1). The soil indigenous K supply was not sufficiently high to meet maize K requirement over a long period. The proper application of K fertilizers is necessary for maize production in the region. Thus, the appropriate application of N and P fertilizers for the growth of both crops, while regularly combining K fertilizers for maize growth, is absolutely necessary for sustainable crop production in the NCP.