We investigated the validity and inter-criteria reproducibility between RECIST (Response Evaluation Criteria in Solid Tumors) guidelines and WHO (World Health Organization) criteria, considering the decrease in patient numbers resulting from inclusion of the minimum lesion size criterion introduced in RECIST guidelines. RECIST guidelines are based on unidimensional measurement and exclusion of small lesions from measurement. The aims of the study were to examine: (1) the effect of the minimum lesion size criterion, (2) the validity of unidimensional and bidimensional measurements, i.e., their relationship with tumor volume, (3) the inter-criteria reproducibility between current RECIST guidelines and previous WHO criteria. One hundred and twenty patients with non-small cell lung cancer (NSCLC) in clinical trials were evaluated. By applying the minimum lesion size criterion, six cases became ineligible without any influence on precision of tumor volume measurement. In the validity study, actual tumor volume was regarded as the gold standard. Although the unidimensional measurement had a lower correlation with tumor volume value than the bidimensional measurement, both the unidimensional measurement and bidimensional measurement correlated sufficiently well with tumor volume changes and the assessed tumor volume response. In the inter-criteria reproducibility study between RECIST guidelines and WHO criteria, the response rate assessed by RECIST guidelines (19.3%) was almost the same as that assessed by WHO criteria (20.0%). In conclusion, RECIST guidelines are adequate for evaluating tumor response to chemotherapy in terms of both validity in relation to tumor volume and inter-criteria reproducibility with the WHO criteria.