Human factors affecting the performance of the speaker dependent isolated Chinese speech recognition system (SRS) were studied. The results showed that the training mode and dialect factors had significant influence upon recognition accuracy; speaker's psychological factor, cooperativeness and educational background had certain influence, while factors such as sex, age, and profession of the speaker had less influence on recognition performance. With the results taken into consideration, ways for improving performance of the speech recognition systems were explored from the point of view of man-machine integration.