More than 1.3 million people lose their lives every year in traffic accidents. Improving road safety requires designing better vehicles and investigating drivers' abilities more closely. Driving simulators are constantly being used for this purpose, but the question which often arises as to their validity tends to be a barrier to developments in this field. Here we studied the validity of a simulator, defined as how closely users' behavior under simulated conditions resembles their behavior on the road, based on the concept of drivers' feeling of presence. For this purpose, the driving behavior, physiological state and declarative data of 41 drivers were tested in the Sherpa2 simulator and in a real vehicle on a track while driving at a constant speed. During each trial, drivers had to cope with an unexpected hazardous event (a one-meter diameter gym ball crossing the road right in front of the vehicle), which occurred twice. During the speed-maintenance task, the simulator showed absolute validity, in terms of the driving and physiological parameters recorded. During the first hazardous event, the physiological parameters showed that the level of arousal (Low Heart Rate/High Heart Rate ratio x10) increased up to the end of the drive. On the other hand, the drivers' behavioral (braking) responses were 20% more frequent in the simulator than in the real vehicle, and the physiological state parameters showed that stress reactions occurred only in the real vehicle (+5 beats per minute, +2 breaths per minute and the phasic skin conductance increased by 2). In the subjects' declarative data, several feeling of presence sub-scales were lower under simulated conditions. These results suggest that the validity of motion based simulators for testing drivers coping with hazards needs to be questioned.