Two main candidates have been proposed for the role of relaxant neurotransmitter in the intestine: (a) the purine nucleotide, 5'-adenosine triphosphate (ATP) and (b) the neuropeptide, vasoactive intestinal peptide (VIP). The candidacy of VIP is favored by its precise location in nerve fibers that innervate circular smooth muscle and tenia coli. We have used a photoaffinity analog of ATP, 3'-O-(4-Benzoyl)benzoyl ATP, that binds irreversibly to ATP receptors and inactivates them in the presence of light, and a specific VIP antiserum to examine the claims of VIP and ATP as relaxant neurotransmitters in tenia coli of the guinea pig. Both VIP and ATP caused dose-dependent, tetrodotoxin-insensitive relaxation of tenia coli. The effect of ATP was equipotent to that of its stable isostere alpha, beta-methylene ATP and resistant to degradation by adenosine deaminase, indicating interaction of ATP with purinergic-P2 receptors. Photoactivated 3'-O-(4-Benzoyl) benzoyl adenosine triphosphate selectively inhibited relaxation induced by ATP but had no effect on relaxation induced by VIP or by field (i.e., neural) stimulation. Vasoactive intestinal peptide antiserum (final dilution 1:60), on the other hand, inhibited relaxation caused by VIP and by field stimulation but had no effect on relaxation caused by ATP. Neither normal rabbit serum nor preneutralized VIP antiserum had any effect on relaxation induced by ATP, VIP, or field stimulation. Inhibition of neurally induced relaxation by VIP antiserum ranged from 52% +/- 7% (p less than 0.01) at the lowest frequency of stimulation to 15% +/- 4% (p less than 0.01) at the highest frequency, consistent with competitive interaction between antiserum and neurally released VIP. Near-maximal field stimulation at 1 Hz caused an eightfold (800% +/- 49%, p less than 0.01) increase in VIP release into the bathing medium. The results favor VIP (and probably peptide histidine isoleucine, a relaxant homologue known to be cosynthesized with VIP) as the main neural mediator of relaxation in tenia coli.