The effect of 5-hydroxytryptamine on the induction of long-term potentiation by a train of high frequency pulses (100 Hz; 1 s) or by a stimulation consisting of one burst of five pulses at 100 Hz delivered 170 ms after a single pulse (primed burst) was investigated in the CA1 region of the rat hippocampal slice in vitro with extracellular recordings. Superfusion with 5-hydroxytryptamine (3-30 microM) produced a concentration-dependent decrease in amplitude of the population spikes evoked by test stimuli. The presence of 5-hydroxytryptamine (30 microM) did not affect the magnitude of long-term potentiation produced by the high-frequency stimulation but it prevented the long-term potentiation induced by a primed burst. The action of 5-hydroxytryptamine was mimicked by the 5-hydroxytryptamine1A agonist 5-carboxamidotryptamine (0.3 microM) and blocked by the 5-hydroxytryptamine2/5-hydroxytryptamine1A antagonist spiperone (3 microM) or by the 5-hydroxytryptamine1/5-hydroxytryptamine2 antagonist methiothepin (1-10 microM). The selective 5-hydroxytryptamine2 antagonist ritanserin (1 microM) did not antagonize the block of long-term potentiation produced by 5-hydroxytryptamine. The selective 5-hydroxytryptamine3 antagonists (3-tropanyl)-1H-indole-3-carboxylic acid ester (ICS 205-930; 1 nM) and ondansetron (GR-38032; 30 nM) did not affect the reduction in the population spike produced by application of 5-hydroxytryptamine. In contrast, a primed burst delivered at the fifth minute of 5-hydroxytryptamine application in the presence of a 5-hydroxytryptamine3 antagonist induced a long-term potentiation.(ABSTRACT TRUNCATED AT 250 WORDS)