The brain activity induced by heroin-related cues may play a role in the maintenance of heroin dependence. Whether the reinforcement or processing biases construct an everlasting feature of heroin addiction remains to be resolved. We used an event-related fMRI paradigm to measure brain activation in response to heroin cue-related pictures versus neutral pictures as the control condition in heroin-dependent patients undergoing short-term and long-term abstinence. The self-reported craving scores were significantly increased after cue exposure in the short-term abstinent patients (t = 3.000, P = 0.008), but no increase was found in the long-term abstinent patients (t = 1.510, P = 0.149). However, no significant differences in cue-induced craving changes were found between the two groups (t = 1.193, P = 0.850). Comparing between the long-term abstinence and short-term abstinence groups, significant decreases in brain activation were detected in the bilateral anterior cingulated cortex, left medial prefrontal cortex, caudate, middle occipital gyrus, inferior parietal lobule and right precuneus. Among all of the heroin dependent patients, the abstinence duration was negatively correlated with brain activation in the left medial prefrontal cortex and left inferior parietal lobule. These findings suggest that long-term abstinence may be useful for heroin-dependent patients to diminish their saliency value of heroin-related cues and possibly lower the relapse vulnerability to some extent.