Expression noise results in cell-to-cell variability in expression levels, and feedback regulation may complicate the tracing of sources of this noise. Using a representative model of gene expression with feedbacks, we analytically show that the expression noise (or the total noise) is decomposed into three parts: feedback-dependent promoter noise determined by a continuous approximation, birth-death noise determined by a simple Poisson process, and correlation noise induced by feedbacks. We clarify confused relationships between feedback and noise in previous studies, by showing that feedback-regulated noisy sources have different contributions to the total noise in different cases of promoter switching (it is an essential reason resulting in confusions). More importantly, we find that there is a tradeoff between response time and expression noise. In addition, we show that in contrast to single feedbacks, coupled positive and negative feedbacks can perform better in tuning expression noise, controlling expression levels, and maintaining response time. The overall analysis implies that living organisms would utilize coupled positive and negative feedbacks for better survival in complex and fluctuating environments.