We report the experimental observation and a theoretical explanation of collective suppression of linewidths for multiple superconducting qubits coupled to a good cavity. This demonstrates how strong qubit-cavity coupling can significantly modify the dephasing and dissipation processes that might be expected for individual qubits, and can potentially improve coherence times in many-body circuit QED.