Collective strong light-matter coupling provides a versatile means to manipulate physicochemical properties of molecules and materials. Understanding collective polaritonic dynamics is hindered by the macroscopic number of molecules interacting collectively with photonic modes. We develop a many-body theory to investigate the spectroscopy and dynamics of a molecular ensemble embedded in an optical cavity in the collective strong coupling regime. This theory is constructed by a pseudoparticle representation of the molecular Hamiltonian, which maps the polaritonic Hamiltonian into a coupled fermion-boson model under particle number constraints. The mapped model is then analyzed using the nonequilibrium Green's function theory with the self-energy diagrams identified through a large N expansion. We demonstrate that in the thermodynamic limit, the necessary condition to have any collective effects is to have a macroscopic cavity field. Numerical illustrations are shown for the driven Tavis-Cummings model, which shows an excellent agreement with exact results.