Apolipoprotein E4 ( APOE4 ) is the leading genetic risk factor for Alzheimer's disease. While most studies examine the role of APOE4 in aging, imaging, and cognitive assessments reveal that APOE4 influences brain structure and function as early as infancy. Here, we examined human-relevant cellular phenotypes across neurodevelopment using induced pluripotent stem cell (iPSC) derived cortical and ganglionic eminence organoids (COs and GEOs). In COs, we showed that APOE4 decreased BRN2+ and SATB2+ cortical neurons, increased astrocytes and outer radial glia, and was associated with increased cell death and dysregulated GABA-related gene expression. In GEOs, APOE4 accelerated maturation of neural progenitors and neurons. Multi-electrode array recordings in assembloids revealed that APOE4 disrupted network formation and altered response to GABA, resulting in heightened excitability and synchronicity. Together, our data provides new insights into how APOE4 may influence cortical neurodevelopmental processes and network formation in the human brain.