It is known that as the FeAs4 tetrahedron in the Fe-based superconductor is close to the regular tetrahedron, critical temperature (Tc) can be greatly increased. Recently, a Co-based superconductor of LaCoSi (4 K) with "111" structure was found. In this work, we improve the Tc of LaCoSi through structural regulation. Tc can be increased by the chemical substitution of Co by Fe, while the superconductivity is suppressed by the Ni substitution. The combined analysis of neutron and synchrotron X-ray powder diffractions reveals that the change of the Si-Co-Si bond angles of the CoSi4 tetrahedron is possibly responsible for the determination of superconducting properties. The Fe chemical substitution is favorable for the formation of the regular tetrahedron of CoSi4. The present new Co-based superconductor of LaCoSi provides a possible method to enhance the superconductivity performance of the Co-based superconductors via controlling Co-based tetrahedra similar to those well established in the Fe-based superconductors.