We present results of extensive theoretical studies concerning stability, morphology, and band structure of single wall carbon nanotubes (CNTs) covalently functionalized by -CH(n) (for n = 2,3,4), -NH(n) (for n = 1,2,3,4), -COOH, and -OH groups. These studies are based on ab initio calculations in the framework of the density functional theory. For functionalized systems, we determine the dependence of the binding energies on the concentration of the adsorbed molecules, critical densities of adsorbed molecules, global and local changes in the morphology, and electronic structure paying particular attention to the functionalization induced changes of the band gaps. These studies reveal physical mechanisms that determine stability and electronic structure of functionalized systems and also provide valuable theoretical predictions relevant for application. In particular, we observe that functionalization of CNTs causes generally their elongation and locally sp(2) to sp(3) rehybridization in the neighborhood of chemisorbed molecules. For adsorbants making particularly strong covalent bonds with the CNTs, such as the -CH2 fragments, we observe formation of the characteristic pentagon/heptagon (5/7) defects. In systems functionalized with the -CH2, -NH4, and -OH groups, we determine critical density of molecules that could be covalently bound to the lateral surface of CNTs. Our studies show that functionalization of CNTs can be utilized for band gap engineering. Functionalization of CNTs can also lead to changes in their metallic/semiconductor character. In semiconducting CNTs, functionalizing molecules such as -CH3, -NH2, -OH, -COOH, and both -OH and -COOH, introduce "impurity" bands in the band gap of pristine CNTs. In the case of -CH3, -NH2 molecules, the induced band gaps are typically smaller than in the pure CNT and depend strongly on the concentration of adsorbants. However, functionalization of semiconducting CNTs with hydroxyl groups leads to the metallization of CNTs. On the other hand, the functionalization of semi-metallic (9,0) CNT with -CH2 molecules causes the increase of the band gap and induces semi-metall to semiconductor transition.