Experiments have suggested that the high- T(c) cuprate YBCO shows marked anisotropy in penetration depth and coherence length measurements. In order to take into account the presence of this anisotropy in the system and its corresponding effect on the various properties of the high- T(c) superconducting materials, we have developed an anisotropic two-order parameter Ginzburg-Landau (GL) theory involving a mixed symmetry state of the order parameter components. For this we have generalized a two-order parameter GL theory, recently developed for the isotropic high- T(c) superconductors involving a mixed symmetry state of the order parameter components (Karmakar and Dey 2008 J. Phys. Condens. Matter 20 255218), in which the effect of the presence of in-plane anisotropy has been taken into account by an effective mass approximation, with the anisotropy being characterized by the parameter γ = m(x)/m(y). The work goes beyond the limitations of the earlier studies in this field as it enables us to carry out a detailed study of the various properties of the system over the entire range of applied magnetic field and wide range of temperature for arbitrary values of the GL parameter κ(y) and vortex lattice symmetry. The model successfully explains not only the observed oblique vortex lattice structure in the presence of in-plane anisotropy but also the experimentally observed angle between the primitive axes of the vortex lattice. The generation of two-fold symmetry of the vortices in the presence of in-plane anisotropy, for a very low applied magnetic field can also be analyzed by our model. We have also compared our theoretical results with various other experiments on high- T(c) cuprate YBCO.