In an optomechanical system, we experimentally engineer the optical density of state to reduce or broaden the effective linewidth of the optical mode by introducing an ancillary mechanical mode, which has a large decay rate, i.e., stimulated backward Brillouin scattering. Based on this dissipation engineering, we could engineer the optical mode linewidth by one order of magnitude. In addition, we can either enhance or suppress the optomechanical cooling and amplification of the target mechanical oscillations. Our scheme demonstrates the cascaded photon-phonon coupling to control the mechanical interactions, and also presents a novel approach for engineering coherent light-matter interaction in hybrid systems, which consist of different types of nonlinear interactions and multiple modes, and promote the performance of quantum devices.