A new X-ray undulator has been designed and constructed which produces linearly polarized X-rays in which the plane of polarization can be oriented to a user selectable angle, from horizontal to vertical. Based on the Apple-II elliptically polarizing undulator (EPU), the undulator rotates the angle of the linear polarization by a simple longitudinal motion of the undulator magnets. Combined with the circular and elliptical polarization capabilities of the EPU operating in the standard mode, this new undulator produces soft X-ray radiation with versatile polarization control. This paper describes the magnetic structure of the device and presents an analysis of the magnetic field with varying undulator parameters. The variable linear polarization capability is then exhibited by measuring the X-ray absorption spectrum of an oriented polytetrafluoroethylene thin film. This experiment, which measures the linear dichroism of the sample at two peaks near the C 1s absorption edge, demonstrates the continuous polarization rotation capabilities of the undulator.