The goal of this study is to develop an advanced point-of-care diagnostic instrument for use in a primary care office using handheld optical coherence tomography (OCT). This system has the potential to enable earlier detection of diseases and accurate image-based diagnostics. Our system was designed to be compact, portable, user-friendly, and fast, making it well suited for the primary care office setting. The unique feature of our system is a versatile handheld OCT imaging scanner which consists of a pair of computer-controlled galvanometer-mounted mirrors, interchangeable lens mounts, and miniaturized video camera. This handheld scanner has the capability to guide the physician in real time for finding suspicious regions to be imaged by OCT. In order to evaluate the performance and use of the handheld OCT scanner, the anterior chamber of a rat eye and in vivo human retina, cornea, skin, and tympanic membrane were imaged. Based on this feasibility study, we believe that this new type of handheld OCT device and system has the potential to be an efficient point-of-care imaging tool in primary care medicine.