Laparoscopic techniques have gained wide acceptance because they offer a safe and less invasive alternative to open surgery. To further reduce the invasiveness of peritoneal access, the next logical step is to eliminate the incision through the abdominal wall using natural orifices as entry points. This Natural Orifice Transluminal Endoscopic Surgery (NOTES) approach has the potential to replace or augment current techniques. Several research groups have cut through the stomach or colon wall (per-oral transgastric or per-anal transcolonic) to perform organ resections in animal models, and some procedures in humans have been reported anecdotally. Widespread use of these techniques will depend on providing the physician with adequate visual feedback, clear indicators of instrument location and orientation, and support in the recognition of anatomic structures. Compared with laparoscopy, successful endoscopy must accommodate several additional complexities: (1) The flexibility of the endoscope tip complicates the understanding of its distal orientation. Successful navigation inside the stomach and in the abdominal cavity generally requires two years of sub-specialty training. (2) Several surgical targets lie in a retrograde position with respect to an incision in the stomach wall. Efficient and safe access to the pancreas, gall bladder, or the kidneys requires detailed knowledge of the tip placement relative to adjacent anatomic structures. (3) Since there is limited direct access to the abdomen, iatrogenic injuries, such as the accidental cutting of an artery, will be more dangerous and difficult to manage. We present here approaches to resolving these limitations though augmented reality techniques using pre-procedure CT or MRI imaging, real time tracking and reference image registration, and display to the operating physician. As an example, the utility of image registration techniques for orientation for the gastric access puncture is discussed in detail. It is anticipated that such augmentation will make intra-cavitary interventional techniques easier to master and use in practice, and thus more likely to be widely adopted.