Enhancing the ubiquitous sensors and connected devices with computational abilities to realize visions of the Internet of Things (IoT) requires the development of robust, compact, and low-power deep neural network accelerators. Analog in-memory matrix-matrix multiplications enabled by emerging memories can significantly reduce the accelerator energy budget while resulting in compact accelerators. In this article, we design a hardware-aware deep neural network (DNN) accelerator that combines a planar-staircase resistive random access memory (RRAM) array with a variation-tolerant in-memory compute methodology to enhance the peak power efficiency by 5.64× and area efficiency by 4.7× over state-of-the-art DNN accelerators. Pulse application at the bottom electrodes of the staircase array generates a concurrent input shift, which eliminates the input unfolding, and regeneration required for convolution execution within typical crossbar arrays. Our in-memory compute method operates in charge domain and facilitates high-accuracy floating-point computations with low RRAM states, device requirement. This work provides a path toward fast hardware accelerators that use low power and low area.