Gate set tomography (GST) characterizes the process matrix of quantum logic gates, along with measurement and state preparation errors in quantum processors. GST typically requires extensive data collection and significant computational resources for model estimation. We propose a more efficient GST approach for qudits, utilizing the qudit Hadamard and virtual Z gates to construct fiducials while assuming virtual Z gates are error-free. Our method reduces the computational costs of estimating characterization results, making GST more practical at scale. We experimentally demonstrate the applicability of this approach on a superconducting transmon qutrit.