Paper by Wesley H. Holliday ([email protected]), Matthew Mandelkern ([email protected]), and Cedegao E. Zhang ([email protected]) available at https://arxiv.org/abs/2401.17169.

• llm-logic.ipynb: The Jupyter notebook containing all the code used to run the experiments and generate the graphs presented in the paper.

• prompts/: Includes all of the inference pattern questions used in the experiments.

• data/: Stores the LLM responses to the inference pattern questions.

• external_data/: Stores the LMSYS Elo ratings for the tested models

• graphs/ and results/: These directories are generated upon running the "llm-logic" notebook and contain the graphs and results of experiments, respectively.

To replicate the experiments or explore the results, open and run the "llm-logic.ipynb" Jupyter notebook.

For any queries related to the paper or the supplementary material, please write to [email protected].