-
Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants
Authors:
Beatriz Borges,
Negar Foroutan,
Deniz Bayazit,
Anna Sotnikova,
Syrielle Montariol,
Tanya Nazaretzky,
Mohammadreza Banaei,
Alireza Sakhaeirad,
Philippe Servant,
Seyed Parsa Neshaei,
Jibril Frej,
Angelika Romanou,
Gail Weiss,
Sepideh Mamooler,
Zeming Chen,
Simin Fan,
Silin Gao,
Mete Ismayilzada,
Debjit Paul,
Alexandre Schöpfer,
Andrej Janchevski,
Anja Tiede,
Clarence Linden,
Emanuele Troiani,
Francesco Salvi
, et al. (65 additional authors not shown)
Abstract:
AI assistants are being increasingly used by students enrolled in higher education institutions. While these tools provide opportunities for improved teaching and education, they also pose significant challenges for assessment and learning outcomes. We conceptualize these challenges through the lens of vulnerability, the potential for university assessments and learning outcomes to be impacted by…
▽ More
AI assistants are being increasingly used by students enrolled in higher education institutions. While these tools provide opportunities for improved teaching and education, they also pose significant challenges for assessment and learning outcomes. We conceptualize these challenges through the lens of vulnerability, the potential for university assessments and learning outcomes to be impacted by student use of generative AI. We investigate the potential scale of this vulnerability by measuring the degree to which AI assistants can complete assessment questions in standard university-level STEM courses. Specifically, we compile a novel dataset of textual assessment questions from 50 courses at EPFL and evaluate whether two AI assistants, GPT-3.5 and GPT-4 can adequately answer these questions. We use eight prompting strategies to produce responses and find that GPT-4 answers an average of 65.8% of questions correctly, and can even produce the correct answer across at least one prompting strategy for 85.1% of questions. When grouping courses in our dataset by degree program, these systems already pass non-project assessments of large numbers of core courses in various degree programs, posing risks to higher education accreditation that will be amplified as these models improve. Our results call for revising program-level assessment design in higher education in light of advances in generative AI.
△ Less
Submitted 7 August, 2024;
originally announced August 2024.
-
Ab-initio variational wave functions for the time-dependent many-electron Schrödinger equation
Authors:
Jannes Nys,
Gabriel Pescia,
Alessandro Sinibaldi,
Giuseppe Carleo
Abstract:
Describing the real-time evolution of many-electron quantum systems is crucial for understanding the dynamical properties of condensed matter, molecular systems in quantum chemistry, and the behaviors of complex materials. However, the real-time evolution of non-equilibrium quantum electronic systems poses a significant challenge for theoretical and computational approaches. This work introduces a…
▽ More
Describing the real-time evolution of many-electron quantum systems is crucial for understanding the dynamical properties of condensed matter, molecular systems in quantum chemistry, and the behaviors of complex materials. However, the real-time evolution of non-equilibrium quantum electronic systems poses a significant challenge for theoretical and computational approaches. This work introduces a variational approach for fermionic time-dependent wave functions, surpassing mean-field approximations by capturing many-body correlations. Our methodology introduces a parameterization of the time-evolving quantum state, enabling an accurate approximation of its evolution. To account for electron correlations, we employ time-dependent Jastrow factors and backflow transformations, which can be further enhanced by incorporating neural networks for function parameterization. We utilize the time-dependent variational Monte Carlo technique to efficiently compute optimal time-dependent parameters. Additionally, we introduce a new time-evolution method based on Trotter-root expansions of the propagator, enhancing the accuracy and efficiency of our simulations. The approach is demonstrated in three distinct systems: the solvable harmonic interaction model, the dynamics of a diatomic molecule in intense laser fields, and a quenched quantum dot. In all cases, we show clear signatures of many-body correlations in the dynamics that are not captured by mean-field methods. The results showcase the ability of our variational approach to accurately capture the time evolution of quantum states, providing insight into the quantum dynamics of interacting electronic systems, beyond the capabilities of mean-field.
△ Less
Submitted 9 August, 2024; v1 submitted 12 March, 2024;
originally announced March 2024.
-
NetKet 3: Machine Learning Toolbox for Many-Body Quantum Systems
Authors:
Filippo Vicentini,
Damian Hofmann,
Attila Szabó,
Dian Wu,
Christopher Roth,
Clemens Giuliani,
Gabriel Pescia,
Jannes Nys,
Vladimir Vargas-Calderon,
Nikita Astrakhantsev,
Giuseppe Carleo
Abstract:
We introduce version 3 of NetKet, the machine learning toolbox for many-body quantum physics. NetKet is built around neural-network quantum states and provides efficient algorithms for their evaluation and optimization. This new version is built on top of JAX, a differentiable programming and accelerated linear algebra framework for the Python programming language. The most significant new feature…
▽ More
We introduce version 3 of NetKet, the machine learning toolbox for many-body quantum physics. NetKet is built around neural-network quantum states and provides efficient algorithms for their evaluation and optimization. This new version is built on top of JAX, a differentiable programming and accelerated linear algebra framework for the Python programming language. The most significant new feature is the possibility to define arbitrary neural network ansätze in pure Python code using the concise notation of machine-learning frameworks, which allows for just-in-time compilation as well as the implicit generation of gradients thanks to automatic differentiation. NetKet 3 also comes with support for GPU and TPU accelerators, advanced support for discrete symmetry groups, chunking to scale up to thousands of degrees of freedom, drivers for quantum dynamics applications, and improved modularity, allowing users to use only parts of the toolbox as a foundation for their own code.
△ Less
Submitted 18 August, 2022; v1 submitted 20 December, 2021;
originally announced December 2021.
