Abstract

Choosing an undergraduate major is an important decision that impacts academic and career outcomes. We investigate using ChatGPT-4, a state-of-the-art large language model (LLM), to augment human advising for major selection. Through a 3-phase survey, we compare ChatGPT suggestions and responses for undeclared Freshmen and Sophomore students (n=18) to expert responses from university advisors (n=18). Undeclared students were first surveyed on their interests and career goals. These responses were then given to both campus advisors and to ChatGPT to produce a major recommendation for each student. In the case of ChatGPT, information about the majors offered on campus was added to the prompt. Advisors, overall, rated the recommendations of ChatGPT to be highly helpful and agreed with their recommendation 39% of the time. Additionally, we find that the AI major recommendations substantially influenced advisor recommendations, however, this result was just shy of statistical significance, likely owing to our relatively low amount of data collected thus far. The results provide a first signal as to the viability of LLMs for personalized major recommendation and shed light on the promise and limitations of AI for advising support.

Kasra Lekan

Master’s Student

My research interests include natural language processing, human-AI interaction, and modelling complex systems.