Pilot Adaptation and Preliminary Validation of the Generative AI Literacy for Learning Scale (GenAI-LLs) in Engineering Education

Abstract

The widespread adoption of Generative Artificial Intelligence (GenAI) in higher education has increased the need for valid instruments capable of assessing the competencies required for effective and responsible AI-supported learning. This pilot study aims to linguistically and contextually adapt and preliminarily validate the Generative AI Literacy for Learning Scale (GenAI-LLs) developed by Gümüş and Kara (2025) for engineering Project-Based Learning (PBL). The sample comprised 138 engineering students from the Technical University of Cluj-Napoca, Romania. Data were analysed using Exploratory Factor Analysis (EFA), descriptive statistics, Cronbach's alpha, Pearson correlation analysis, independent-samples t-tests, and one-way ANOVA. The EFA retained a four-factor solution explaining 61.30% of the total variance, characterised by the integration of Needs analysis and Prompting and language skills into a single factor and the differentiation of Autonomous learning into collaborative and autonomous components. The empirically derived GenAI literacy factors demonstrated good to excellent internal consistency. Correlation analyses indicated predominantly positive relationships among the four factors. No significant gender differences were identified, whereas students with more positive attitudes towards GenAI reported significantly higher levels of several GenAI literacy dimensions. The findings provide preliminary evidence supporting the psychometric adequacy of the Romanian adaptation of the GenAI-LLs and establish a foundation for future confirmatory validation studies in engineering higher education.