Digital Skills in Distance University Students: A Psychometric Analysis Competencias Digitales en Estudiantes Universitarios a Distancia: Un Análisis Psicométrico
-
Esther Fernández-MárquezPablo de Olavide University (España)
Cómo citar: Aires, L., Amante, L. ., Fernández-Márquez, E. ., & Nunes, C. . Digital Skills in Distance University Students: A Psychometric Analysis. Revista Fuentes, 27(2), 149-163. https://doi.org/10.12795/revistafuentes.2025.26469
Abstract
In recent years, research on the digital skills of higher education students has increased significantly. However, many of the existing studies have focused primarily on identifying these skills rather than exploring the factors that influence their development. This study investigates the digital skills of first-year undergraduate students enrolled at an online distance education university, as well as their key determining factors. Employing a quantitative methodology, the students completed a Likert-scale questionnaire as part of an online familiarization module. Based on the findings, the students’ digital skills were grouped into the following categories: collaboration and creativity, networking, content sharing, information management, information evaluation, and critical thinking. Additionally, the key influencing factors were identified, including the perceived ease of using ICT for problem-solving, orientation towards learning goals, personal initiative, and self-regulation in the use of ICT and the Internet. The results suggest that both individual and social strategies related to Internet use play a crucial role on the acquisition of digital skills. Future research should explore theoretical frameworks and, especially, refine assessment tools, shifting the focus from simply describing digital competencies to examining the factors that influence and predict their development. Such theoretical and methodological adjustments are expected to significantly reshape training strategies in this field.
Resumen
La investigación sobre las competencias digitales se ha incrementado los últimos años. Sin embargo, estas investigaciones se han centrado más en identificar estas competencias que en considerar los factores que influyen en su desarrollo. En este estudio se analizan las competencias digitales de un grupo de estudiantes de una universidad de educación a distancia. Optándose por una metodología cuantitativa, se aplicó un cuestionario tipo Likert online a un grupo de estudiantes de nuevo ingreso. Los datos obtenidos han sido analizados con técnicas estadísticas descriptivas e inferenciales. Los resultados demostraron que las competencias digitales mejor valoradas por los estudiantes son: colaboración, creatividad, trabajo en red, intercambio y gestión de información, evaluación de información y pensamiento crítico. Además, se han identificado los siguientes factores clave que determinan el desarrollo de estas competencias: facilidad de uso de las TIC en la resolución de problemas; orientación hacia los objetivos de aprendizaje; iniciativa personal, autorregulación en el uso de TIC/Internet. Se concluye que estrategias individuales, sociales y actitudinales especificas hacia Internet tienen un impacto importante en la adquisición efectiva de estas competencias. En investigaciones futuras, se recomienda profundizar en los marcos teóricos y, especialmente, en los instrumentos de evaluación de estas competencias, ampliando el foco de la descripción de las competencias digitales hacia los factores que condicionan y predicen su desarrollo. Se estima que cambios teóricos y metodológicos de este tipo impulsarán transformaciones significativas en las estrategias formativas en este ámbito.
Palabras clave / Keywords
Competencia Digital, Internet, Educación a Distancia, Estudiantes, Educación Superior, Instrumento de Evaluación.
Digital Skills, Internet, Distance Education, Students, Higher Education, Assessment tool, Learning.
1. Introduction
In recent years, there has been a growing body of research on the digital skills of higher education students. However, many of the existing studies have focused primarily on describing the students’ existing skills, rather than exploring the underlying factors that influence their development. The literature review highlights a prevailing tendency to address digital competencies as isolated entities, disconnected from the broader contexts and factors that determine them (Hargittai, 2021; Wu et al., 2021; Раkhomova et al., 2023).
Apart from theoretical models, effective assessment tools are essential for measuring digital competence. However, despite the existence of various frameworks designed to evaluate digital skills (Schwarz et al., 2024), such as DigComp 2.0 or DigComp 21 (Vuorikari et al, 2016; Carretero et al., 2017), current assessment systems struggle to establish systematic and efficient evaluation methods. This highlights the need to improve measurement approaches (Sillat et al., 2021) and the identification of the key predictor variables of digital proficiency (Cabezas- González et al, 2022).
In light of ongoing digital transformations and the challenges educational communities faced during the Covid-19 pandemic, digital competencies (or lack thereof) and the methods to evaluate them have become a central topic of discussion across all educational levels, including universities (Zhao et al., 2021; Saienko et al., 2022; Monferrato, 2024).
This study examines digital skills within the broader framework of 21st-century competencies, focusing not only on students’ digital skills, but also on the factors that influence their unequal development. The literature review conducted emphasized the relevance of the studies by Van Laar et al. (2017, 2020) and Helsper (2019), who offer valuable insights into the key determinants of digital skills and their development. Their findings are particularly relevant for online distance learning universities, which have long recognized the crucial role of digital competencies in promoting student inclusion within virtual campuses, enhancing academic success, and improving overall educational outcomes (Mohammadyari & Singh, 2015; Pham Tra & Dau Thi Kim, 2024). As Fernández-Gómez et al. (2021) pointed out, distance education models favor the adoption of more collaborative methodologies, which foster digital literacy. Therefore, as online distance universities prioritize the inclusion of digital skills in their pedagogical models (Pereira et al., 2007; Gros Salvat et al., 2009), they actively contribute to advancing research in this field (Quintas-Mendes et al., 2019; Sezgin & Fırat, 2024).
This study is part of a research project that aims to analyze the digital skills of newly enrolled university students. The first phase focuses on evaluating the digital skills of first-year students and the factors that determine their development. A questionnaire was administered to the students during a mandatory online familiarization module (OFM) completed prior to the start of their degree programs. The results highlight the need for changes both in the methodological approaches and assessment tools currently employed to evaluate digital skills.
1.1. Digital Skills in the 21st Century
Many studies associate the digital divide and digital skills to political, social, and educational agendas (EU, 2021), yet their role in key educational priorities in the 21st century is not evident (Voogt & Roblin, 2012; Soule & Warrick, 2015; van Deursen et al., 2017; Sezgin & Fırat, 2024). This gap emphasizes the need to define or clarify the concepts of digital skills and 21st-century skills. There are numerous definitions of digital skills depending on the framework used to categorize them. In general, digital skills refer to the ability to use digital technologies for work, leisure, and communication (From, 2017). However, a more detailed and operational definition, in line with Ferrari (2012), describes digital skills as the set of abilities required to perform tasks, solve problems, and create and share knowledge in a critical, ethical, and autonomous manner, across the domains of work, social participation, leisure, and personal development (Koch & Fehlmann, 2025). Current investigations on the topic has found that digital inclusion and digital skills also foster multiple experiences of internet use (Hargittai et al., 2018; Helsper, 2019; van Laar et al., 2020).
Twenty-first-century skills encompass foundational personal, social, and professional abilities that empower individuals to engage in various dimensions of the knowledge society. These competencies are transversal and multidimensional, combining knowledge, attitudes, and values, as well as fostering higher-order thinking to solve complex problems, and deal with uncertain situations (OECD, 2018). Given that these constructs are interconnected, van Deursen et al. (2017) and van Laar et al. (2020) have proposed merging 21st-century skills and digital skills into a single framework, termed 21st-century digital skills (see Table 1). This proposal aims to enhance the role of digital competencies in education, work, and social participation (Helsper et al., 2016), placing individuals, their contexts, communities, and broader relationships at the core of digital education.
Table 1
Twenty-first century digital skills
| 21st Century Digital Skills |
| Information Management |
| Communication |
| Collaboration |
| Creativity |
| Critical Thinking |
| Problem Solving |
(Source. van Laar et al., 2018, 2019b, 2020).
Theoretical perspectives on digital literacy and digital skills have developed significantly and been consolidated over the past five years, but much still needs to be done regarding the development of assessment tools. There remains a critical need for tools that go beyond merely descriptive evaluations, capable of offering predictive and comprehensive assessments that account for contextual and individual differences. They should encompass a broader range of competencies, including critical thinking and other higher-order cognitive abilities, while also actively engaging students, teachers, and community members. Reliable assessment of digital skills and soft skills are essential, rooted in robust reliability methodologies that do not compromise students’ readiness to face the diverse demands of daily life (Peláez-Sánchez et al., 2024).
1.2 Determinants of 21st century Digital Skills
Research on digital skills has often relied on unidimensional approaches that fail to consider factors that influence digital skill acquisition. Yet, it is critically important not only to understand how individuals acquire these digital skills but also to identify the factors that shape their development (van Deursen et al., 2020; Casillas-Martín et al., 2020; Cabezas-González et al., 2022). However, recent studies (can Laar et al., 2019a, 2020; Cabezas-González et al., 2022) have introduced a multidimensional approach to 21st-century Internet skills, highlighting the factors that influence their development. This proposal departs from the instrumental perspective that has traditionally dominated training in this domain, positioning the development of digital competences as a central priority within the context of lifelong learning. Our study is in line with this perspective by focusing on the social, individual, and motivational determinants that are considered relevant to examine individual differences. Factors such as ease of use, perceptions of ICT, self-directed learning, performance, and socio-contextual variables, including support for internet use, are propose as potential predictors that can account for variations in digital skills acquisition (Table 2).
Table 2
Determinants of 21st century Digital Skills
| Determinants |
| ICT Attitude |
| Perceived ease of use |
| ICT self-regulation |
| Self-directed learning |
| Performance goal orientation |
| Avoidance learning goal orientation |
| Personal Initiative |
(Source: van Laar et al. 2019b; 2020)
1.3 Higher Education Students and Digital Skills
When students enter Higher Education and begin their undergraduate studies, they are required to demonstrate a broad range of skills to ensure their participation in new learning environments. Among these, digital skills are critical for students to effectively engage in diverse campus contexts and are especially crucial in online campuses (Guttierrez et al., 2009, Almerich et al., 2019; Perera & Gardner, 2018; EU, 2017; 2021; Sezgin & Fırat, 2024; Smith et al., 2024). In online settings, students will encounter learning experiences that require more than basic technological usage. The online learning skills they must develop encompass pedagogical, individual and social competencies tailored to learning contexts. Key skills for successful online learning include independent learning, time management, online communication, interpersonal interactions, and wellbeing (Pereira et al., 2003; Koch & Fehlmann, 2025). Therefore, universities must prioritize the development of these skills in their educational goals and strategies (Palmeiro, Pereda & Aires, 2019; De Pablos, 2018; Saienko, Kurysh, & Siliutina, 2022).
To better understand the students’ needs in the domain of digital competencies, this study pursued the following goals:
G1. To analyze the digital skills of first-year students enrolled at an online distance education university.
G2. To identify the key factors influencing the development of students’ digital skills.
G3. To optimize an assessment tool to evaluate digital skills.
2. Methods
This quantitative study analyzed the perceptions of incoming undergraduate students at a distance education university regarding their digital skills, aiming to answer the following questions:
Q1: What are the students’ perspectives on their digital skills as they begin their undergraduate studies at an online distance university?
Q2: What social and individual determinants influence the students’ digital skills?
2.1 Research Context
The students who participated in this study were enrolled at an online distance education university, and were required to attend a mandatory online familiarization module (OFM). Specifically designed for newly admitted students, the OFM is in accordance with the university’s pedagogical model (Pereira et al., 2007; Mendes et al., 2019; Aretio, 2014, 2021; Willems et al., 2019). The OFM serves to familiarize new students with the university’s digital learning environment and teach them «how to be an online student». It offers each student the opportunity to evaluate their readiness to adapt to this learning system.
2.2 Sample
The study sample consisted of 327 students about to start their degrees in September 2020. Newly admitted undergraduate students were selected because it was crucial to understand their baseline digital skills, particularly their Internet skills, upon entering university. The insights obtained help to identify the students’ potential and their needs in the area of digital literacy. Prior informed consent was obtained from all students involved in the study.
Data Collection Instrument
Data were collected using a Likert-scale questionnaire adapted from previous studies (van Deursen et al., 2017; van Laar et al., 2018; van Laar et al., 2019a). The questionnaire was designed to assess a broad range of digital skills, such as information management, communication, collaboration, critical thinking, creativity, and problem-solving. Additionally, it incorporated constructs related to the students’ personal, motivational, and social perceptions, including perceived ease of use, self-regulation of ICT use, autonomous learning, orientation towards learning goals, avoidance of learning-goal orientation, and personal initiative (van Laar et al., 2019b) (Table 4).
This questionnaire included a total of 91 items: five items on descriptive data (gender, age, education level, academic status, and professional activity), and 86 items related to indicators that cover 21st-century digital skills and their determining factors, using a 5-point Likert scale (“Never,” “Rarely,” “Sometimes,” “Often,” and “Almost always”) (van Laar et al., 2019a, 2019b). For the purposes of this study, we focused on the following three sections of the questionnaire: (i) sociodemographic data, (ii) Internet use within the OFM, and (iii) strategies for online discussion and critical thinking.
This research project complied with the ethical standards for educational research. All research procedures and questionnaires were reviewed and approved by the Ethics Committee of LE@D (Laboratório de Educação e Ensino a Distância, Universidade Aberta) in September 2019.
2.4 Data Analysis Procedures
All the data analyses were conducted using IBM SPSS (version 25) and R software. Statistical significance was set at P < 0.05.
The data analysis was performed in two phases. In the first phase, a descriptive analysis was performed on the results of the questionnaire’s two main sections: (i) Uses of the Internet within the OFM; and (ii) Online Discussion Strategies. The first section comprised 74 items aimed assessing the students’ self-perceived usage of the Internet within the scope of the course. In this section, students self-assessed their digital competencies in information management, communication, collaboration, creativity, and problem solving, as well as the determinants of digital competencies, such as perceived ease of use, self-regulation of ICT use, autonomous learning, learning goal orientation, performance goal orientation, avoidance of learning goal orientation, and personal initiative (van Laar et al., 2019a, 2019b).
The second section consisted of 12 items focusing on online discussions in the OFM’s forums. In this section, students self-assessed the resources they mobilized in online discussions, such as critical thinking (Appendices 1 and 2).
A descriptive data analysis was conducted for each section. To verify internal consistency, Cronbach’s alpha was calculated (Cronbach, 1984). Normality was assessed using skewness and kurtosis. The suitability of applying factor analysis was tested using the Kaiser-Meyer-Olkin (KMO) coefficient and Bartlett’s test of sphericity (Hair et al., 2014).
The second phase of the data analysis consisted of an Exploratory Factor Analysis using Principal Components and the Varimax rotation with Kaiser normalization. The main goal was to obtain the minimum number of components that explained the greatest amount of total variance observed, and to identify the factors that students considered essential in each domain. In addition, a Confirmatory Factor Analysis (with unweighted least-squares estimation) was performed to verify whether the data confirmed the structural model obtained by the Exploratory Factor Analysis.
In view of the results obtained, we believe that this is a reliable and trustworthy assessment tool, in line with goal 3 (G3) of this research.
A total of 800 access instances to the questionnaire link were recorded, resulting in 327 completed questionnaires, representing a 70% response rate. Table 3 presents the demographic data of the students who participated in the study.
3. Results
3.1 Sample Description
Table 3
Student demographics (authors’ elaboration)
| N (%) | ||
| Gender | Female | 198 (60,6%) |
| Male | 129 (39,4%) | |
| Age (years) | 18-30 | 61 (18,7%) |
| 31-45 | 171 (52,4%) | |
| 46 – 60 | 95 (29,1%) | |
| Education | Basic | 6 (1,8%) |
| Secondary | 250 (76,5%) | |
| Undergraduate | 55(16,8%) | |
| Master | 14 (4,3%) | |
| PhD | 2 (0,6%) | |
| Academic Situation | Full-Time Student | 24 (7,3%) |
| Part-Time Student | 303 (92,7%) | |
| Employment Situation | Employed | 288 (88,1%) |
| Unemployed | 36 (11%) | |
| Never had a job | 3 (0,9%) |
The sample was mostly composed of female students, aged predominantly 31 to 45 years old. Secondary education is the most common level of completed education, and a significant portion of the sample were student workers (Table 3).
3.2 Data Analysis
For the first section on “Uses of the Internet within the OFM,” Cronbach’s alpha was 0.962, while in the second section “Online Discussion Strategies”, the Cronbach’s alpha was 0.946, reflecting a high degree of internal consistency in both sections.
All items were assessed for normality. Skewness was on average -0.114 ± 0.242 and kurtosis averaged 2.954 ± 0.435, indicating, therefore, that no transformations were required.
The exploratory factor analysis conducted on the section “Uses of the Internet within the OFM” identified 11 factors, based on the Kraiser criteria, explaining 68.68% of the variance. The Kaiser-Meyer-Olkin (KMO) coefficient was 0.930 and the result obtained from Bartlett’s test of sphericity was P < 0.001.
All items had factor loadings of greater than 0.5, and the component matrix exhibited a well-defined structure. Fifteen items with loadings below 0.5 were removed (Aires, Pereda, Palmeiro, Amante. & Nunes, 2021).
The confirmatory factor analysis confirmed that all the paths in the model were statistically significant. Fit indices for residuals indicated a good model-data fit: Root Mean Square Error of Approximation (RMSEA) = 0.028; Standardized Root Mean Square Residual (SRMR)=0.066. The Comparative Fit Index (CFI) was 0.991 and the Tucker-Lewis Index (TLI) was 0.990, indicating a good model-data fit. The Cronbach’s alpha for the remaining 59 items was 0.952, indicating a high degree of internal consistency.
In the exploratory factor analysis of the “Online Discussion Strategies” section, the Scree Plot component extraction method was applied because there were only 12 items (Hair, Black, Babin, & Anderson, 2014). The KMO coefficient was 0.942 and a P < 0.001 was obtained in Bartlett’s test of sphericity.
All items had factor loadings greater than 0.5, and the component matrix exhibited a well-defined structure. Two factors were identified, which explained 71.95% of the total variance. The confirmatory factor analysis showed that all the paths in the model were statistically significant. The fit indices for residuals indicate a good model-data fit: RMSEA < 0.001; SRMR=0.036. The CFI and TLI were both greater than 0.999, indicating a good model-data fit. The Cronbach’s alpha remains the same at 0.946, since no items were removed.
Overall, thirteen factors were identified in the exploratory factor analysis. These factors were grouped into two broad categories: seven factors related to 21st-century digital skills, and six factors associated with the determinants of digital skills (Table 4, Figures 2 and 3). Considering these two broad categories, Cronbach’s alpha values were 0.956 and 0.914, respectively, reflecting a high degree of internal consistency. When examining the factors individually, Cronbach’s alpha ranged from 0.894 to 0.940.
Figure 1: Digital Internet Skills: Categories with the highest and lowest levels (authors’ elaboration).
Figure 1 presents a comparative overview of the digital skill categories, highlighting the areas in which students obtained the highest and lowest scores. The highest-scoring categories include Information Management, Communication: Expressiveness, Critical Thinking, Perceived Ease of Use, and Autonomous Learning. These results suggest that students possess strong competencies in managing information, expressing themselves in digital communication, thinking critically, and learning independently, as well as perceiving digital tools as easy to use. The strategies adopted in online distance interactions, particularly critical thinking, yielded consistently high scores. All items in this category have a median of 4, except for the item «Use the Internet to justify [my] choices», which had a median is 3.
In contrast, the lowest-scoring categories include Communication: Networking, Communication: Knowledge Sharing, and Avoidance Learning Goal Orientation.
4. Discussion
The sample was predominantly composed of female students, the majority of which falling within the 31-45 age group. Most participants had completed secondary education, and a significant portion were student workers, highlighting the crucial role of distance education universities in lifelong learning.
The descriptive statistical analysis highlights that students are proficient in handling digital information effectively, articulating ideas clearly in digital interactions, applying critical thinking skills, and independently navigating digital learning environments with confidence in their usability. However, the findings also suggest that students face challenges in establishing academic or professional networks, actively engaging in digital knowledge-sharing practices, and maintaining a proactive learning mindset in online environments. These findings are in line with previous research indicating user proficiency in accessing information and digital communication. However, while these insights are valuable, further exploration of the collected data through factor analysis is essential to obtain more substantiated answers to the research goals and questions.
The exploratory factor analysis identified thirteen factors. In the “Uses of the Internet within the OFM”, eleven factors were identified based on Kraiser criteria, explaining 68.68% of the variance. All items had factor loadings greater than 0.5, and the component matrix had a well-defined structure. Fifteen items with loadings below0.5 were removed (Aires, Pereda, Palmeiro, Amante & Nunes, 2021). The confirmatory factor analysis showed that all the model’s paths were statistically significant. Fit indices for residuals indicated a good model-data fit: Root Mean Square Error of Approximation (RMSEA) = 0.028; Standardized Root Mean Square Residual (SRMR)=0.066. The Comparative Fit Index (CFI) was 0.991 and the Tucker-Lewis Index (TLI) was 0.990, indicating a good model-data fit. The Cronbach’s alpha for the remaining 59 items was 0.952, indicating a high degree of internal consistency.
In the exploratory factor analysis of the “Online discussion strategies” section, the Scree Plot component extraction method was applied because there were only 12 items (Hair, Black, Babin, & Anderson, 2014). All items had factor loadings greater than 0.5, and the component matrix had a well-defined structure. Two factors were identified, which explained 71.95% of the total variance. The confirmatory factor analysis showed that all the model’s paths were statistically significant. Fit indices for residuals indicate a good model-data fit: RMSEA < 0.001; SRMR=0.036. The CFI and TLI were both > 0.999, indicating a good model-data fit.
Thirteen factors were clustered into two broad categories: seven factors related to 21st-century digital skills, and six factors associated with the determinants of digital skills (Table 4).
In the following sections, we will discuss the results of each of these dimensions, in line with the main goals and questions of this study.
Table 4
Resulting factors clustered in the 21st century Digital Skills and Determinants of 21st century Digital Skills (authors’ elaboration)
| 21st Century Digital Skills | Determinants of 21st Century Digital Skills |
| Factors Categories | Factors Categories |
| Factor 1 Collaboration and Creativity | Factor 2 Ease of Use of ICT and Problem Solving |
| Factor 3 Communication: networking | Factor 4 Orientation towards learning objectives |
| Factor 5 Communication: content sharing | Factor 6 Personal Initiative |
| Factor 9 Information management | Factor 7 Autoregulation in ICT/ Internet |
| Factor 11 Information evaluation | Factor 8 Avoidance learning goal orientation |
| Factor 12 Critical Thinking: Individual perspective | Factor 10 Sharing |
| Factor 13 Critical Thinking: Social perspective |
Table 4 presents the students’ digital competences in the left-hand column and the key determining factors for these competences in the right-hand column.
4.1 Twenty-first century digital skills
The results of the Factor Analysis related to 21st-century digital skills in learning contexts are presented in Figure 2. These results show the factors, items in each factor, respective loadings, and percentage of variance explained by each factor.
Figure 2: 21st Century Digital Skills identified through Factor Analysis (authors’ elaboration).
In relation to the students’ perceptions on their digital skills at the start of their undergraduate studies at an online distance university—addressing research question 1 (Q1)—the analysis grouped the competencies into seven factors (Figure 2). In turn, these seven factors highlight three key dimensions:
- Factor 1 – Collaboration and Creativity: This factor includes joint activities and creative processes within the same context. It is the most significant, explaining 29.79% of the variance.
- Factors 3, 5, 9, and 11 – Communication and Information: These factors have high loadings and emphasize networking communication, content-sharing competences, as well as two other fundamental competencies in the learning process: managing and accessing information.
- Factors 12 and 13 – Critical Thinking: These factors highlight a key 21st-century digital competency in both social and individual dimensions. However, the individual perspective explains 64.08% of the variance, indicating a strong emphasis on personal information analysis and problem-solving competencies.
Based on these dimensions, factors, and categories identified through the factor analysis, the data provides a provisional mapping of essential digital skills for distance higher education in the 21st century. Creativity in digital learning environments is reinforced through collaboration (Dwyer et al., 2014). Additionally, the vast amount of information available online requires students to develop specific skills in managing and evaluating information (Saienko et al., 2022; Sillat & Laanpere, 2021).
Furthermore, critical thinking—a fundamental 21st-century digital competency—is particularly relevant in its individual dimension (Dewey, 1933; Hitchcock, 2018; Starkey, 2011; van Laar et al., 2020; Author, 2021) (Figure 2). This is crucial in online learning environments, where students must effectively assess and critically evaluate digital content. Given the complexity of critical thinking, this broad and multifaceted topic deserves further research to better understand its impact on 21st-century digital skills in higher education (Smith et al., 2024).
To summarize, this mapping captures the core competencies of digital literacy: collaboration, problem- solving, communication and information management, and critical thinking in its individual dimension. These results align with the studies of van Laar et al. (2019b; 2020) and Cabezas-González et al. (2022) and provide a comprehensive response to the first research question (Q1): what are the students’ perspectives on their digital skills as they begin their undergraduate studies at an online distance university?
4.2 Determining factors of Digital Skills
Regarding the social and individual determining factors of students’ digital skills, addressed under research question 2 (Q2 -What are the social and individual determinants of the digital skills of students?), we identified six factors. Figure 3 shows the factor items and their respective loadings, as well as the percentage of the total variance explained by each factor. These factors include ease of use of ICT in problem solving (Factor 2), orientation towards learning goals (Factor 4), personal initiative (Factor 6), autoregulation in ICT/Internet use (Factor 7), avoidance of learning goal orientation (Factor 8), and sharing.
Figure 3: Determinants of 21st century digital skills identified through Factor Analysis (authors’ elaboration).
At this stage of the analysis, we found a strong relationship between the Ease of Use of the Internet and Online Problem-Solving in digital educational settings. This is a crucial factor influencing the development of digital skills, highlighting the importance of the OFM (Online Facilitation Model) for newly enrolled students in shaping their learning processes.
The students’ perceptions of the ease of using digital technologies in problem-solving within learning contexts emerged as a key phenomenon that warrants further investigation.
Our analysis identified six determining factors of digital skills, each contributing to different aspects of learning. The factors with the highest percentage of explained variance were:
- Ease of Use of ICT and Problem Solving (9.26%) – The most significant factor, reflecting how students perceive technology as a tool for learning and problem-solving.
- Orientation Towards Learning Goals (4.92%) – Highlighting students’ ability to align their use of digital tools with their learning goals.
Other important individual competencies that emerged as determinants of digital skills in learning environments included:
- Personal Initiative (3.18%) – Capturing students’ motivation to take ownership of their learning process.
- Autoregulation in ICT/Internet use (2.70%) – Reflecting students’ ability to manage their own learning process in digital environments.
- Avoidance of Orientation Towards Learning Goals (2.51%) – A factor that suggests some students may struggle to set and follow structured learning goals.
However, Sharing (Factor 10) appeared as the least significant factor, explaining 2.14% of the variance, as it included only two items with limited relevance. The low regularity in sharing may be attributed to the fact that the questionnaire was administered to new students, meaning that insufficient time had passed for them to establish a learning community (Figure 3).
These determining factors also emphasize a critical yet often overlooked issue in research on the digital gap and digital skills: the causal role of personal competencies in developing digital and Internet skills in learning environments (Factors 2, 4, 6, 7) (van Laar et al., 2020).
On the one hand, the design of online, hybrid, or digitally-mediated learning environments must consider how students’ digital literacy levels influence their learning quality, particularly in early academic stages. On the other hand, when designing training programs to develop digital skills, it is crucial to account for the individual and social factors influencing their acquisition and development.
These findings are in line with existing literature (Sezgin & Fırat, 2024; Smith et al., 2024; Cabezas-González et al., 2022) and are equally relevant for the development and assessment of competencies in highly digitalized work environments.
The analysis of key areas confirms that the data collection tool used in this study is both reliable and credible. It has contributed valuable insights into the need for improved frameworks, systems and tools to assess digital proficiency effectively. The findings align with Sillat et al. (2021) and Peláez-Sánchez et al. 2024), who emphasize the challenges of evaluating digital skills and the failure of existing systems to establish systematic assessment processes. Thus, this study highlights the urgent need for more structured and field-specific methods and tools for assessing digital competencies.
5. Conclusions
Students entering an online distance education university must possess both digital competencies and personal attributes that support their adaptation to the demands of this educational model. This study aimed to broaden understanding in this domain by analyzing digital competencies of newly enrolled students at an online distance education university (G1), exploring the factors determining the acquisition of digital/Internet competencies, and optimizing an assessment tool for digital competencies (G3).
The results highlight that Communication and Collaboration, Networking and Sharing, and Critical Thinking are core competencies. They are strongly associated with Ease of Use of ICT, Problem-Solving, and Orientation Towards Learning Goals, emphasizing students’ ability to align digital tools with their academic goals. Additionally, other key competencies emerged that influence digital skill development, such as Personal Initiative, Self-Regulation in ICT/Internet Use, and Attitudes Towards Learning Goals, which are fundamental soft skills in the 21st century.
The pedagogical implications of these findings are significant, given the importance of digital competencies in learning processes and academic success. The findings underscore the need for educational institutions to design training programs more focused on those competencies than on instrumental digital skills. Moreover, this study addresses a critical gap identified in recent literature: the need for improved methods and tools to assess digital competencies in the 21st century.
Notwithstanding, some limitations of this study should be noted. The exploratory nature of the research and the intentional sampling strategy limit the generalizability of the findings. Additionally, the homogeneity of participants in terms of study level and educational modality, as well as the assessment tool’s focus on self-perceived competencies rather than actual digital performance, should be considered when interpreting the findings.
Despite these limitations, this study offers valuable insights into digital literacy in distance education, highlighting its role in student success and the development of educational strategies in online universities. Future research can advance this field by driving innovation at multiple levels: in research, by developing more comprehensive methods and tools to evaluate digital competencies; in pedagogical models, by integrating digital skills development as a central component of the learning process; and in university policies, by promoting digital literacy training tailored to the students’ needs.
Funding
This research was financed by national funds through FCT— Fundação Para a Ciência e a Tecnologia, I.P., through the project UIDB/04372/2020.
Data availability statement
The anonymized raw data supporting the conclusions of this research will be made available by the authors.
Author Contributions
Luísa Aires: Conceptualization, Project administration, Methodology, Formal analysis, Supervision, Validation, Writing – original draft. Lúcia Amante: Data curation, Formal analysis, Writing – original draft. Esther Fernández-Márquez: Writing – review & editing. Catarina Nunes: Formal analysis, Visualization, Writing – original draft.
Ethics statement
The study was approved by the Ethics Committee of the Laboratory of Distance Education and eLearning (LE@D, FCT R&DU 4372), Universidade Aberta, Portugal.
All participants provided written informed consent to participate in this study.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as potential conflicts of interest.
Referencias
Aires, L., Pereda, V., Palmeiro, R., Amante, L., & Nunes, C. (2021). Competências digitais no séc. XXI: instrumentos de autoavaliação. LE@D, Universidade Aberta, https://doi.org/10.34627/leadw.2021.1
Almerich, G., Suarez-Rodríguez, J., & Días-Garcia, I., Cebrián-Cifuentes, S. (2019). 21st-century competences: The relation of ICT competences with higher-order thinking capacities and teamwork competences in university students. Journal of Computed Assisted Learning, 36(4) 1–12. https://doi.org/10.1111/jcal.12413
Aretio, L. (2021). COVID-19 y educación a distancia digital: preconfinamiento, confinamiento y posconfinamiento. RIED. Revista Iberoamericana de Educación a Distancia, 24(1), 09–32. https://doi.org/10.5944/ried.24.1.28080
Cabezas-González, M., Casillas-Martín, S., & García-Valcárcel Muñoz-Repiso, A. (2022). Mediation Models Predicting the Level of Digital Competence of 12–14-Year-Old Schoolchildren in the Area of Digital Problem Solving. Journal of New Approaches in Educational Research, 11(2), 168-185. https://doi.org/10.7821/naer.2022.7.789
Casillas-Martín, S., Cabezas-González, M., & García-Valcárcel, A. (2020). Psychometric analysis of a test to assess the digital competence of compulsory education students. Relieve, 26(2). https://doi.org/10.7203/relieve.26.2.17611
Cronbach, L. J. (1984). Essentials of psychological testing (4th ed.) Harper & Row.
De Pablos, J. (2018) Las tecnologías digitales y su impacto en la Universidad. Las nuevas mediaciones. RIED. Revista Iberoamericana de Educación a Distancia 21(2): 83–95, https://doi.org/10.5944/ried.21.2.20733
Dewey, J. (1933). How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process. D.C. Heath.
Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity, 12, 43-52. https://doi.org/10.1016/j.tsc.2013.12.004
Fernández-Gómez, E.; López-Martínez, A.; Feijoo-Fernández, B. (2022). Experiencia docente en la enseñanza de competencias online en el grado en Publicidad. Campus Virtuales, 11(2), 87-96. https://doi.org/10.54988/cv.2022.2.1069
Gros Salvat, B., Lara Navarra, P., García González, I., Mas García, X., López Ruiz, J., Maniega-Legarda, D. & Martínez, T.(2009). The UOC’s educational model. Evolution and future perspectives. Barcelona: UOC. https://openaccess.uoc.edu/handle/10609/369
Gutiérrez, K. D., Morales, P. Z., & Martinez, D. C. (2009). Re-mediating Literacy: Culture, Difference, and Learning for Students From Nondominant Communities. Review of Research in Education, 33(1), 212-245. https://doi.org/10.3102/0091732X08328267
Hair, J. F., Black, W.; Babin, B., & Anderson, R. E. (2014). Multivariate Data Analysis. Pearson.
Hargittai, E. (2021). Chapter 1: Introduction to the Handbook of Digital Inequality. In Eszter Hargittai (ed.), Handbook of Digital Inequality, (pp. 1-7). Edward Elgar Publishing, https://doi.org/10.4337/9781788116572.00006
Helsper, E. (2019) Why location-based studies offer new opportunities for a better understanding of socio-digital inequalities? In Barbosa A. (Coord.) Desigualdades Digitais no Espaço Urbano: Um Estudo Sobre o Acesso e o Uso da Internet na Cidade de São Paulo (pp.19-44). Núcleo de Informação e Coordenação do Ponto BR, https://cetic.br/media/docs/publicacoes/6/Desigualdades_Digitais_no_Espaco_Urbano.pdf.
Helsper, E.J., van Deursen, A.J.A.M. & Eynon, R. (2016). Measuring Types of Internet Use. From Digital Skills to Tangible Outcomes project report. https://bit.ly/3DDgOX8
Hitchcock, D. (2018). Critical thinking, The Stanford Encyclopedia of Philosophy, The Metaphysics Research Lab, Department of Philosophy, Stanford University. https://plato.stanford.edu/archives/fall2018/entries/critical- thinking/
Koch, C., & Fehlmann, F. (2025). Beyond digital literacy: Exploring factors affecting digital performance of university staff.
Media and Communication, 13, Article 8913. https://doi.org/10.17645/mac.8913
Monferrato, M. (2024). The role of the university in developing digital competencies and reducing skills mismatch from a human capital perspective for the 21st century. Debate Universitario CAEE-UAI, 14(23), 45–57. https://dialnet.unirioja.es/servlet/articulo?codigo=9396798
Mendes, M.A., Bastos, G., Amante, L., Aires, L., Cardoso, T. (2019) – Virtual pedagogical model [Em linha]: development scenarios. Lisboa: Universidade Aberta, 41 p. (eUAb. MPV_Inovação). ISBN 978-972-674-851-9, https://repositorioaberto.uab.pt/handle/10400.2/9710
Mohammadyari, S., & Singh, H. (2015). Understanding the effect of e-learning on individual performance: The role of digital literacy. Computers & Education, 82, 11-25. https://doi.org/10.1016/j.compedu.2014.10.025.
OECD (2018). The future of education and skills. Education 2030. Paris. https://bit.ly/3Hr9XBn
Pakhomova, T., Hryhorieva, V., Omelchenko, A., Kalenyk, M., & Semak, L. (2023). The formation of digital competence by means of information and communication technologies among students of higher education. Revista Eduweb, 17(2), 78–88. https://doi.org/10.46502/issn.1856-7576/2023.17.02.7
Palmeiro, R., Pereda, V. & Aires, L. (2019). Digital inclusion programs: The case of the Basque country,Programmes d’inclusion numérique: Le cas du pays Basque,Programas de inclusão digital: O caso do país Basco, Programas de inclusión digital: El caso del país Vasco». Revista Lusofona de Educação, 45, 73-88. https://revistas.ulusofona.pt/index.php/rleducacao/article/view/6789
Peláez-Sánchez, I. C., Glasserman-Morales, L. D., & Rocha-Feregrino, G. (2024). Exploring digital competencies in higher education: Design and validation of instruments for the era of Industry 5.0. Frontiers in Education, 9, 1415800. https://doi.org/10.3389/feduc.2024.1415800
Pereira, A., Mendes, A., Morgado, L., Amante, L. & Bidarra, J. (2007). Modelo pedagógico virtual da Universidade Aberta: para uma universidade do futuro. Universidade Aberta. Lisboa, https://repositorioaberto.uab.pt/bitstream/10400.2/1295/1/Modelo%20Pedagogico%20Virtual.pdf
Perera, U., & Gardner, L. (2018). Analyzing the relationships between digital performance: The role of digital literacy. Computers & Education, 82, 11-25. https://doi.org/10.1016/j.compedu.2014.10
Pham Tra, L., & Dau Thi Kim, T. (2024). Digital competence of students in higher education: An assessment framework in Vietnam. Asian Education and Development Studies. Advance online publication. https://doi.org/10.1108/AEDS- 04-2024-008
Saienko, V., Kurysh, N., & Siliutina, I. (2022). Digital Competence of Higher Education Applicants: New Opportunities and Challenges for Future Education. Futurity Education, 2(1), 45–54. https://doi.org/10.57125/FED/2022.10.11.23
Schwarz, S., Bieg, T., Svecnik, E., Schmölz, A., Geppert, C., & Gerdenitsch, C. (2024). Digital Competence Scale (DCS): A short self-assessment instrument for measuring digital competences. Nordic Journal of Digital Literacy, 19(3), 126–143. https://doi.org/10.18261/njdl.19.3.2
Sezgin, S., & Fırat, M. (2024). Exploring the digital divide in open education: A comparative analysis of undergraduate students.The International Review of Research in Open and Distributed Learning, 25(1), 109–126. https://doi.org/10.19173/irrodl.v25i1.7236.
Sillat, L. H., Tammets, K., & Laanpere, M. (2021). Digital Competence Assessment Methods in Higher Education: A Systematic Literature Review. Education Sciences, 11(8), 402. https://doi.org/10.3390/educsci11080402
Smith, A., Johnson, L., & Wang, M. (2024). Exploring digital competencies in higher education: Design and validation of a comprehensive assessment instrument. Frontiers in Education, 9. https://doi.org/10.3389/feduc.2024.1415800
van Deursen, A.J.A.M., Helsper, E., Eynon, R., & van Dijk, J. (2017). The Compoundness and Sequentiality of Digital Inequality. International Journal of Communication, 11, 452–473. http://ijoc.org.
van Dijk, J. (2020). Closing the Digital Divide. The Role of Digital Technologies on Social Development, Well-Being of AI and the Approach of the Covid-19 Pandemic. The United Nations. https://bit.ly/3HTOt1e
van Laar, E., van Deursen, A., van Dijk, J., & de Haan, J. (2017). The relation between 21st-century skills and digital skills or literacy: A systematic literature review, Computers in Human Behavior, 72, 577-588. https://doi.org/10.1016/j.chb.2017.03.010.
van Laar, E., van Deursen, A., van Dijk, J., & de Haan, J. (2019a). The Sequential and Conditional Nature of 21st-Century Digital Skills. International Journal of Communication 13, 3462–3487 1932–8036/20190005, the Netherlands Institute for Social Research SCP. http://ijoc.org
van Laar, E., van Deursen, A., van Dijk, J., & de Haan, J. (2019 b). Determinants of 21st-century digital skills: A large-scale survey among working professionals. Computers in human behavior, 100, 93-104. https://doi.org/10.1016/j.chb.2019.06.017
van Laar, E., van Deursen, A. J. A. M., van Dijk, J. A. G. M., & de Haan, J. (2020). Determinants of 21st-Century Skills and 21st-Century Digital Skills for Workers: A Systematic Literature Review. SAGE Open, 10 (1) 1–14. https://doi.org/10.1177/2158244019900176
Vuorikari, R., Punie, Y., Carretero, S., & Van Den Brande, L. (2016). DigComp 2.0: The Digital Competence Framework for Citizens. Update Phase 1: The Conceptual Reference Model. Publication Office of the European Union. Retrieved from https://doi.org/10.2791/11517, https://doi.org/10.2791/11517
Willems, J., Farley, H., & Campbell, C. (2019). The increasing significance of digital equity in higher education: An introduction to the Digital Equity Special Issue. Australasian Journal of Educational Technology, 35(6), 1–8. https://doi.org/10.14742/ajet.5996
Wu, X., Wang, Z., Wang, Q., Chen, W., & Pi, Z. (2021). Supporting digitally enhanced learning through measurement in higher education: development and validation of a university students’ digital competence scale. Journal of Computer Assisted Learning, 37(4), 1063-1076. https://doi.org/10.1111/jcal.12546
Zhao, Y., Pinto Llorente, A.M., Sánchez Gómez, M. C. (2021). Digital competence in higher education research: A systematic literature review. Computers & Education, Volume 168, 2021, 104212, https://doi.org/10.1016/j.compedu.2021.104212