Páginas: 267-282

Recibido: 2022-12-06

Revisado: 2022-12-14

Aceptado: 2023-03-25

Preprint: 2023-04-21

Publicación Final: 2023-09-01

 

 

www.revistascientificas.us.es/index.php/fuentes/index

DOI: https://doi.org/10.12795/revistafuentes.2023.22642

 

Development and psychometric properties of a questionnaire on environmental literacy in Indonesian secondary school

Desarrollo y propiedades psicométricas de un cuestionario sobre alfabetización ambiental en la escuela secundaria de indonesia

Mujib Mujib
Universitas Islam Negeri Raden Intan Lampung (Indonesia)

Mardiyah Mardiyah
Universitas Islam Negeri Raden Intan Lampung (Indonesia)

Suherman Suhermen
University of Szeged (Hungary)

 

 

 

 

Abstract

Concurrently, the amount of research dedicated to mitigating the effects of environmental issues is increasing. Environmental Education has emerged as one of the most promising research fields in preventing and addressing environmental problems. Nevertheless, it is important to note that environmental literacy is not limited by age, education level, or occupation, particularly in the Indonesian context. Therefore, this study aimed to evaluate the psychometric properties of an environmental literacy questionnaire for secondary school students. The sample consisted of 1,021 students from both public and private schools in Indonesia, ranging from grades 7-9. A three-factor model, including environmental consciousness, environmental anxiety, and environmental awareness, was validated using confirmatory factor analysis (CFA). Model fit was assessed using a variety of fit indices, such as the chi-squared test, comparative fit index (CFI), Tucker-Lewis Index (TLI), Good-Fit Index (GFI), root mean square approximation error (RMSEA), standardized root mean square residual (SRMR), and KMO index. Cronbach Alpha and McDonald's Omega were used to evaluate the reliability of the three factors, with values ranging from 0.66 to 0.78 and 0.68 to 0.79, respectively. The results indicate that the environmental literacy questionnaire is a reliable and valid tool for assessing the environmental literacy of secondary school students in the Indonesian context.

 

Resumen

Al mismo tiempo, la cantidad de estudios sobre la mitigación de los efectos de los problemas ambientales está creciendo. La Educación Ambiental es uno de los campos de investigación más prometedores en términos de prevención y consecuencias de los problemas ambientales. Sin embargo, la alfabetización ambiental no está limitada por la edad, el nivel educativo o la ocupación, especialmente en el contexto indonesio. Este estudio tuvo como objetivo evaluar psicométricamente un cuestionario de alfabetización ambiental para la educación secundaria. La muestra del estudio estuvo compuesta por 1.021 estudiantes de 7º a 9º grado de escuelas secundarias seleccionadas de escuelas públicas y privadas en Indonesia. La validez de la estructura de tres factores del modelo fue evaluada mediante análisis factorial confirmatorio (CFA), a saber, la conciencia ambiental, la ansiedad ambiental y la conciencia ambiental. Después del CFA, se utilizaron índices de ajuste para evaluar el ajuste del modelo, como la prueba de chi-cuadrado, el índice de ajuste comparativo (CFI), el índice de Tucker-Lewis (TLI), el índice de buen ajuste (GFI), el error de aproximación de la raíz cuadrada media (RMSEA), el residuo cuadrático medio estandarizado (SRMR) y el índice KMO. Los valores de confiabilidad para los tres factores oscilaron entre 0,66 y 0,78 con Alfa de Cronbach, mientras que los de Omega de McDonald oscilaron entre 0,68 y 0,79. El cuestionario de alfabetización ambiental es un instrumento viable para evaluar la alfabetización ambiental de los estudiantes de secundaria en el contexto indonesio.

 

Palabras clave / Keywords

alfabetización ambiental, escuela secundaria, validación psicométrica, análisis factorial confirmatorio, mapa de Wright, psicométrico, educación, desarrollo

environmental literacy, secondary school, psychometric validation, confirmatory factor analysis, wright map, psychometric, education, development

1. Introduction

 

Every day, more and more individuals are becoming aware of and affected by environmental challenges. The media regularly reports on the repercussions of global warming and ozone layer depletion, which are felt worldwide. Meanwhile, research into mitigating the effects of environmental issues is on the rise. Some studies aim to eliminate the effects of environmental problems, while others concentrate on preventing them from occurring in the first place, as this approach is simpler and more effective. Environmental Education is one of the most promising research fields for preventing environmental problems and their consequences. Human actions are primarily responsible for environmental issues (Saribas et al., 2017), and therefore humans must be responsible for solutions and processes (Suherman et al., 2020). Thus, boosting environmental awareness and concern among the population is crucial in finding solutions.

Environmental education, a term first introduced at the Tbilisi Congress in 1977, can be defined as the process of educating individuals to obtain information and skills necessary to improve motivation and attitudes towards proposing ideas for solutions to environmental problems (UNESCO, 1978). Environmental education is important as it leads to a higher level of environmental literacy among individuals (Atabek-Yiğit et al., 2014). An environmentally literate person, according to Janmaimool & Khajohnmanee (2019a), understands the relationship between nature and social systems, acknowledges human attitudes and the impact of technological development on the environment, and is aware that environmental knowledge can be acquired throughout life. Knowledge, attitude, behaviour, and consciousness are the four components of environmental literacy, all of which an individual must possess to be deemed environmentally literate (Harring & Jagers, 2018). Students can learn environmental literacy through their activities (Arsyad & Villia, 2022; Corebima et al., 2017).

The foremost way to address environmental issues is through environmental education and environmental literacy. From this perspective, it is evident that environmental literacy must be fostered. Environmental literacy in Indonesia is a growing concern as the country faces various environmental issues, such as deforestation, air pollution, and plastic waste, among others. According to a study conducted by Sudarmadi et al. (2001), Indonesian students lack knowledge and understanding of environmental issues, resulting in low environmental literacy levels. The study involved 434 high school students from three different regions in Indonesia, and the results showed that the average score for environmental literacy was only 46.2 out of 100.

One of the main problems with environmental literacy in Indonesia is the lack of education and awareness about environmental issues. According to a report by the Indonesian Ministry of Environment and Forestry, only a few schools in the country integrate environmental education into their curricula (UNESCO, 2016). Furthermore, the level of environmental literacy among students is low, and there is a lack of teacher training in environmental education (Safitri et al., 2020). The study also noted that the current environmental education curriculum in Indonesia is insufficient to promote environmental literacy. Another issue is the lack of public awareness and participation in environmental protection. Many young people in Indonesia do not identify themselves as environmentalists and often lack the knowledge, resources, and opportunities to act on their attitudes in their daily lives (Parker et al., 2018). Additionally, there is a lack of public participation in environmental decision-making processes, such as environmental impact assessments and public hearings, which hinders the implementation of effective environmental policies and regulations (Swangjang, 2018). In summary, the problems of environmental literacy in Indonesia include the lack of education and awareness about environmental issues, shortage of qualified teachers, inadequate funding and resources, lack of public awareness and participation, and inadequate implementation of environmental policies and regulations. These issues require urgent attention and action from the government, education sector, and civil society to improve environmental literacy and promote sustainable development in Indonesia.

Research has been conducted on the development and validation of environmental literacy at various scales. For instance, Erdoğan (2003) developed a scale to assess the attitudes of college students toward environmental issues, and Dunlap et al. (2000) enhanced the New Environmental Paradigm scale. Berberoglu & Tosunoglu (1995) created the Environmental Attitude Scale (EAS) after conducting a validity and reliability study with Turkish university students, identifying four variables and forty-seven items. Teksöz et al. (2010) originally created the Environmental Literacy Scale at Michigan State University, and Leeming et al. (1995) established the Children's Environmental Attitudes and Knowledge Scale. Alp et al. (2006) administered a Turkish version of the measure to sixth, eighth, and tenth-grade students to assess their environmental attitudes and knowledge, as well as the influence of grade and gender on environmental attitudes. Naim & Sağlam (2006) created the Environmental Attitude Scale for high school students.

Although research on environmental literacy has been conducted at different levels, it is not limited by age, education level, or occupation, as protecting the environment is everyone's responsibility. Nurhidayati et al. (2022) even developed a scale to measure students' concern for the local potential of natural resources in their environment. However, most of the previous research was conducted on university students, prospective teachers, and elementary or high school students, and developed in Western and Middle Eastern countries to assess their attitudes, knowledge, or environmental literacy. This study aims to develop an inventory of environmental literacy using Confirmatory Factor Analysis (CFA) in junior high school students in the Indonesian context, which is a novel contribution to the existing literature.

 

 

2. Literature Review

 

2.1. Environmental Literacy

 

The definition of environmental literacy states that an environmentally literate person has a basic skill, understanding, and feeling about the human-environment relationship. Such a person must comprehend the interrelationship between natural and social systems, the unity of humans and nature, how technology affects decision-making on environmental issues, and that learning about the environment is a lifelong endeavour. According to Cristovão et al. (2022), environmental literacy refers to "the knowledge, skills, values, and attitudes necessary to understand and participate in activities aimed at solving environmental problems and creating sustainable societies" (p. 77). It requires a deep understanding of environmental issues, their causes, and potential solutions, as well as the ability to critically evaluate and communicate environmental information. Environmental literacy also involves a sense of responsibility and motivation to take action to protect the environment and promote sustainable practices.

Thus, "the need for capable awareness, knowledge, skills, and attitude funds to incorporate appropriate environmental considerations in making decisions for consumption, lifestyle, career, and citizenship both individually and in groups." Kelani argues that "Environmental literacy is the 'knowledge necessary to comprehend relatedness, and an attitude of care or stewardship” (Kelani, 2017). Additionally, "Environmental literacy is essentially the capacity to perceive and interpret the relative health of the environmental systems and to take appropriate action to maintain, restore or improve the health of those systems" (Lloyd‐Strovas et al., 2018). It can be concluded from these views that environmental literacy is a skill possessed by a person or people in understanding and interpreting action and then acting efficiently and effectively in accordance with the environmental context.

 

2.2. Assessment of an Environmental Literacy

 

There are numerous dimensions to the assessment tools used to measure environmental literacy. In this perspective, we have developed three dimensions: environmental consciousness level, environmental anxiety level, and environmental commitment level. The environmental consciousness level refers to personal environmental awareness demonstrated by support for environmental sustainability based on values centered around a philosophy of life that prioritizes environmental values and the natural environment (Arifin et al., 2021). Examples of instruments in this category include statements like "I believe that the government should support renewable energy sources (solar, wind, water, geothermal)," "Environmental education should be provided from the beginning of basic education to promote environmental awareness," and "I will use a recycling box if available" (Atabek-Yiğit et al., 2014). The environmental anxiety level can be described as a range of subjective and usually unpleasant sensations regarding environmental problems (Gao et al., 2021), such as worry, stress, and anger (Gao et al., 2021). Indicators of this level include statements like "I think we will not find a place for a picnic in a few generations," "I think everyone should plant a tree in their life," and "I think plants should be saved for future life" (Atabek-Yiğit et al., 2014). Finally, the environmental commitment level refers to an individual's dedication and engagement in pro-environmental behaviors and practices, such as reducing energy consumption, recycling, and reducing waste. It encompasses attitudes, beliefs, and behaviors that contribute to sustainable living and environmental protection (Grilli & Curtis, 2021). Indicators of this level include statements like "I want to learn about environmental issues," "I prefer to buy environmentally friendly goods rather than economical ones," and "I prefer to use public transportation rather than private transportation to protect the environment" (Atabek-Yiğit et al., 2014).

3. Method

 

The study was conducted on secondary school students in Lampung province, Indonesia. Sampling for the study used random sampling. The sample of 1021 consisted of grades 7 to grade 9 with an age range of 11-16 years (M = 13.69; SD = 0.963). The research was conducted using a google form, due to the pandemic situation. The characteristics of the research sample are as follows:

 

Table 1

Characteristics of the research sample

 

Characteristics

Frekuensi

n

%

Gender

 

 

     Female

623

61.3

     Male

395

38.7

Grade

 

 

     7

294

28.8

     8

394

38.6

     9

333

32.6

School Type

 

 

    Public

332

32.5

    Private

689

67.5

Place Type

 

 

    City

384

37.6

   District

637

62.4

Ethnic

 

 

    Java

633

62.0

    Lampung

206

20.2

    Sunda

24

2.4

    Batak

54

5.3

    Padang

22

2.2

    Bugis

3

0.3

   Others

79

7.7

 

3.1. Instrument

This study adapted the ELSA instrument developed by Atabek-Yiğit et al. (2014). This instrument was developed in the Indonesian context with 3 scales/inventories namely Environmental Consciousness Level of 9 items, Environmental Anxiety Level of 6 items, and Environmental Commitment level of 5 items with a total of 20 items of questions. Alternative responses use five Likert scales, including strongly agree with a value of 5 to strongly disagree with value of 1. There are some instruments to evaluate environmental literacy. The instruments are in Table 2.

 

Table 2

Instruments of environmental literacy

 

Study

Instrument

Number of items

Psychometric properties

(Atabek-Yiğit et al., 2014)

Environmental Literacy Scale for Adults (ELSA)

29 items

Cronbach-Alpha for the whole scale = 0.881; 1st dimension =  0.807; 2nd dimension = 0.765; 3rd dimension = 0.715. 

(N = 332)

(Lloyd‐Strovas et al., 2018)

Environmental Literacy Instrument (ELI)

The ELI instrument consists of 40 items, with 10 items in each of the four domains: knowledge, attitudes, behaviors, and skills.

Cronbach's alpha coefficients for the environmental knowledge, attitudes, and behaviors subscales were 0.84, 0.82, and 0.71, respectively.

(Tuncer Teksoz et al., 2014)

ELQ: environmental literacy questionnaire

The items were 12 assess four components (knowledge, attitude, attitude towards environmental responsibility and concern).

N = 648. Data is valid and reliable based on Rasch measurement.

(Negev et al., 2008)

The Middle School Environmental Literacy Instrument

Three dimensions including environmental knowledge, attitudes, and behavior.

N = 3121. Validity and reliability used Cronbach’s

(Szczytko et al., 2019)

The Environmental Literacy Instrument for Adolescents (ELI-A)

Three dimension, namely,

Ecological knowledge, Hope, Behaviour

N = 665. Data analysis using CFA and SEM.

 

The above researchs lend support to the construct validity as a measurement of the scale. However, there is a paucity of research into environmental litracy in Indonesia, particularly in the context of secondary education. Recent research on environmental literacy has primarily focused on adolescents, teachers, and western populations (De Leyn et al., 2022). Unfortunately, we didnt see of the environmental literacy inventory in the Indonesian context. This is prior reasons that we need develope and looking for psychometric properties of instrument in Indonesian context.

 

3.2. Research Design

The research has 3 stages, the first is to translate the original English questionnaire by experts, in this case it will be validated by mathematics and language experts, namely Ph.D. students in the UK, Japan, and Hungary. The next step is to distribute the questionnaire to junior high school students in Lampung province. The final stage is to analyze the questionnaire results using CFA.

 

3.3. Data Analysis

Data analysis was conducted using SPSS Version 26.0 and Jeffreys's Amazing Statistics Program (JASP) version 0.14.1. Confirmatory Factor Analysis (CFA) was used to check the model fit in the measurement model (Jomnonkwao & Ratanavaraha, 2016), and JASP version 0.14.1 was used for the analysis. Following CFA, several fit indices were used to evaluate the model fit, including the chi-squared test, comparative fit index (CFI), Tucker-Lewis Index (TLI), Good-Fit Index (GFI), root mean square approximation error (RMSEA), standardized root mean square residual (SRMR), and KMO index (Kline, 2015). The chi-square statistic, including degrees of freedom and p-values, is represented mathematically. According to Kline (2015), the “Chi-square test statistic” is extremely sensitive to sample size, with statistically significant chi-square values being more frequently observed with larger samples. Therefore, the CFI value, which is insensitive to sample size, was considered. Values greater than 0.90 indicate that the model is fit and acceptable. Furthermore, a GFI value higher than 0.85 indicates a good fit (Hair et al., 2010), while an RMSEA with a range between 0.03 and 0.08 is considered a good model fit. To assess reliability, alpha reliability and composite Cronbach alphas were examined. “Internal consistency (Crbα; Cronbach's alpha)” and “composite reliability (ω; McDonald's omega coefficient; (Raykov, 1997)” were used as measures of reliability. As previously stated by Habók & Magyar (2018), values greater than 0.70 indicate favourable results for empirical research. In addition, construct reliability (CR) should be > 0.70, and the average variance extracted (AVE) should be more than 0.50. Lower values are acceptable when the CR value is greater than 0.60; however, “lower values are not acceptable when the CR value is less than 0.60” (Fornell & Larcker, 1981). Discriminant validity was assessed using HTMT, with a threshold value of 0.85 being acceptable (Kline, 2015).

 

 

4. Results

 

4.1. EFA

 

EFA is commonly used when the relationship between observed instrument variables is unclear (Glynn et al., 2011). As ELSA is a three-factor instrument specifically designed for an Indonesian context and based on aspect-based factors, EFA was used to analyze the students' responses to the questionnaires in this study. The results of the EFA were consistent with the average Bartlett's test of sphericity (Chi-square = 779.719; DF = 132; p < 0.001) and the KMO measure of sampling adequacy (KMO = 0.886), which indicated that the instrument was reliable and provided a high-quality sample for further analysis. These results were consistent with previous research (Field, 2013; Rytkönen et al., 2007; Suherman & Vidákovich, 2022a).

4.2. Reliability

Reliability measures the internal consistency of respondents' answers across questions in an instrument. Since all the items in this research instrument are intended to capture the same underlying construct, respondents' scores should be correlated with each other (Wieland et al., 2017). Cronbach's alpha is used to assess internal consistency, while McDonald's Omega is used to determine composite reliability. The results of the analysis are presented in the following table:

 

Table 3

Consistency reliability and composite reliability

 

Factors

Environmental Consciousness Level

0.77

0.78

Environmental Anxiety Level

0.78

0.79

Environmental Commitment Level

0.66

0.68

Note. , Cronbach’s alpha; , McDonald’s coefficient omega

 

The results of Table 3, show that the Cronbach Alpha range is 0.66 to 0.78 while at McDonald's Omega, it is in the range of 0.68 and 0.79 for the three factors. This indicates that within these ranges, reliability is acceptable.

 

4.3. Construct Validity

 

4.3.1. Convergent Validity

Convergent validity is a measure of the degree of correlation between multiple variables within the same construct of an instrument. This means that convergent validity is established when the variables within a factor are strongly related. To achieve convergent validity, we need to evaluate the convergent reliability (CR), factor loadings, and average variance extracted (AVE) (Ab Hamid et al., 2017). Typically, as the sample size decreases, the loading score requirement increases. However, regardless of sample size, it is recommended to have loading scores greater than 0.5 for each element. Each composite factor's AVE threshold should be better than 0.5, and the CR threshold should be greater than 0.70 (Hair Jr et al., 2021). Even if the AVE value is less than 0.5 and the CR is greater than 0.6, the convergent validity of the construct still meets the minimum requirements (Fornell & Larcker, 1981; Malhotra & Dash, 2011). The AVE and CR values were obtained using the major validity instrument (Gaskin & Lim, 2016), and EFA was used to calculate the factor loadings. The results of the item loading score values determined by the convergent validity test using CR and AVE are displayed in Table 4.

 

Table 4

Convergent validity

 

Factors

CR

AVE

Environmental Consciousness Level

0.80

0.35

Environmental Anxiety Level

0.79

0.39

Environmental Commitment level

0.69

0.37

 

Table 4 shows that the CR scores for the questionnaire's three latent components range from 0.69 to 0.80 respectively, while AVE values range between 0.35 and 0.39. Although there are low AVE values, the convergent validity and reliability can still be established based on CR alone, even if AVE is often too strict. This is because AVE is a measure of the degree of correlation between two variables (Fornell & Larcker, 1981; Malhotra & Dash, 2011).

 

4.3.2. Discriminant Validity

Discriminant validity is used to assess the extent to which latent factors differ from each other empirically (Hair et al., 2010; Henseler et al., 2015). In this study, we also used a new criterion for assessing discriminant validity, the Hetero Trait Mono Trait (HTMT). Conceptually and practically, the HTMT value threshold should be below 0.9 and 0.85, respectively (Henseler et al., 2015). Table 5 below presents the results of the study.

 

 

Table 5

Rasio with HTMT0.85

 

 

ECoL

EAnL

ECL

ECoL

-

0.66

0.63

EAnL

-

0.66

0.63

ECL

 

-

0.84

ECol: Environmental Consciousness Level; EanL: Environmental Anxiety Level; ECL: Environmental Commitment level

 

Table 5 explains that the value of HTMT0.85 establishes discriminant validity less than 0.85. The range of values is 0.66 to 0.84. So discriminant validity is accepted less than 0.85 (Hair et al., 2010; Henseler et al., 2015).

 

4.4. CFA

 

The aim of this study is to evaluate the measurement model using CFA and JASP. CFA serves to confirm the latent factors in the measurement model, ensuring their adequate operation and achieving the GoF index. This helps researchers identify relationships between latent factors and develop hypotheses in subsequent studies with greater confidence (O’Byrne et al., 2018). To ensure the level of quality, we conducted analyses to determine CR, convergent validity, and discriminant validity, following the recommendations of Chuah et al. (2016). Utilizing the pattern matrix builder plugin developed by Tabachnick et al. (2007), we constructed CFA diagrams in the measurement model to evaluate model fit. The CFA results in the table below confirmed the appropriateness of the model

 

 

Table 6

Model fit

 

Index

Value

CFI

0.933

TLI

0.922

RMSEA

0.069

SRMR

0.077

GFI

0.962

 

Table 6 presents the analysis findings, which indicate that the CFI value is 0.933, the TLI value is 0.922, the RMSEA value is 0.069, the SRMR value is 0.077, and the GFI value is 0.962. To further improve the model fit using CFA, we analyzed the modification indices and identified covariance between items within the same factor with values greater than 0.30. As recommended by Fornell & Larcker (1981), the most suitable modification for the measurement model was the covariation of error terms within the same factor. Table 7 displays the factor loading fit.

 

Table 7

Factors loading of the items

 

Dimentions

Items

Loading Factors

Environmental consciousness level

I believe that government should support the renewable energy sources (sun, wind, water, geothermal).

0.532

I, as well as others, have responsibility for the protection of the environment.

0.510

I’m in favour of using solar power in traffic lights and street lamps in order to keep the future generations’ life.

0.559

I’m in favour of using energy sources like solar power and natural gas since the gases given out from stoves are more harmful.

0.470

I would use recycling boxes if there were any.

0.564

I would use e-bill in order to protect the environment.

0.602

I would throw away my garbage if there were nobody there.

0.587

There is nothing wrong with pouring waste cooking oil into the sink.

0.372

Environmental anxiety level

I think we will not find a place to have picnic within a few generation.

0.573

I think everybody should sow a tree in his or her life.

0.570

I think seeds should be kept for the future of life.

0.689

I would throw old newspapers; empty glass-plastic bottles, and cans to recycling boxes.

0.752

I think indiscriminate hunting can cause environmental problems.

0.433

I would warn people if they caused harm to the environment.

0.751

Environmental awareness level

When I read a newspaper I pay attention to the topics related to the environment.

0.636

For the protection of environment caused by waste, I watch TV programs that give information about re-use of them.

0.691

I would rather buy environmentally friendly items than economic ones.

0.537

I prefer to use public transportation rather than private transportation to protect the environment.

0.398

 

Additionally, Figure 1 displays a more accurate model fit. Figure 1 presents an illustration of the CFA diagram that follows the modification index and provides details about the GoF values. The questionnaire has achieved outstanding measurement criteria, as determined by Soeharto & Csapó (2021), who defined the cutoff requirements for fit indices in covariance structure analysis.

 

 

5. Discussions

 

This study was conducted with the objective of adapting and validating an environmental literacy instrument for the secondary school student population in the Indonesian context. The results indicate that various statistical procedures were performed, leading to the modification of the model through CFA analysis, and ultimately validating the designed measuring instrument. As a result, the study was conducted using statistical procedures to ensure the validity of the instrument.

In this study, the researchers developed a 20-item questionnaire to measure the environmental literacy level of junior high school students. The questionnaire was subjected to exploratory factor analysis, which revealed three variables for the measure. The KMO value of 0.886 indicated that the instrument was able to accurately distinguish the three latent factors in the questionnaire. However, two of the twenty items were removed from the questionnaire due to their factor loading values being below 0.3. These items were "Environmental education should be provided from the beginning in order to provide environmental awareness" and "I want to learn about environmental issues".

The results from graph 1, which is based on an ethnic/tribal sample in Indonesia, suggest that there is no bias among the participants towards filling in the items of the questionnaire. The sample includes individuals from Javanese (1), Lampung (2), Sundanese (3), Batak (4), Padang (5), Bugis (6), and other (7) ethnicities. This finding indicates that the questionnaire is suitable for use with diverse groups in Indonesia. Furthermore, the level of environmental literacy among the students can be observed from Figure 2, which shows the distribution of their scores. This information is useful in understanding the strengths and weaknesses of the students' environmental literacy and can inform the development of targeted interventions to improve their understanding of environmental issues.

The level of environmental literacy possessed by individuals or organizations can help to determine their perspective on environmental issues. It is widely known that one approach to solving environmental problems is to develop innovative technologies and solutions, typically after problems have already arisen. Another approach is to educate the general public to prevent the manifestation of these issues. Individuals with high environmental literacy are better equipped to utilize the latter approach. Additionally, research has shown that children tend to model their behavior after their parents, so environmentally conscious parents are more likely to have environmentally conscious children. Therefore, it is crucial to assess a person's level of environmental literacy. While there are some studies on environmental issues in the global literature, there are relatively few studies that specifically explore the perspectives of students on this topic (Fujitani et al., 2017; Kaya & Elster, 2018; Nunez & Clores, 2017).

Morrone et al. (2001) have provided research findings that can contribute to environmental literacy research by presenting a tested and validated survey instrument to measure ecological knowledge, which is one of the components of environmental literacy. In a study conducted by Erdogan & Ok (2011) to measure the environmental literacy level of secondary school students in Turkey, the results showed that 61% of students had a moderate level of environmental literacy despite using a literacy questionnaire. Atabek-Yiğit et al. (2014) developed ELSA to determine the environmental literacy of adults.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 1. CFA Model Fit (N = 1021)

 

Figure 2. Differential item functioning graph based on ethnicity /tribe

 

Figure 3. Wrap map on students' ability to answer items

 

Hence, it is crucial for the upcoming generation, particularly at the secondary school level, to have a heightened awareness of the environment. Students with higher scores in environmental literacy can suggest more thoughtful approaches to environmental problems (Yeh et al., 2021) and can creatively contribute (Suherman & Vidákovich, 2022b) to environmental change (Janmaimool & Khajohnmanee, 2019) as well.

 

Although this study provides valuable information on validating the instrument and evaluating the components of environmental literacy, there are several limitations to our findings. Firstly, our research only aimed to test the measurement model and did not investigate the relationships between the latent components further. Secondly, this study is cross-sectional, which has some drawbacks, such as difficulties in analyzing behaviors over different time periods and collecting samples based on population characteristics. Despite strictly following recommended data collection procedures and taking necessary precautions, our investigation may still have bias.

The distribution of students' responses to the questionnaire is presented in Figures 4, 5, and 6. Figure 4 shows the responses of male and female students from public and private schools, broken down by grade level (7, 8, and 9), regarding their level of environmental consciousness. For public schools, male students generally scored higher than females, except for grade 8, where females had a slightly higher mean score. Standard deviations were generally smaller for males than for females across all grades. In the "Female" subgroup, the mean score for grade 7 was 34.54 with a standard deviation of 4.705, while for grade 8, it was 29.90 with a standard deviation of 3.22, and for grade 9, it was 34.22 with a standard deviation of 5.04. For private schools, male students had higher mean scores than females in grades 8 and 9 but lower scores in grade 7. The standard deviations for both males and females were generally higher than those for public schools. Overall, there appear to be some differences in performance between genders and school types. In the "Male" subgroup, the mean score for grade 7 was 33.73 with a standard deviation of 4.89, for grade 8 it was 33.13 with a standard deviation of 5.05, and for grade 9 it was 31.33 with a standard deviation of 5.07.

Figure 5 illustrates the environmental anxiety levels based on gender and grade. The mean scores for female students in public schools range from 23.26 in grade 7 to 23.39 in grade 9. The standard deviation (SD) also varies across grades, from 2.37 in grade 7 to 3.85 in grade 9. Male students in public schools have mean scores ranging from 19.95 in grade 7 to 20.12 in grade 8, followed by a significant drop to 18.02 in grade 9. The SD also varies across grades, from 3.55 in grade 7 to 4.07 in grade 9. For female students in private schools, the mean scores range from 22.59 in grade 7 to 23.59 in grade 8, followed by a decrease to 19.60 in grade 9. The SD varies across grades as well, from 2.68 in grade 7 to 4.80 in grade 9. Male students in private schools have mean scores ranging from 19.79 in grade 7 to 19.33 in grade 8, followed by a slight increase to 18.02 in grade 9. The SD also varies across grades, from 3.31 in grade 7 to 4.07 in grade 9.

In Figure 6, we can observe the scores of students in environmental commitment levels based on gender and grade. For female students in the public group, the mean scores were 18.37, 17.06, and 18.56 for grades 7, 8, and 9, respectively. The mean scores for male students in the public group were 15.98, 16.02, and 14.88 for the same grades. The mean scores for female students in the private group were 17.78, 18.26, and 16.53 for grades 7, 8, and 9, respectively. The mean scores for male students in the private group were 15.92, 16.02, and 15.66. The standard deviation (SD) for each group and gender also provided information on the variability of scores within each group. For female students in the public group who scored a 7, the SD was 1.86, indicating that the scores were tightly clustered around the mean. However, for female students in the public group who scored a 9, the SD was 3.38, indicating more variability in scores for this group. Overall, the SDs for the private group were slightly higher than those for the public group, suggesting that scores in the private group were more variable.

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 4. Pirate plot for comparing type school student in environmental consciousness level based on gender and grade

 

 

 

 

 

 

 

 

 

 

 

 

Figure 5. Pirate plot for comparing type school student in environmental anxiety level based on gender and grade

 

 

 

 

 

 

 

 

 

 

 

Figure 6. Pirate plot for comparing type school student in environmental commitment level based on gender and grade

 

According to the data, it appears that males in both public and private institutions tend to have slightly higher scores than females across all three grade levels. Additionally, in both public and private institutions, mean scores tend to increase as grade level increases. However, caution should be exercised in interpreting these findings, as other factors such as quality of teaching, class size, and socioeconomic status may also influence academic performance (Guo et al., 2015). Research on gender differences in academic performance has yielded mixed results. While some studies have found that males outperform females in certain subjects such as mathematics and science (Ahmad & Greenhalgh-Spencer, 2017; Steegh et al., 2019), others have found no significant differences between genders. Moreover, research suggests that teaching quality and classroom environment can influence academic performance. Previous research has shown that students attending private institutions tend to have higher academic achievement than those attending public institutions (Kim & Conrad, 2006). However, this may be due to factors such as the socioeconomic status of the students and the resources available to the institution. This study contributes to the development of assessment practices that promote a positive assessment climate, which has been associated with better academic achievement and student engagement (Vergara Morales et al., 2022).

 

 

6. Conclusion

 

Based on statistical analysis, the environmental literacy instruments established in this study have been found to be valid and reliable. The EFA analysis showed that the instrument can effectively distinguish the three latent variables in the questionnaire, with a Chi-square of 779.719, DF of 132, and p < 0.001. The KMO measure of 0.886 further supports the instrument's validity. Two of the twenty items were excluded due to their low loadings of below 0.3. The Cronbach Alpha reliability and McDonald's Omega reliability for the three parameters range between 0.66 and 0.78, and 0.68 and 0.79, respectively, indicating satisfactory reliability within this range. The CR values for the three latent components in the questionnaire are between 0.69 and 0.80, indicating satisfactory convergent validity. However, the discriminant validity is less accepted with an HTMT value of 0.85, which falls within the range of 0.66 to 0.84. The AVE value ranges between 0.35 and 0.39, which is acceptable. The CFA shows that the instrument can effectively assess the environmental literacy of pupils in Indonesia, with CFI = 0.933, TLI = 0.922, RMSEA = 0.069, SRMR = 0.077, and GFI = 0.962.

 

Acknowledgement

We would like to thank Universitas Islam Negeri Raden Intan Lampung, Indonesia, for its support of our research through contract number 177, the year 2022. The Doctoral School of Education at the University of Szeged, Hungary, gave suggestions and debates for a concept from a research perspective.

 

 

References

 

Ab Hamid, M. R., Sami, W., & Sidek, M. M. (2017). Discriminant validity assessment: Use of Fornell & Larcker criterion versus HTMT criterion. Journal of Physics: Conference Series, 890(1), 012163. https://doi.org/10.1088/1742-6596/890/1/012163

Ahmad, F., & Greenhalgh-Spencer, H. (2017). Trends in international mathematics and science study and gendered math teaching in Kuwait. Policy Futures in Education, 15(3), 327–340. https://doi.org/10.1177/1478210317707845

Alp, E., Ertepinar, H., Tekkaya, C., & Yilmaz, A. (2006). A statistical analysis of children’s environmental knowledge and attitudes in Turkey. International Research in Geographical and Environmental Education, 15(3), 210–223.

Arifin, S., Zulkardi, Z., Putri, R. I. I., & Hartono, Y. (2021). On creativity through mathematization in solving non-routine problems. Journal on Mathematics Education, 12(2), 313–330. https://doi.org/10.22342/jme.12.2.13885.313-330

Arsyad, S., & Villia, A. S. (2022). Exploring the Effect of Digital Literacy Skill and Learning Style of Students on Their Meta-Cognitive Strategies in Listening. International Journal of Instruction, 15(1), 527–546. https://doi.org/10.29333/iji.2022.15130a

Atabek-Yiğit, E., Köklükaya, N., Yavuz, M., & Demirhan, E. (2014). Development and validation of environmental literacy scale for adults (ELSA). Journal of Baltic Science Education, 13(3), 425.

Berberoglu, G., & Tosunoglu, C. (1995). Exploratory and confirmatory factor analyses of an environmental attitude scale (EAS) for Turkish university students. The Journal of Environmental Education, 26(3), 40–43.

Chuah, S. H.-W., Rauschnabel, P. A., Krey, N., Nguyen, B., Ramayah, T., & Lade, S. (2016). Wearable technologies: The role of usefulness and visibility in smartwatch adoption. Computers in Human Behavior, 65, 276–284. https://doi.org/10.1016/j.chb.2016.07.047

Corebima, A. D., Susilo, H., & Zubaidah, S. (2017). Creative Thinking of Low Academic Student Undergoing Search Solve Create and Share Learning Integrated with Metacognitive Strategy. International Journal of Instruction, 10(2). https://doi.org/10.12973/iji.2017.10216a

Cristovão, V. L. L., Sanches, B., & Smart, G. (2022). Environmental discourse in Brazilian English-as-a-foreign-language textbooks: Socio-discursive practices and their implications for developing students’ critical environmental literacy. Environmental Education Research, 28(1), 75–94. https://doi.org/10.1080/13504622.2021.2007855

De Leyn, T., Waeterloos, C., De Wolf, R., Vanhaelewyn, B., Ponnet, K., & De Marez, L. (2022). Teenagers’ reflections on media literacy initiatives at school and everyday media literacy discourses. Journal of Children and Media, 16(2), 221–239. https://doi.org/10.1080/17482798.2021.1952463

Dunlap, R., Liere, K. V., Mertig, A., & Jones, R. E. (2000). Measuring endorsement of the new ecological paradigm: A revised NEP scale. Journal of Social Issues, 56(3), 425–442.

Erdogan, M., & Ok, A. (2011). An assessment of Turkish young pupils’ environmental literacy: A nationwide survey. International Journal of Science Education, 33(17), 2375–2406. https://doi.org/10.1080/09500693.2010.550653

Erdoğan, Ş. (2003). Öğretmen adaylarının çevre sorunlarına yönelik tutumları. Gazi Üniversitesi Gazi Eğitim Fakültesi Dergisi, 23(2).

Field, A. (2013). Discovering statistics using IBM SPSS statistics. sage.

Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104

Fujitani, M., McFall, A., Randler, C., & Arlinghaus, R. (2017). Participatory adaptive management leads to environmental learning outcomes extending beyond the sphere of science. Science Advances, 3(6), e1602516. https://doi.org/10.1126/sciadv.1602516

Gao, J., Zhao, J., Wang, J., & Wang, J. (2021). The influence mechanism of environmental anxiety on pro‐environmental behaviour: The role of self‐discrepancy. International Journal of Consumer Studies, 45(1), 54–64.

Gaskin, J., & Lim, J. (2016). Model fit measures. Gaskination’s StatWiki, 1–55.

Glynn, S. M., Brickman, P., Armstrong, N., & Taasoobshirazi, G. (2011). Science motivation questionnaire II: Validation with science majors and nonscience majors. Journal of Research in Science Teaching, 48(10), 1159–1176. https://doi.org/10.1002/tea.20442

Grilli, G., & Curtis, J. (2021). Encouraging pro-environmental behaviours: A review of methods and approaches. Renewable and Sustainable Energy Reviews, 135, 110039. https://doi.org/10.1016/j.rser.2020.110039

Guo, J., Marsh, H. W., Parker, P. D., Morin, A. J., & Yeung, A. S. (2015). Expectancy-value in mathematics, gender and socioeconomic background as predictors of achievement and aspirations: A multi-cohort study. Learning and Individual Differences, 37, 161–168.

Habók, A., & Magyar, A. (2018). Validation of a self-regulated foreign language learning strategy questionnaire through multidimensional modelling. Frontiers in Psychology, 9, 1388. https://doi.org/10.3389/fpsyg.2018.01388

Hair, J. F., Anderson, R. E., Babin, B. J., & Black, W. C. (2010). Multivariate data analysis: A global perspective: Pearson Upper Saddle River. NJ.

Hair Jr, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2021). A primer on partial least squares structural equation modeling (PLS-SEM). Sage publications.

Harring, N., & Jagers, S. C. (2018). Why do people accept environmental policies? The prospects of higher education and changes in norms, beliefs and policy preferences. Environmental Education Research, 24(6), 791–806. https://doi.org/10.1080/13504622.2017.1343281

Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8

Janmaimool, P., & Khajohnmanee, S. (2019). Roles of environmental system knowledge in promoting university students’ environmental attitudes and pro-environmental behaviors. Sustainability, 11(16), 4270. https://doi.org/10.3390/su11164270

Jomnonkwao, S., & Ratanavaraha, V. (2016). Measurement modelling of the perceived service quality of a sightseeing bus service: An application of hierarchical confirmatory factor analysis. Transport Policy, 45, 240–252. https://doi.org/10.1016/j.tranpol.2015.04.001

Kaya, V. H., & Elster, D. (2018). German students’ environmental literacy in science education based on PISA data. Science Education International, 29(2). https://doi.org/10.33828/sei.v29.i2.2

Kelani, R. R. (2017). Teachers Candidates’ Knowledge, Attitudes and Behaviors Within the Context of Environmental Education. International Journal of Progressive Sciences and Technologies, 5(2), 76–87. https://doi.org/10.52155/ijpsat.v5.2.134

Kim, M. M., & Conrad, C. F. (2006). The impact of historically Black colleges and universities on the academic success of African-American students. Research in Higher Education, 47, 399–427. https://doi.org/10.1007/s11162-005-9001-4

Kline, R. B. (2015). Principles and practice of structural equation modeling, 4th Edn. Guilford publications.

Leeming, F. C., Dwyer, W. O., & Bracken, B. A. (1995). Children’s environmental attitude and knowledge scale: Construction and validation. The Journal of Environmental Education, 26(3), 22–31. https://doi.org/10.1080/00958964.1995.9941442

Lloyd‐Strovas, J., Moseley, C., & Arsuffi, T. (2018). Environmental literacy of undergraduate college students: Development of the environmental literacy instrument (ELI). School Science and Mathematics, 118(3–4), 84–92. https://doi.org/10.1111/ssm.12266

Malhotra, N. K., & Dash, S. (2011). Marketing Research an Applied Orientation. London: Pearson Publishing.

Morrone, M., Mancl, K., & Carr, K. (2001). Development of a metric to test group differences in ecological knowledge as one component of environmental literacy. The Journal of Environmental Education, 32(4), 33–42. https://doi.org/10.1080/00958960109598661

Naim, U., & Sağlam, N. (2006). Orta öğretim öğrencileri için çevresel tutum ölçeği geliştirme ve geçerliliği. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 30(30), 240–250.

Negev, M., Sagy, G., Garb, Y., Salzberg, A., & Tal, A. (2008). Evaluating the environmental literacy of Israeli elementary and high school students. The Journal of Environmental Education, 39(2), 3–20. https://doi.org/10.3200/JOEE.39.2.3-20

Nunez, M. B., & Clores, M. A. (2017). Environmental Literacy of K-10 Student Completers. International Journal of Environmental and Science Education, 12(5), 1195–1215.

Nurhidayati, S., Susantini, E., Rachmadiarti, F., & Sukri, A. (2022). Development of Local Potential-Based Environmental Care Instrument and its Ability to Reveal Students’ Caring Attitudes at Genders and Grades. International Journal of Instruction, 15(4). https://doi.org/10.29333/iji.2022.15433a

O’Byrne, I., Radakovic, N., Hunter-Doniger, T., Fox, M., Kern, R., & Parnell, S. (2018). Designing spaces for creativity and divergent thinking: Pre-service teachers creating stop motion animation on tablets. International Journal of Education in Mathematics, Science and Technology, 6(2), 182–199. https://doi.org/10.18404/ijemst.408942

Parker, L., Prabawa-Sear, K., & Kustiningsih, W. (2018). How young people in Indonesia see themselves as environmentalists: Identity, behaviour, perceptions and responsibility. Indonesia and the Malay World, 46(136), 263–282. https://doi.org/10.1080/13639811.2018.1496630

Raykov, T. (1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21(2), 173–184. https://doi.org/10.1177/01466216970212006

Rytkönen, K., Aunola, K., & Nurmi, J.-E. (2007). Do parents’ causal attributions predict the accuracy and bias in their children’s self-concept of maths ability? A longitudinal study. Educational Psychology, 27(6), 771–788. http://files/379/01443410701309316.html

Safitri, W. I., Suryawati, E., & Yustina, Y. (2020). Environmental literacy analysis of junior high school students in Pekanbaru. Journal of Educational Sciences, 4(1), 116–123. https://doi.org/10.31258/jes.4.1.p.116-123

Saribas, D., Kucuk, Z. D., & Ertepinar, H. (2017). Implementation of an environmental education course to improve pre-service elementary teachers’ environmental literacy and self-efficacy beliefs. International Research in Geographical and Environmental Education, 26(4), 311–326. https://doi.org/10.1080/10382046.2016.1262512

Soeharto, S., & Csapó, B. (2021). Building a house from Lego blocks: Using cross cultural validation to develop the Constructed Motivation Questionnaire (CMQS) in Science. Pedagogika, 2021, t. 142, Nr. 2, p. 218–241. https://doi.org/10.15823/p.2021.142.12

Steegh, A. M., Höffler, T. N., Keller, M. M., & Parchmann, I. (2019). Gender differences in mathematics and science competitions: A systematic review. Journal of Research in Science Teaching, 56(10), 1431–1460. https://doi.org/10.1002/tea.21580

Sudarmadi, S., Suzuki, S., Kawada, T., Netti, H., Soemantri, S., & Tri Tugaswati, A. (2001). A survey of perception, knowledge, awareness, and attitude in regard to environmental problems in a sample of two different social groups in Jakarta, Indonesia. Environment, Development and Sustainability, 3, 169–183. https://doi.org/10.1023/A:1011633729185

Suherman, S., Prananda, M. R., Proboningrum, D. I., Pratama, E. R., Laksono, P., & Amiruddin, A. (2020). Improving higher order thinking skills (hots) with project based learning (pjbl) model assisted by geogebra. Journal of Physics: Conference Series, 1467(1), 012027. https://doi.org/10.1088/1742-6596/1467/1/012027

Suherman, S., & Vidákovich, T. (2022a). Adaptation and Validation of Students’ Attitudes Toward Mathematics to Indonesia. Pedagogika, 147(3), 227–252. https://doi.org/10.15823/p.2022.147.11

Suherman, S., & Vidákovich, T. (2022b). Assessment of Mathematical Creative Thinking: A Systematic Review. Thinking Skills and Creativity, 101019. https://doi.org/10.1016/j.tsc.2022.101019

Swangjang, K. (2018). Comparative review of EIA in the Association of Southeast Asian Nations. Environmental Impact Assessment Review, 72, 33–42. https://doi.org/10.1016/j.eiar.2018.04.011

Szczytko, R., Stevenson, K., Peterson, M. N., Nietfeld, J., & Strnad, R. L. (2019). Development and validation of the environmental literacy instrument for adolescents. Environmental Education Research, 25(2), 193–210. https://doi.org/10.1080/13504622.2018.1487035

Tabachnick, B. G., Fidell, L. S., & Ullman, J. B. (2007). Using multivariate statistics (Vol. 5). Pearson Boston, MA.

Teksöz, G., Şahin, E., & Ertepinar, H. (2010). Çevre okuryazarlığı, öğretmen adayları ve sürdürülebilir bir gelecek. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 39(39), 307–320.

Tuncer Teksoz, G., Boone, J. W., Tuzun, O. Y., & Oztekin, C. (2014). An evaluation of the environmental literacy of preservice teachers in Turkey through Rasch analysis. Environmental Education Research, 20(2), 202–227. https://doi.org/10.1080/13504622.2013.768604

UNESCO. (2016). Global Education Monitoring Report 2016: Education for People & Planet: Creating Sustainable Futures for All. UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000245656

UNESCO, T. D. (1978). Intergovernmental conference on environmental education. Final Report. Paris.

Vergara Morales, J. R., Díaz Larenas, C. H., Tagle Ochoa, T., & Ortiz Navarrete, M. (2022). Adaptación y Validación de la Escala de Percepción del Clima de Evaluación en el Aula (EPCEA). Revista Fuentes. https://doi.org/10.12795/revistafuentes.2022.19232

Wieland, A., Durach, C. F., Kembro, J., & Treiblmaier, H. (2017). Statistical and judgmental criteria for scale purification. Supply Chain Management: An International Journal, 22(4), 321–328. https://doi.org/10.1108/SCM-07-2016-0230

Yeh, F.-Y., Tran, N.-H., Hung, S. H., & Huang, C.-F. (2021). A study of environmental literacy, scientific performance, and environmental problem-solving. International Journal of Science and Mathematics Education, 1–23. https://doi.org/10.1007/s10763-021-10223-9