How data were collected » Data collection instruments:
Contextual data collection instruments
Contextual data were collected through three instruments: a student questionnaire, a teacher questionnaire, and a school leader questionnaire (head teacher / principal).
Students completed a questionnaire attached to the numeracy assessment and their teachers and school leaders were supplied with questionnaires separately.
While collecting contextual information is recognised as best practice for international large-scale assessments, contextual data collection is relatively new to PILNA. The 2018 PILNA cycle was the first cycle to implement contextual instruments.
Analysing the contextual data alongside the cognitive data provided information underpinning student performance. Furthermore, analysing the associations between the contextual data and student performance allowed for more evidenced and targeted improvement strategies to be created. And as the contextual information extended beyond the classroom environment, interventions and improvement strategies could also be crafted beyond the classroom environment.
Areas of investigation
The contextual questionnaires collect information from common areas of investigation as used in international large-scale assessments and tailored to the Pacific region. The nine areas are described below.
- Early learning opportunities. This includes attendance at an early childhood education (ECE) programme, grade repetition, early reading, and age of starting school.
- Language at home and school. This includes the language of instruction, test language, mother tongue, language spoken at home and in social settings, and languages for writing and reading.
- Family and community support. This includes communication with parents, cultural capital and social capital, parental and family involvement in schooling, home support for study, and descriptive variables regarding types of communities.
- Quality of instruction. This includes variables at the student, teacher and school level, such as pedagogical practices, obstacles to learning (lack of nutrition or sleep), academic expectations, assessing and monitoring learning progress, classroom organisation and management, homework, teacher practice, and professional development.
- Learning time. This includes factors such as enrolment, attendance, instruction time, and work habits outside school.
- Student home environment. This includes parental education, main source of income and occupation, home facilities and possessions, and educational resources.
- School resources. This includes school facilities, classroom facilities, teaching resources, WASH factors, funding sources, school location, travel distance (for students and teachers), school safety and violence, and staff stability and satisfaction.
- Student well-being. This includes students’ mental well-being, social support and levels of disadvantage, school safety, and bullying.
- Teacher satisfaction. This includes general measures of teacher satisfaction, such as mental well-being, work-life balance, workplace experiences, and fulfillment.
International instrument adaptations
Some questions that were used in the PILNA 2021 instruments were taken from existing large-scale international instruments. The areas of the PILNA instruments that use these questions and the instruments they are adapted from are shown below in Table X.
| Area of PILNA instrument | Adapted from |
| UNICEF & SEAMEO. (2019). SEA-PLM 2019: School Questionnaire |
| TIMSS 2015 Teacher Questionnaire Grade 4. Copyright © 2014 International Association for the Evaluation of Educational Achievement (IEA). Publisher: TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College. |
| OECD (2018), “Teaching and Learning International Survey (TALIS): Teacher Questionnaire” |
The PILNA cognitive and contextual data are used by all stakeholders to understand trends in the region but comparisons between countries are not part of the PILNA programme. This has been explicitly directed by the PILNA Steering Committee.
Instruction documents for supervisors of students completing PILNA instruments were supplied. These were specific to each year level and each domain (literacy, numeracy and contextual information).
