Comparison of PISA, TIMSS and NAPLAN

Introduction

This report describes three different educational assessments that are provided to students within the Australian Education System. Each assessment has differing assessment methods and a differing target audience. Whilst two are conducted internationally (PISA and TIMSS), NAPLAN is conducted only within Australia.

Review of National and International assessments.

Program for International Student Assessment – PISA

The Program for International Student Assessment (PISA) is a survey that is held every three years in excess of 65 countries to test the skills and knowledge of approximately half a million 15 year old students. This sample of students represents approximately 1.8% of 15 year old students worldwide (OECD, n.d). PISA’s reporting allows student outcomes to be classified by sex, indigenous status, geographical location and socio-economic background indicators (ACER, 2014a).
PISA differs from other student assessments in that it tests the general knowledge of students which may not be contained within the student curriculum. The goal of PISA is to gauge the extent to which students can use their general knowledge in real life situations and therefore provide an indication of how well equipped the individual is for society in general (OECD, n.d).
The selection of students who are assessed is random by school and the tests contain both multiple choice and open-ended questions that have been derived to depict real-life situations. The tests alternate every three years between a focus on reading, mathematics or science (OECD, n.d).
In 2012 the scope of PISA assessment was broadened to include an assessment on financial literacy. This assessment tested the knowledge and skills of these 15 year olds in dealing with basic financial issues (OECD, n.d).
The results of the most recent assessment are available for download from the OECD website in a PDF format (OECD, n.d).

Trends in International Mathematics and Science Study – TIMSS

TIMSS is a large scale assessment designed to provide an international comparison on the teaching and learning in science and mathematics, and is a project of the International Association for the Evaluation of Educational Achievement (IEA). TIMSS is directed by the TIMSS International Study Centre at Boston College and the Australian Council for Educational Research (ACER) is responsible for collaborating of the data collection in Australian schools (ACER, 2014b).
Since 1995, TIMSS has been conducted every four years and in the next assessment period (2015) more than 60 countries will participate, including Australia. Within Australia, 580 schools will be involved and this will capture in excess of 14,000 Australian students alone. TIMSS is targeted to students in years four and eight and outputs extensive data pertaining to the regional, school and classroom learning environments. Australia has been a participant in all five iterations of TIMSS since its inception (ACER, 2014b).
Approximately half of the TIMSS assessment criteria are released to the public after each round of testing. These are used for educators to inform discussion about the mathematics and science curriculum to develop links between taught concepts and benchmarks of students understanding. They are also used to reflect on the performance of all students in all of the participating countries (Institute of Education Sciences, n.d).
TIMSS results are analysed and used to monitor achievement trends globally and to establish goals and standards for educational improvement thus providing stimulation for curriculum reform with the aim of improved teaching and learning (IEA, 2014). TIMSS results are provided via the TIMSS and PIRLS International Study Centre website in the form of a PDF file or can be purchased in a hard copy format. Previous questions are made available for students to test their knowledge of mathematics and science at their leisure. These are available for download in PDF format from National Centre for Education Statistics website (Institute of Education Sciences, n.d).

National Assessment Program Literacy and Numeracy – NAPLAN

Since its introduction in 2008, NAPLAN has been conducted on all Australian students in years 3, 5, 7 and 9. NAPLAN tests four areas consisting of reading, writing, language conventions and numeracy. NAPLAN measures a student’s performance against an assessment scales in each of the above areas (ACARA, 2011a).
Each scale consists of six bands which indicate whether the performance meets, exceeds or is below the national standard for that age (ACARA, 2011b). Students who fall below that national minimum standard have not achieved the learning outcomes for their year level and are identified as requiring targeted intervention in order to progress satisfactorily. Likewise students who are performing at the minimum standard are identified as possibly requiring additional assistance to enable them to succeed. The standards are only intended to be an indication of typical achievement and do not test the full extent of the student’s knowledge or potential achievement (ACARA, 2011c). NAPLAN is divided into ten bands and not all bands are reported against each age group. Each age group are assessed against six bands which increment as the age of the student increases. Year 3 students are tested against bands 1-6, Year 5, 3-8 Year 7, 4-9 and Year 9, 5-10 (ACARA, 2011b).
NAPLAN is intended to provide diagnostics pertaining to individual progress against national standards. The report shows the students result and compares it to the national average, the school average and provides the range of achievement for the middle 60% of students in that year and in that subject (Thomson, 2010).
Individual results are provided to the student through their school in the format of a formal report. School averages are available via the ‘My School’ website and national statistics are provided through ACARA (ACARA, 2013).

Comparison of three different educational assessments.

Comparing NAPLAN, PISA and TIMSS is problematic since they involve different models. NAPLAN and TIMSS are curriculum based, NAPLAN being nationally focused and TIMSS being international focused. PISA is a yield study which gauges the extent to which students can use their knowledge in real life situations (Thomson, 2010).
Both TIMSS and PISA collect background data on students including the resources they have available, their family’s education levels and their attitude to schooling and themselves as students. This contextual information is not available through NAPLAN testing. NAPLAN is compulsory for all Australian students in the given grades whilst PISA is a random selection of schools throughout the world in participating countries. TIMSS participation is through an ‘opt-in’ registration process (ACER, 2014a).

Conclusion

Each of the three assessments targets different age groups, and each has a different focus. NAPLAN (numeracy) assesses an individual’s numeracy proficiency against national standards, along with the overall performance of the school and the national average. TIMSS also provides data on numerical proficiency but returns international comparisons. TIMSS also considers environmental and social contexts in the assessment, to include the student’s sex, indigenous status, geographical location and socio-economic background. PISA focuses on older students, and is focussed on assessing the ability of the student to apply their numeracy skills in real life situations, thus forming an opinion of how well equipped the individual is for living in society.
Individually, the assessments provide valuable insight into the effectiveness of a student’s numeracy ability. Collectively, they provide a far more powerful window into an individual’s position within their peers, their nation, the entire world, and lastly, provides a guide as to how easily they student will be able to use numeracy to their advantage in future endeavours.

References

ACARA, (2011a). National Assessment Program. Retrieved from http://www.nap.edu.au/naplan/naplan.html
ACARA. (2011b). National Assessment Program. Retrieved from http://www.nap.edu.au/results-and-reports/how-to-interpret/how-to-interpret.html
ACARA, (2011c). National Assessment Program. Retrieved from http://www.nap.edu.au/results-and-reports/how-to-interpret/standards.html
ACER, (2014a). Programme for International Student Assessment. Retrieved from http://www.oecd.org/pisa/keyfindings/pisa-2012-results.htm
ACER, (2014b). Trends in International Mathematics and Science Study. Retrieved from http://www.acer.edu.au/timss
Institute of education sciences (IES), (n.d). National Centre for Education Statistics. Retrieved from http://nces.ed.gov/TIMSS/index.asp
International Association for the Evaluation of Educational Achievement (IEA), (2014). TIMSS and PIRLS International Study Center (2014). Retrieved from http://timssandpirls.bc.edu/home/pdf/TP_About.pdf
OECD, (n.d). Programme for International Student Assessment. Retrieved from http://www.oecd.org/pisa/
Thomson, S. (2010). Mathematics learning: What TIMSS and PISA can tell us about what counts for all Australian students.