Introduction

The number of countries now involved in the project (Appendix 1) has continued to increase, with possible future participation from Russia and the Ukraine.

The methodology of this research is based on regular testing of pupils from age 13+ for a period of 2-3 years, monitoring the progress made by groups of pupils of similar ability, and then investigating further those schools, classes (i.e. teachers) and pupils which show progress much above or below the norm.

At the start of the project, all pupils take a mathematics Potential Test. This is a 40 minute test with 26 questions of increasing difficulty. It is taken only once and aims to assess the pupils' aptitude for mathematics with questions based on mathematical logic and spatial awareness. The test is designed for pupils aged 12-14 years and was trialled extensively before being used in this project.

The pupils then take tests in

• Number
• Algebra
• Shape and Space
• Handling Data (optional)

These 40 minute tests, each with 50 marks available, are designed to show progress in attainment and are based on the intersection of the English, Scottish and German mathematics curricula for pupils aged 13-16 years. In National Curriculum terms, they cover the basic topics from Levels 4 to 9 in the relevant attainment target. Most questions are relatively short and precise with few follow-through marks and, unlike Key Stage 3 assessment, relatively content-free. Similar tests, with questions either repeated or replaced with equivalent questions, are taken at the beginning of each of the following years of the project.

Testing began in 1993 with pupils aged 13+ in England, Scotland and Germany and these English and Scottish pupils are now taking exams (GCSE in England and SEB Standard Grade in Scotland) this summer. Other countries joined one year later, either with pupils aged 13+ (equivalent to Y1 of the project, enabling comparison to be made retrospectively) or 14+ (Y2 of the project).

As well as test data (each response entered on computer) we collect further information using School, Teacher, Class and Pupil Questionnaires.

To complete the picture we visit schools to:-

(i)	observe mathematics teaching, recording the important characteristics of the lesson.
(ii)	interview teachers and pupils individually (usually concentrating on pupils who have made either very good or very poor progress over the year).

We are guided on where these in-depth visits should be made by Performance Indicators, which are calculated for each pupil on progress made over Y1-Y2 and Y2-Y3. This progress is compared with the average progress made by groups of pupils with similar potential and similar topic test scores at the start of the year. The formula used is

Here T1 and T2 are the total scores on the three tests (Number, Algebra and Shape and Space) in Year 1 and Year 2, and d is the average increase for the group of pupils with similar potential and first year attainment.

Results

In England, our representative sample consists of 12 schools taken from a total of 40 project schools. It is chosen to be representative according to geographical location and type of school (independent, grant maintained, l.e.a. comprehensive). As a final check, a weighted average according to cohort size of the 1995 Mathematics GCSE results of all the schools is taken and compared with national 1995 GCSE Mathematics figures using a chi-squared test. It should also be noted that the results are relatively insensitive to minor changes in the schools in the sample, and we have confidence that our sample is a good representation of the whole country.

Some overall results for representative samples of pupils for Year 1 (age 13+) and Year 2 (age 14+) are summarised below.

Not only were England and Scotland well behind some of the other European countries in total attainment for these core topics on the first testing but the progress made during the year was less than in other countries. It should also be noted that Singapore is doing so well that it begins to become more difficult for many of the pupils to show any real progress - so their increase of 16.8 over the year is an excellent result.

The results do indicate underperformance by English and Scottish pupils, particularly in comparison with Poland and Singapore. We can gain further insight into this possible underperformance by looking at particular ability groups. For England, Germany, and Singapore, we can divide our representative sample into three equal parts, low (L), middle (M) and high (H), and monitor the progress of each group.

Our observations of teaching in these countries (and others such as Hungary which also showed very high attainment in Y1) have led us to make draft recommendations for mathematics teaching and learning in England in order to raise expectations and Appendix 2 gives details of these. We are not saying that we have 'proved' that these recommendations will work - only that our data indicates under-achievement and our observations led us to suggest that these factors might enhance performance.
These recommendations are being implemented through the Mathematics Enhancement Programme(MEP) demonstration project.

Further Analysis

We are in the position to publish confirmed Year 3 results for England. These are given below