comment.htm

Home

Top

SUMMATIVE ASSESSMENT IN PRACTICE

Some Thoughts on the South African Secondary School Examination 2005

Fred B Shaw

It is that time of the year again when Senior Certificate Examination results have been announced; the time when all kinds of trivial and important analyses regarding the examination are made; the time for celebration of success and that of bitter failure. Minister Naledi Pandor pronounced the results as credible and the standard satisfactory, although she would like to have seen a 70% pass.

Not only is the Senior Secondary Examination an important exercise in summative testing, it is also a high stakes test, because of the numbers of candidates involved and the timing of the examination.

The Star, in its overview, used the heading “Matric a true labour of love”[i]. For the majority of candidates this is true. It is the culmination of 12 years of schooling, not just the work of the final year, although the final preparation nevertheless plays an important part in determining achievement or lack of it. Some people are sceptical of what the examination tests and believe that within 12 months much of what was learnt is forgotten. However, what is forgotten is detail, while many learnt principles have been and are integrated into the memory, and used and applied (unconsciously) in the next phase of education. Unfortunately, all learning is contextual and many persons may have to be reminded of the original learning context. Success in Tertiary education depends upon successful integration of the curriculum of secondary education. Students who have not mastered much of what should have been learnt at school battle to keep up with the pace of tertiary education.

During the 1970s the buzz word was “predictability”. Much research was targeted at determining “predictors for success” in tertiary education. According to research carried out it was found that there was a strong relationship between success at varsity, and the mark obtained in mother tongue, the mark attained for HG mathematics, as well as the grand total of the marks obtained in all subjects. Furthermore, it was deduced that if a candidate did not obtain 50% in the grand total, his/her chance of passing a degree in five years or less was low[ii]. Since education is a social science there are always outliers in predictability studies, which means that high marks do not guarantee success, nor do poor marks guarantee failure, but certain trends exist. Since 1980,when this research was carried out, school populations have changed considerably and new research is required to confirm the probability of success in tertiary education now.

It is not difficult from the above study to conclude that one of the main purposes of the final school examination is to provide individuals an opportunity to demonstrate their ability and willingness to learn at that stage of their lives.

Success, therefore, is not portrayed in a dichotomous result but is best expressed on a graduated scale of relative achievement. For example, a “7” on a 10-point scale is better than a “6”, but it is quite possible that two adjacent scores may be interchangeable under different circumstances. On the other hand a “7” is better than a “5”, and the achievement of two candidates having these scores respectively is likely to be more stable in two similar assessments. Thus, candidates should be encouraged to demonstrate their highest possible achievement during their final school year. The mistake made when interpreting examination results is to treat them in absolute rather than in relative terms. The result for any individual is simply a culmination of applied work ethic under given circumstances. The score of any candidate can be compared with previous scores of the same candidate or with the scores of other candidates.

What is the significance of a pass rate of 68.3%? The division of candidates into two tends to mask the meaning of a pass. At the pass-fail cut-off point some candidates just managed to pass, while others just did not make the grade. In reality there is little difference between any of these two types of candidates. The fact that some just passed, while others just failed, is probably more a factor of luck than good design. However, when scores are separated by two or more points, the difference is more significant. There is a vast difference between those who attained an average of more than 60 % and those who just passed (40%). Thus, while the result of the examination may be dichotomized into a pass and a fail category, there is a degree of a pass and a degree of failure.

It is of interest that the different provinces have demonstrated fairly different pass-rates. An important question is why? Different resources, work ethic, rural and urban population characteristics, distribution of schools, educator training, experience and methodology, different choices of subjects – all of these contribute to the different pass rates. Why, for example, is the Northern Cape doing so well compared to other provinces? Here is a topic worthy of research. If we can unlock the cause of these differences, we might well be on the way to providing a better education for all.

Umalusi attempts to moderate the standard of the examination in all provinces towards a similar level, while there are six national subjects in which the same examination is written in all provinces, viz. the papers for Accounting, Biology, Business Economics, History, Mathematics and Physical Science. An analysis of the average achievement and distribution of marks for these subjects may give some clues that inform us about the potential of candidates in each province. An analysis of these results may provide some clue to the standard of education in each of the provinces. There is strength in diversity, but we need to know the strengths and weaknesses of each provincial system in order to improve achievement generally.

Table of different pass rates 2005

Province	Western Cape	Northern Cape	Free State	Gauteng	KwaZulu Natal	Limpopo	North West	Mpuma-langa	Eastern Cape
Pass rate[iii]	84.4%	78.9%	77.8%	74.9%	70.5%	64.9%	63.0%	58.6%	56.7%

Although much is made of the different pass rates, pass rates on their own are meaningless without knowledge of how the marks are distributed and the spread of subjects in which candidates wrote the examination of each province. Certainly, a pass rate indicates a candidate’s chances of passing, e.g. if one wrote the examination in the Western Cape one has a better chance of passing than if one wrote the examination in the Eastern Cape. It does not necessarily mean that the Western Cape examination was easier than the Eastern Cape examination. Control exercised by Umalusi tends to equalize standards and it is assumed that the examinations of the different provinces are of equal standards. Furthermore, Umalusi uses statistical techniques that tend to iron out fluctuations in the standard of examination per subject from year to year.

Thus, unless a concerted effort is directed at the improvement of teaching and learning in Grade 12, one must not expect a significant change in the pass rate in any province. From the reports in newspapers[iv], it appears that an individual’s chances of passing is dependent upon the school(s) attended, the teachers of the school as well the ability and diligence of the candidate. It must be remembered that a pass in the Senior Certificate examination is not only dependent upon the instruction received in the final year, but also upon instruction received across the entire secondary school phase.

Only 17% of candidates who wrote the provincial examination, in contrast to 78% who wrote the IEB examination, obtained Matriculation Exemption, i.e. the minimum entrance to a tertiary institution. This means that there are a limited number of candidates eligible for enrolment at a tertiary institution.

Also according to the statistics more girls wrote the examination, while a greater percentage of girls failed the examination, although they achieved better than boys. These are the kinds of contradictions in the statistics that can be expected in large-scale summative assessment, and valid deductions are difficult to make. Again, research is required to establish reasons for these observations.

Evaluation according to Beeby is “The systematic collection and interpretation of evidence leading as part of the process, to a judgment of value, with a view to action”[v]. In principle evaluation and assessment both refer to testing and both are done purposely. Certainly, the Senior Certificate Examination is an assessment that is designed and controlled to provide appropriate evidence of achievement across a number of fields. As with all assessment, summative assessment culminates in a value judgment, which may be used for decision-making. Summative assessment results in different judgments, which will in turn be related to individuals and context. Parents may draw one set of conclusion, schools, teachers, future employers, tertiary institutions and candidates others. For example, a prospective employer who is looking for a person to carry out routine tasks will probably not consider candidates at the top of the range of achievement, because they will not fit the job. On the other hand a tertiary institution would probably prefer a candidate with a higher achievement profile, since the evidence shows that the person has strong academic ability. In like manner, certificates assume certain values, but with the passage of time, the value diminishes because the evidence collected from earlier is updated by later assessments. Even a person who has failed an assessment can use the evidence to confirm strengths and weaknesses and make informed decisions on future study directions.

Furthermore, the assessment may be used to evaluate the general quality of schools over the last five years (the length of the secondary curriculum). The examination, although focused on the last year, nevertheless, provides evidence of the then overall quality and effectiveness of education in the secondary school. It is not insignificant that the national pass rate dropped in all but one province this year. This cohort was the first to experience outcomes based education in the primary school. Problems with the introduction of OBE must have had some impact on the system. And the philosophy between the old and new is not the same. It is essential that problems be attended to ensure an overall general improvement of quality education in schools.

Conclusions:

1. Pass–fail criteria and statistics provide limited, yet important information.

2. Interested parties must draw their own conclusions regarding individual results. There may be unfortunate reasons behind failure and need to be attended to.

3. The results do provide important information regarding the quality of secondary education at individual schools. A concerted and continuous effort is necessary to bring about improvement at these schools. Specific programmes are required in this respect, while a team of experts is required in each province to bring about change.

4. Future pass rates will not be improved without directed effort to raise standards in schools where pass rates are low.

5. In spite of the weaknesses of the external examination system, the examination exercises a moderating role on the secondary school system as a whole and has to remain in place for the foreseeable future for quality assurance.

6. The quality of education has to be improved, not just the measure of marks.

7. Although the examination marks the end of a phase of education, it is not the end of learning and it does not mean that the learning is complete. More formal and informal learning follows. Employers and institutes of higher and further education are expected to continue with life-long learning from where schools left off. Nevertheless, schools are required to exercise the highest standard possible.

[i] The Star, 31/12/2005

[ii] 1982. Crouse C F, Erens G and Steffens F E. Ondersoek na die stel van minimum Matriekulasie-vrystellingvereisters

[iii] Business Day, 30/ 12/2005

[iv] ibid. p 3 – ‘Unacceptable‘ pass rates still cause concern.

[v] Cited by Wolf 1986.Evaluation in Education. Pergamon Press. p.8.

For any further information or contact, please contact Chantelle on:

E-mail: Chankley@twr.ac.za

Tel: (0110 406-2365

Fax: (011) 406-8138

Table of different pass rates 2005

Province