ABSTRACTS
|
ABSTRACTS
ABSTRACTS
Contributors
Contributors Abstracts '98
Conference Schedule
Contents
Journals & Bulletins
Search
DiscussionsAndrich, David (D1.1a, D1.2a, D1.3a, D1.4a) One day Professional Development and Training Workshop:
Fundamentals of Measurement in Education and Training
The training session will consist of two main parts: first, the construction of assessment tasks using Outcomes Based Education (OBE) as a guiding principle; second the analysis and interpretation of results of assessments, which are compatible with OBE, using elementary and advanced principles of modern measurement theory.
Part 1: Principles of Outcomes Based Education and test construction
(More advanced)
OBE focuses on student performance as evidence of competence at some prescribed level. OBE also implies a hierarchy of achievement as a student progresses through levels in a range of educational learning areas. This focus and implication has consequences for how to construct assessment tasks. Concrete examples based on the work of a large monitoring standards project in education in Western Australia will be used to illustrate the key points.
Part 2: Principles of Item analysis and interpretation consistent with OBE
The analysis of assessment results from school level to large system level can be made compatible with the construction of the assessments so that the analysis is informative about both the quality of the assessment tasks, the competence of the students, and the understanding of the learning area construct .
Part 2 will also fall into two parts. The first part will deal with an intuitive and informal approach to analysing and interpreting data. It essentially involves a Guttman framework for item and person analysis. This approach can be carried out readily with a calculator. The second part will deal with a more formal approach to measurement and analysing large scale data. It essentially involves the Rasch framework for item and person analysis.
The audience for the workshop is expected to be professional educators who are embarking on outcomes based education and who are involved in constructing, administering and analysing compatible assessments.
Andrich, David (D3.1)
Outcomes Based Education (OBE)
Outcomes Based Education is either a theme, or is explicit, in many countries where processes are in place to change, improve, or revive education. Taking the perspective that OBE is a worthy goal to pursue, this paper first considers its historical predecessors, in particular the work of Bloom and his colleagues in the 1960s in the development of the Taxonomy of Educational Objectives. This Taxonomy became required reading in most courses in teacher education around the world, and in particular, influenced the construction of tests and other assessment instruments.
Second, the paper considers how the Taxonomy became entangled with the behaviourism as a model of learning, and how it lost favour when "Educational Objectives" became "Behavioural Objectives". One trap in moving from Educational to Behavioural Objectives is that in the latter, the behaviours become ends in themselves rather than evidence of the attainment of Educational Objectives. A second trap is that the behavioural objectives become very small and require a compendium of large check lists assessing minute behaviours. The paper finally considers how OBE might resist falling into the traps into which the Taxonomy fell, and to consider how OBE can be used constructively to define learning goals and to monitor progress towards the achievement of these goals.
Barry, Denise (D3.2a)
WORKSHOP: OBE and classroom reform: How to use performance-based assessment for the implementation of OBE principles within the Continuous Assessment Model - How to do it!
International experience indicates clearly that changing the educational bureaucracy does not automatically change the system of education or the nature of its provision.. Research in international curriculum reform indicates the need for control to be exercised far closer to the individual school than has been the custom previously. The most successful fundamental system changes have been achieved through the change of teacher practice. The fundamental unit of education) namely the classroom and its teacher is the unit that needs to be affected in any reform of education.
Educators in SA realise that to change education in SA there is a need to undergo systemic change rather than merely component change. The policy planners now have to move from the curriculum development mode into the implementation mode. There is an urgent need to train, develop and support teachers in understanding outcomes-based educational aims, methodology and assessment for the successful implementation of curriculum 2005. Herein lies the challenge in education!
This workshop will focus on how the principles of OBE influence assessment and how to plan, design and implement outcome-based assessment in the classroom to ensure positive backwash in teaching and learning. Special attention will be paid to the use of performance assessment in the classroom to ease the tension between effective formative assessment practices to drive positive teaching and learning practices and summative assessment for high stakes decision making and accountability purposes.
Basson, Ray (D2.7a)
Evaluating curriculum development in the community of Boitekong, a consolidation of informal squatter communities in North West Province.
Curriculum development at Boiltekong and taking the form of schooling for children and 'catch-up' and continuing education for adults in face-brick primary and secondary schools and a community college, was initiated by the community in close alliance with apostleates of the Order of Jean Baptiste de la Salle and in loose co-operation with the state, local mines, donor agencies and NGOs. This unsolicited ''connoisseurship" (Eisner 1984) evaluation focusing on describing good practice and disclosure, used a range of ethnographic tools including presentations by the community, observations of Boitekong Education Forum meetings and interviews, it suggests that: curriculum development in this community exemplifies development in rising Africa countries; language usage in Forum meetings is experienced as empowering, thereby enabling the community to take a measure of control over educational provision for the community in response to its mandate; and, that it redefines the meaning of the term 'curriculum developer' bringing into focus the disempowered and under-educated as key actors in this process at Boitekong.
Bellis, Ian (D3.6)
Fixed points that don't stand still: issues of content, process, product and praxis in Assessment and Curriculum
1. Outcomes-based, competency-based approaches as the current context of assessment in education, training and development.
2. Identifying and describing the intended destinations of learning; the significance of verbs.
3. "That which is to be assessed" as the starting point of curriculum and assessment design,
4. The implications for assessment when
- (outcomes) learning and performance are in cognitive, affective and psychomotor domains,
- full achievement is the change issue,
- interim or stepped achievement is the change issue,
- movement is the change issue.
Berkhout, Sanie (D2.2a)
From matriculation to articulation - on the possibility of quality assessment
Apart from the formal and intentional transformation of the South African education system it should also been seen in the wake of a global transition. Testing, evaluation or assessment can be viewed as one of the most powerful mechanism of social control and as such plays a salient role in any transformation process. The 'matriculation examination' (the colloquial name for the school- leaving examination) is also targeted for restructuring. This examination is seen to be one of the symbols reflecting the historic imbalances of the South African education system. Within the context of the National Qualification Framework it is expected to be transformed to promote life long learning by enabling mobility, progression and articulation. This paper aims at analysing the impact of some international and local contextual developments on the possibility of realising quality assessment.
Chamberlain, Jeff (D2.4a)
Developmental assessment in education - an Australian Perspective
The paper is introduced with a brief outline of basic concepts in educational assessment that have been widely (i.e., internationally) discussed during the past century, such as: a definition of measurement; the qualities of a good test; norm- and criterion-referencing. The implications of the concept of developmental assessment and its relation to the profiles for Australian schools is outlined. Finally a number of developmental assessment resource kits for assessing English language that were developed at the Australian Council for Educational Research will be discussed with the main emphasis being on the Developmental Assessment Resource for Teachers.
Chamberlain, Liz (D1.1c)
Workshop: Curriculum assessment using outcome/level frameworks
Course objectives / outcomes are mapped on to prepared frameworks, giving an indication of coverage of the required outcomes. At present the frameworks we have only apply to science, teaching and technology education, but the process is easily transferred to other learning areas.
Clarke, Nikki (D2.4b)
Implementing criterion-based assessment
In order for a criterion-based assessment system to work, the criteria selected for assessment need to be clearly measurable and testable. Teachers and learners must be able to easily recognise and without hesitation and uncertainty know that the criteria have been attained or not attained.
To define criteria so narrowly and specifically, however, is not the intention of Curriculum 2005. In fact, the Curriculum Framework Document warns against reducing outcomes to "narrow statements of measurable behaviour" for fear of South Africa ending up with a mechanistic education system that doesn't lead to the development of the critical outcomes.
And so we find that the specific outcomes, assessment criteria and range statements that have been developed, are at every level broad, generic and open-ended. And we are left with the question, "How does one assess such broad, generic and open-ended outcomes?"
In this paper, three suggestions will be made as to how the broad, generic, open-ended criteria can be assessed:
1. Using level descriptions to assess process outcomes.
2. Using context-specific performance indicators to assess outcomes which have a wide range of requirements in terms of knowledge, skills and attitudes.
3. Using formal, continuous assessment when outcomes need to be assessed differently in different contexts.
These three suggestions will be explained and examples for each provided. The supporting reporting and recording forms for these suggestions will also be included in the paper.
Coats, Maggie (D4.1)
Learning-as-assessment: an outcomes-based approach
Attitudes to assessment are changing. The move to outcomes-based assessment has implications for all aspects of curriculum design, as well as delivery. Introducing an outcomes-based curriculum can lead to a more controlled approach to teaching and learning, or, alternatively, to a positive and creative enhancement of the learning process. It is the assessment methodology and its implementation that influences the learning experience.
This paper argues that an outcomes-based curriculum provides an opportunity to move the emphasis from the traditional uses of assessment for measurement and judgement, for classification and selection, to a greater appreciation of the importance of assessment to the learning experience. It takes the discussion further in considering assessment as an integral part of the learning process - assessment-as-learning - in a way that is compatible with an outcomes-based curriculum.
In this approach assessment is seen as a process in which teacher and learner both play an active part, in preparation, monitoring, self-assessment and reflection. It uses an adaptation of the learning cycle to encourage reflection-in-action and reflection-on-action, thus developing learner autonomy and the skill of learning how to learn. The paper will address both theory and practice, drawing on work in Australia and the USA, as well as our experiences at the UK Open University. It will present a framework that can be implemented in any course, at any level and in all educational sectors.
Dommisse, Joan (D1.3c)
Workshop; Effective assessment in English teaching - the essential role of setting criteria.
Teachers' energies an time are being stretched to the limits in the present teaching situation and it is a matter of urgency to use our personal resources productively.
In line with the principles of OBE where the individual's development is addressed it is important to examine methods of assessment that achieve the best possible results. An integral part of assessment is the setting of criteria. The pupil should be aware of criteria beforehand.
The most prevalent method of assessment of original writing in most schools is probably summative. Such summative assessment is unavoidable but there is need for greater interplay between process and product in these assessments. The mark given is final and there is little incentive for the pupil to apply the constructive advice given in the assessment. Studies show that a process writing approach achieves better results. The workshop will cover the above areas of concern and investigate different forms of assessment.
Drost, Andries. W. (D3.5b)
Pruning a bamboo bush: Some thoughts about assessment of quality in education
In the evolved cultural climate of the twentieth century Western thought the quest for excellence has become characterized by a focus on quantity: More, quicker and bigger have been the default modes of the drive towards progress.
In the backyard of my sub-urban garden I planted a bamboo stalk in 1973. Since then we could observe how this bamboo plant steadily grew into a dense bush, spreading wider each year and progressively producing thicker, longer and healthier stalks. By 1993 a strong, thick, straight and healthy stick of record length (9 m long) was harvested from the bamboo bush, which by that time covered the largest portion of the backyard. During the last 5 years, however, the quality of the new sprouts declined each year in spite of good rain and ample supplements of organic fertilizer.
During my winter vacation of 1998 I decided that it was time to prune our bamboo bush, in order to create scope for new growth and sprouts of a better quality. While engaging in this pruning process, some interesting correlatives between the growth of my bamboo bush and the development of our education system came to light, which I would like to share as the topic of a paper at this conference on assessment in education.
Farhangpour, P (D1.2c)
Workshop: Creative ways to assessment of attitudinal objectives/outcomes
The workshop consists of two parts:
Part I: Theoretical (15 minutes)
A. Introduction: Short introductory explanation on Objectives and Outcomes. The relationship between them and their differences.
B. Types of Objectives & Outcomes: The variety of outcomes are discussed with reference to Outcome Based Education and Curriculum 2005.
C. Attitudinal Outcomes: What and why of attitudinal objectives/outcomes are explained briefly and a few ideas for implementing and assessing them are suggested.
Part II Practical (60 minutes)
A. Two different activities which aim to improve attitudes are introduced. The first activity consists of a case study, while the second is a game.
B. The activities are assessed by the individuals and the groups.
Fourie, Magda and Hay, Driekie (D3.2b)
Assessment challenges as a result of the massification of Higher Education
Massification of higher education has amongst other things, led to larger class groups, particularly at the undergraduate level, without an attending increase in lecturing and tutorial staff. This has lead to the increasing use of multiple choice test and exam questions for assessment, and a decline in the use of essay-type questions. The pro's and cons of the two types of assessment ( multiple choice vs. Essay-type) and the effect that they have on student learning and development will be addressed.
Massification has further lead to increasing numbers of students from historically disadvantaged educational background entering higher education. These students are mostly second or third language speakers of the institution's medium of instruction, and they often have problems in developing communicative competence in subject discipline discourse or to become literate in the subject specific terminology of academic disciplines.
This paper further contends that the over-use of multiple choice questions to the detriment of essay-type questions will not promote the development of academic discourse skills of students, which is prerequisite to function adequately in their job environment. Possible alternatives to remedy the situation will be proposed.
French, Edward (D3.3b)
Assessment and educational reform in the United States
This paper reflects on a study tour which focussed on the use of assessment in educational reform in the United States, and particularly in Kentucky.
Assessment plays a major role in South Africa's plans for the transformation of education and training. The pressures to do this have partly been global, but are concentrated in the Anglophone world. However, the models on which SA have drawn are largely from the (British) Commonwealth. This paper looks at what we might learn from the American experience of using assessment in educational reform. The range and variety of this experience is overwhelming, and its contexts differ widely from ours. As in other countries, the experience is too fresh to have been evaluated persuasively. But the debate is intense.
Particular issues touched on in this paper are:
1. public and educator perceptions of assessment and their impact on reform
2. the promise, contradictions and problems in using formative (authentic, performance) approaches for a range of purposes: eg summative, systemic evaluation
3. the distortions and dilemmas of "low stakes", "high-stakes" and disciplinary uses of assessment, and
4. the cost of forms of assessment that promise much more than scores.
Gerber, Robert E (D4.2a)
Viewing continuous assessment from a different perspective
The National Qualifications Framework (NQF) stresses the importance of evaluating a learner's competence/ability to demonstrate learning outcomes related to knowledge, skills and attitudes appropriate for job related ventures. The South African Qualification Authority (SAQA) proposed seven critical cross-field outcomes and added five guidelines to the critical outcomes. The Department of Education's Outcomes-Based Education (OBE) policy also underlines the idea of teaching and learning to be focussed on attaining certain outcomes as a demonstration of accomplished competencies in preparation for a job and citizenship.
Continuous assessment (CA) is singled out as one of the important vehicles to assess learning and teaching as a demonstration of specific outcomes as accomplished competencies in preparation for a job and citizenship. Current literature and guidelines mainly focus on CA as an evaluation strategy.
To accomplish the critical outcomes, guidelines and the aims of the National Department of Education, the focus should be on CA as part of a learning-teaching-assessment-feedback strategy and not merely as an assessment strategy.
This paper will endeavour to state the necessity of applying CA as a part of a learning-teaching-assessment-feedback strategy by:
(1) pointing out the current views on CA;
(2) indicating what the main goal in using CA should be;
(3) differentiating between assessment and evaluation;
(4) discussing the learning-teaching-assessment-feedback cycle; and
(5) discussing CA as a part of a learning-teaching-assessment-feedback strategy.
Gerber, Robert & Lombard, Elsa (D1.3b, D1.4b)
Workshop: The development of Assessment grids and rubics as assessment tools
The focus that the outcomes-based education (OBE) approach places on continuous assessment necessitates knowledge of skills in and attitudes towards the development a variety of criterion-referenced assessment tools. These tools should also facilitate feedback to improve learning and teaching. Assessment grids and rubrics are criteria referenced assessment tools that supply feedback to the learner and teacher about the level of outcome attainment.
This training session will inform participants about:
(1) the rationale for using assessment grids and rubrics,
(2) the structure of assessment grids and rubrics, and supply an opportunity to develop skills in generating assessment grids and rubrics in variety of learning areas.
Green, Sylvia (D3.3a)
The Cambridge Primary Assessment Scheme (CamPAS) Project
This paper will report on the Cambridge Primary Assessment Scheme (CamPAS) Project, which is part of a research programme being undertaken by the Primary Assessment Unit. The aim of the project is to develop support assessment materials for Primary teachers which can be used flexibly within the classroom context, providing valid and reliable information about pupil attainment. The development of materials has been informed by research which has identified key elements which contribute to effective assessment at primary level. The model for the research and development in Mathematics and Science will be described. The focus of the CamPas project is currently Literacy' and there will be information on this latest phase, which is at an early stage of development.
Hatting, Mariette (D3.2d)
Poster presentation: Assessment of outcomes in food practical
A question marks hangs over the issues of objectivity, reliability and validity when looking at the assessment of outcomes in food preparation. A variety of training establishments offer courses in food preparation and it seems that the assessment practices differ one from another. Unit standards are therefor in question. It seems that each trainer applies his/her own sets of criteria when assessing food practical examinations. When employers want to appoint a candidate, they are unsure about the method of assessment and the level of outcome. The students are also unsure of what is expected of them, because of the difference in criteria. The issues were followed up in perspective by doing a pilot study. Trainers were asked to identify their methods of assessments and problems were confirmed. Research is therefore planned to address these limitations and to propose a solution. The first step would be to start analyzing and evaluating assessment practices of as many as possible cooking schools. The research will be done by attending practical sessions and by sending questionnaires to students, employers, trainers and evaluators. An alternative evaluation system that will support outcomes-based education, is seen as the end product of the research. An evaluation model will be designed to be used by all trainers and evaluators in the Hospitality industry.
Hay, Driekie and Fourie, Magda (D4.5b)
An action research experiment with continuous assessment and portfolios
The implementation of Curriculum 2005 and the shift to Outcomes-Based Education have significant implications, not only for school education, but also for teacher training in South Africa. One of the prerequisites for the successful implementation of the new teaching-learning strategies is the development of new assessment competencies and practices by teachers, which includes continuous assessment.
By continuously evaluating pupils' learning oven an extended period of time, continuous assessment assesses a range of skills and abilities much wider than those normally evaluated by conventional tests and examinations. This paper not only reviews a number of continuous assessment methods, but also argues that, in teacher training, it is essential to provide opportunities for student teachers to actually experience continuous assessment methods, instead of simply hearing about them.
The primary focus of the paper is to report on a small experiment with continuous assessment and portfolios conducted with student teachers in second language teaching and learning. An action research approach was followed, and valuable perspectives, particularly on the use of portfolios as part of continuous assessment, were gained.
Herbst, Dorothea (D2.2c)
The development of a dynamic assessment instrument for use in South African Schools
Dynamic assessment is considered useful in cognitive education where the focus is child-centered and special attention is given to cognitive processes and assessment of learning potential. Dynamic assessment is a procedure where the examiner interested in a child's responsiveness to instruction or learning ease, generally uses a test-teach-test format.
The early identification of learning potential of learners especially between the ages 6 to 8 years old necessitates the development of a dynamic assessment instrument. The class teacher needs to be adequately equipped to identify learning problems and to take remedial action at a very early stage. Dynamic assessment methods are used to identify students who are likely to experience academic problems, and with providing descriptions of those learner's strengths and weaknesses in such a way that remedial programmes can be developed.
The HSRC and the Group: Human Resources regard the development of a dynamic assessment instrument that focuses on the process of learning and cognitive functions as a priority. The approach holds promise as a non-discriminatory approach to the assessment of the learning potential of culturally different, handicapped, language different and learning-impaired persons.
Howie, S J (D4.2b)
South African final-year students' performance in TIMSS: The value of international comparative surveys
The paper discusses the value of international surveys of students' achievement for South Africa and pays particular attention to some of the lessons that can learn from the Third International Mathematics and Science Study. It reports on some of the mathematics and science results of the South African component of TIMSS and the comparisons drawn with a number of selected countries. It highlights the fact that the overall performance in South Africa was exceptionally weak and investigates some of the possible causes of this, including the influence of language and the resultant disadvantaged position of most South African students tested in the study. Finally, some overall conclusions are made from TIMSS which could serve as a contribution for improving mathematics and science education in South Africa.
Kanjee, Anil (D2.1)
Assessment - the South African challenges for the next decade
The issue of assessment in the education and training sector is one of the most crucial aspects of the current transformation initiatives. Since 1990, and especially post 1994, this field has expanded to levels never seen before in South Africa and has presented all stakeholders, especially researchers, with exciting challenges. This paper traces the development of assessment practices in the educational and training sector in South Africa with specific emphasis to post-1990. Second, some of the current developments in the area of assessment are discussed. These include: implementation issues in the context of OBE and the NQF, policy issues, admissions, international links and profile, financial implications, certification and computerised testing. Third, the challenges facing assessment practitioners in South Africa, both long and short terms are noted.
Specifically, the use of assessment for the improvement of "dysfunctional schools" will be discussed. Other challenges include equity and legal issues, the social, political and economic consequences of assessment, implementation of a national assessment system, application of technology, the development of a relevant research agenda, implementation of training programs to meet the increasing demand of assessment expertise, and the issue of a professional assessment organisation. Finally, comparisons to international trends and developments will also be discussed.
Kivilu, J M (D2.4c)
Improving The Quality Of Inferences Drawn From Test Scores: The Issue Of Test Bias and Fairness in Assessment Practice
Background.
There is historical evidence all over the world that assessment tests and methods in education and psychology have been abused and used against certain groups by political establishment and even professionals to deny members of such groups certain rights especially the right to education and to employment opportunities.
Aim.
The issue of bias and fairness in the use of test scores will be discussed. Some computer programs used to identify biased items will be explained and the interpretation of the indices given.
Content.
The distinction between test bias as a statistical characteristic and fairness as a value judgement will be highlighted. Importance of both concepts in making valid inferences in educational and psychological assessment will be emphasised.
Conclusion:
There is no one method of identifying biased item that is adequate. There is a need to consider a number of procedures available in both classical test theory (CTT) and the item response theory (IRT) especially in 'high-stake' situations. Recommendation is made that assessment procedures should be developed or chosen so that inferences drawn about the knowledge, skills, and behaviours possessed by students are valid and not open to misinterpretation.
Lenaghan, Mike (D2.5b)
Lifelong Learning - RPL - Assessment and Accreditation
The pain of change is with us in the field of education and training.
The major problem that this part of Africa has is the fact that the need to give credit to work-based, experiential and lifelong learning of the people is in the hands of a few - who because of the privileged positions they have held, has kept them from glving the opportunlty tc the rnajority of the people
LEARNING FOR ALL PEOPLE is a right The assessment and acceditation of their LIFELONG LEARNING is also a right. Although now a part of both the Bills on Education and Labour it is the world of formal education that is not prepared to join the initiative The private education field has accepted the importance of RPL.
In thls paper the value of assessment and accreditation is taken to its natural end - its value to the countries that accept RPL and use it correctly, not only to adjust the past but also to place those countries who accept the importance of assessment and accreditation of lifelong learning into the correct position (in the productivity rating m Africa and the world.
The importance of the LEARNING nation - community - organisation and the individual will be looked at in depth
Those who attend this presention will leave with a process that works
Lombard, B J (D2.5a)
Utilizing assessment centres in teacher education
In this paper, it is envisaged to explore the implementation of assessment/appraisal centres with regard to teacher education and development. By using the critical outcomes specified by SAQA as basis, the Outcomes for Teacher Education as proposed by the Technical Committee on the Revision of Norms and Standards for Teacher Education (November 1997) will be investigated in order to justify assessment/appraisal centres as a means of evaluation in teacher education and development.
Long, Clive (D1.4c)
Assessing Process Skills in Learner-centred Science
The session will explore practical approaches to the assessment of using process skills to investigate phenomena related to the Natural Sciences' (Specific Outcome 1), in a learner centred programme.
Research findings indicate that the high expectations science educators have of the outcomes that can be achieved using traditional science practical work are seldom reached in practice. During this workshop participants will discuss what it means for a learner to be better at Natural Science Specific Outcome 1, and examine techniques for increasing the number of learners who achieve this outcome at these higher levels. Participants will try assessing the level at which learners achieve the outcome, based on descriptions of the learner's work and sample criteria.
Lubisi, R Cassius (D3.5a)
Teachers' Judgements in Classroom Assessment: Lessons from Afield
One of the central tenets of outcomes-based education in South Africa is that teachers are supposed to assess students' work and actions with reference to explicitly stated outcomes. An assessment framework which includes assessment criteria, range statements and performance indicators has been adopted in official curriculum documents as the one that would help teachers assess students' attainment of specific learning outcomes. In this paper, I shall cite research from other countries where similar ideas had been tried, in order to make sense of the challenges that face South African teachers in applying the official assessment framework. The main argument in this paper is that the official framework is likely to be determined by different teachers depending upon their views of what constitutes competence in their relative subjects and on material conditions in their schools and classrooms. The role of non-cognitive criteria in teachers' mediation of official assessment criteria will also be explored. Most of the illustrative examples used in this paper come from mathematics classrooms.
Mack, Psaswana Modiba (D2.7c)
Assessment of Pupils in Grade 12 through the South African School leaving examination in the Northern Province
Throughout the South African democratic years viz. 1994 t0 1997, assessment of pupils in Grade 12 in the South African Senior Certificate examination (final school leaving examination) in the Northern Province has resulted in a high failure rate of pupils compared to the other provinces.
This paper focuses on the assessment of pupils in the Northen Province by the provincial Senior Certificate Examination since the democratic years. It starts with a brief historical background of the South African School Leaving examination, setting of question papers; the marking of scripts; related processes as well as the required quality assurance.
Finally, this paper aims at giving an account of the performance of Grade 12 pupils and recommending strategies for improving results.
Mohlala, Mpho (D3.4c)
Challenging common misconceptions regarding outcomes-based assessment strategies
The successful implementation and operation of Outcomes-Based Assessment (OBA) strategies depends, amongst other things, on common understanding of concepts related to such strategies. However, a reflection on our assessment training experiences point to common misconceptions by various stakeholders, and the misconceptions threaten to undermine the success of the new education system. It is therefore imperative to provide clear explanations about these misconceptions so that educational change initiatives may proceed smoothly. Furthermore, the clarifications will undoubtedly help practitioners, in particular, to handle assessment issues in the classroom with relative ease and confidence. Following are some of the common misconceptions: the notion that Continuous Assessment (CASS) consists of highly frequent formal assessment tasks; teaching and assessment are separate entities, formative assessment is good and summative assessment is bad; Outcomes-Based Assessment/education (OBA/E) means the lowering of standards; overall learner performance against fixed / "objective" standards does not change; learner-centered teaching and lots of group work is exactly the same as Outcome-Based Education (OBE); assessment consists of ticking off outcomes as the teacher deals with them; "good" teachers have always been practising OBA/E.
Moodie, Peter (D2.2b)
A curriculum evaluation: Primary-school teachers' reactions to innovation in outcomes-based assessment, and their adaptations to policy
This evaluation explores the possibilities for introducing the intended OBE curriculum in Natural Sciences in the light of its dependence on extensive innovations in assessment. The project at present focuses on Grades 5, 6 and 7.
One main aim is to describe teachers interpretations of OB Assessment as they encounter it in a pilot programme-. The assumption here is that teachers never implement policy: they adapt policy in terms of their own conceptions of teaching and in terms of reducing administrative and management conflicts.
The other aim is to investigate the extent to which children can use process skills and deal with unfamiliar problems, because there is a lack of information available for writing levels statements of outcomes which teachers can use to decide whether children are "getting better at the outcomes".
Through interviews and frequent interaction in classrooms we are building up a picture of principals' und teachers' concerns. We are reaching the sensitive stage where schools will issue reports to parents on the children's process in science outcomes. We are finding that the ways in which teachers mediate tasks is crucial to assessing children's abilities. We are finding that children cope better than their teachers expected with some tasks, and we arc finding that the children cope much worse with other tasks which we were taking for granted.
Moore, Bob (D3.4a)
Administering experiential learning programs
An exploration of the foundations of experiential learning and its importance for professional education provides for an program designed to integrate university level learning and experience. Differentiating between "experiential education" and "training," a model is presented that creates a hierarchial structure integrating experience and learning from entry level through mastery.
Moore, Bob (D4.5a)
Assessing learning: the Capstone Course
Following a professional curriculum integrating cognitive and experiential learning, this final course provides for formal assessment at the end of the students' course of study. The Capstone Course is designed to assess cognitive, affective, and psychomotor learning and to do so in a student-centered and student-directed manner that requires the command of analysis and synthesis of knowledge and skills. Expectations in the course require students and faculty to foster interdisciplinary partnerships and cultivate business and industry alliances.
Moore, Bob (D2.3)
Curriculum planning
The presentation is based on the actual curriculum development plan in the Department of Communications at Elizabethtown (PA) College (USA). A sequential process, the plan includes development of definitions, mission statements, curriculum model, integration of philosophy to actual goals and objective, operationalizing goals and objective through courses, and developing an outcomes assessment plan.
Nel, B R (D2.5c)
Quality teaching, quality assessment, quality learning. Which one comes first?
This paper will argue that one cannot discuss quality assessment and quality learning without discussing quality teaching. It will argue further that quality teaching ought to be the first step in the process that culminates in quality learning.
At the same time it will attempt to unpack what is meant by quality teaching, as well as call for the need for training in the scholarship of teaching, especially in higher education where there appears to be a worldwide (global) movement to establish structures that aim for excellence in teaching.
This will also argue that the evaluation of teaching, learning and assessment is a crucial process that must be in place in order to ensure the element of quality.
In conclusion it will state that quality learning is aided and directed, not through quality assessment, but also through quality teaching and evaluation, both essential skill requirements for the 21st Century in education and training.
Nku, Samuel (D3.5c)
Explorations in local government career pathing using assessment techniques
VISION, POLICY AND IMPLEMENTATION: AN EVALUATION OF THE SCIENCE EDUCATION PROJECT
by Isaac Ntshoe and Charles Potter
Vista University and the University of the Witwatersrand, Johannesburg
In previous evaluations of projects in engineering and science education, Potter (1991) and Potter and Moodie (1992) used an evaluation model which focused on a number of levels. The first was the level of vision and intentions of the stakeholders involved in the project, the second the level of curriculum conceptualisation, policy and planning, and the third the level of implementation, or action.
In this paper, the authors report on an evaluation of the work of the Science Education Project (SEP) using a similar transactional framework, based on Decker Walker's (1972) conception of the curriculum as a curriculum as a dynamic between the vision of those involved in the curriculum development process and the deliberation which occurs around the development (usually centring around a notion of materials-in-use), Stenhouse's (1975) distinction between curriculum as intention and curriculum as action, and Hall et al's (1975) conceptualisation of curriculum implementation in terms of "levels of use".
The evaluation focuses on SEP's work in four regions of South Africa from 1980 to 1995. The project has subsequently ceased operating in these regions due to lack of funding. As the data cover the period leading up to the cessation of the project's work, it has been possible to draw conclusions concerning the value of the project's work in relation to policy issues in science education in South Africa, as well as some of the reasons why institutionalisation of the project's approach did not take place. The evaluation is thus relevant both to the development and implementation of other curriculum development projects in post-apartheid South Africa, as well as the development of science education under the new educational dispensation.
Olivier, Cas (D2.6)
How to educate and train Outcomes-based
Re-engineering learning towards the outcomes-based method is a major attempt to build the country into becoming an international role-player. The shift to outcomes learning versus content- and competency based learning, is analogous towards the total quality movement in business and manufacturing. Outcomes-based learning reflects the notion that the best way to get where you want to be is to first determine what you want to achieve. Once the end goal (product/outcome) has been determined. strategies, techniques and other ways and means will be put into place to achieve the goal.
The outcomes-based learning approach intends to focus equally on mastering of knowledge, skills, processes of learning as well as the achievement of the final outcome/result/product/output. In this sense the process of achieving outcomes through learning can be directly related to the way in which outcomes are achieved in the world of work.
Potter, Charles (D3.2c)
Evaluation, Knowledge and Human Interests
Being an applied field, evaluation has traditionally drawn its methodologies from other fields of social science research, and is essentially an applied, contextually based and holistic field of research, concerned with social development. With respect to the evaluation of social programmes, the designs used by evaluators are based in three broad traditions, namely, positivist, interpretive or critical-emancipatory. Each of these traditions reflects different assumptions concerning science and knowledge, and the role of the evaluator in relation to participants in the programme.
In the paper, the development of these different traditions in evaluation is traced, in terms of an historical account of the development of evaluation as it has evolved from its origins in the testing movement. Contextual issues influencing the development of evaluation are described, and related to the characteristics of evaluation models developed at different stages in the development of evaluation as a field. The paper concludes with comment on current trends in evaluation, and suggestions as to emerging and future trends in evaluation as a field of inquiry.
Samuels, Joe (D4.3)
The Principles and purposes of SAQA
Scheiber, Jackie (D1.1b)
Continuous Assessment, and what it means for them in the classroom
Continuous Assessment, and what it means for them in the classroom, is a concept that is of concern to many teachers. This is a workshop on Assessment for Senior Phase teachers (Grade 7, 8 and 9) that was developed under the auspices of the Ikaheng project, a joint venture between the Gauteng Education Department and AMESA (Association of Maths Educators in South Africa) to try to address this need.
In this workshop we look at how to assess group work and the pupils participation in the group, self assessment, peer assessment, assessment through classroom observation, cumulative checklists and student work portfolios.
Although this workshop was developed for maths teachers, only the first activity is specifically maths related. All the other issues discussed are relevant to teachers of all the other school subjects.
Shaw, Fred (D4.4)
Quality Assessment, Quality Learning: Standards
The author looks at the advantages and disadvantages of different approaches in assessment and learning. The author tries to reconcile the different principles with one another and attempts to draw conclusions from the current theoretical and practical situation in education and training, while taking into account the desired outcomes in the field.
Shingles, Christine (D 1.2b)
Workshop: Assessment for foundation phase teachers
The presenter will facilitate a workshop for Foundation Phase teachers [Grade 1 - Grade 3] in the basic a) principles, b) principles of procedure, and c) strategies for accountable assessment of learning outcomes to meet the requirements of Curriculum 2005. The focus of this workshop will be strictly practical and 'hands-on', involving group work and sharing. The
theoretical component of the workshop will consist of a brief overview of the principles of assessment for early childhood development [ECD] and how these principles relate to the departmental prescriptions circulated to schools. An attempt will be made to look at the practical implications of the various functions of assessment in the Foundation Phase and how these culminate in the end-of-term report.
Steenkamp, Karen (D4.2c)
Implications of Outcomes Based Assessment for a transforming workplace
Twalo, Thembinkosi (D3.3c)
Process writing in a multilingual setting
After an analysis of the education system in South Africa and the kind of students it produces, I propose in this paper that one probable solution to the unsatisfactory academic writing produced by the students would be a continuous and developmental form of writing. In this kind of writing, the learner and the ducator communicate and address the learners' problems through process writing and quality assessment. This is based on the assumption that writing is not an event but a process. I used Rhodes University Linguistics I students as subjects for my research and found out that students (mother tongue and non-mother tongue English speakers) have, for various reasons, serious problems with academic discourse. As a result, their essays fell below the standard set for academic essays. This paper argues that quality feedback from educators can greatly improve students' understanding of academic discourse. In the research students wrote two drafts of essays using feedback given to them on the first draft to write a second draft. The results showed general improvement and understanding of academic discourse.
Van der Vyver, John (D3.4b)
The judicious and effective use of rating scales to promote quality learning in outcomes-based courses
Soweto College of Education Currently seconded to GDE - Teacher Colleges Unit
The shift from subject-based to outcomes-based curricula in general, further and higher education in South Africa has necessitated an increased focus on criterion-referenced assessment instruments and strategies. This shift has been accompanied by the creation of a National Qualification Framework that requires national performance standards on stated outcomes. The description of such standards will of necessity, in the short term, have to accommodate the disparities in the various sectors with regard to facilities, resources and educational experience.
A number of projects have been initiated at a variety of institutions to define and describe outcomes for a range of, amongst others, teacher education courses and to attempt a description of acceptable performance levels for such outcomes in terms of appropriate criteria, range statements and performance indicators.
This paper reports on three projects currently being undertaken at a college of education, a technikon and a university in which rating scales have been developed for a range of outcomes in order to: describe performance requirements with an acceptable degree of specificity acquaint learners with respect to their current status with respect to the outcomesprovide learners with appropriate feedback to increase their level of proficiency with respect to the outcomes determine provisional standards of performance for such outcomes assist institutions to justify and/or defend learner proficiency in terms of a range of practitioner, learner and institutional variables.
The paper provides a provisional set of findings that impact on effective and efficient learning at the institution. It also indicates a possible way forward for institutions to address context variables in order to increase the quality of learning and to promote a measure of consensus around appropriate and possible national standards.
Van Koller, J F (D2.7b)
Objective assessment in social disciplines at a technikon at the end of a millennium
This paper reports on an investigation to determine to what extent objective questions were still used in social disciplines at Technikon A S. The aim was to establish to what extent developments such as the following affected the use of objective questions in social disciplines at Technikon S A.
The introduction of the outcomes-based approach in teaching by Department of Education. This approach places the emphasis more on skills than on the mere reproduction of learned facts.
A global trend towards the exploration and implementation of alternative and more authentic forms of assessment. Even the main proponents of objecttive assessment, such as the US and Japan, have also joined this trend.
The knowledge explosion of the 1990s (information systems, technology and so forth).
The author used a questionaire to test the extent to which objective questions were still used by Technikon S A staff and arranged a few focus group interviews to determine whether the attitude of staff members has changed regarding the use of objective questions in the past few years.
Back to Top
Andrich, David (D1.1a, D1.2a, D1.3a, D1.4a)
One day Professional Development and Training Workshop:
Fundamentals of Measurement in Education and Training
The training session will consist of two main parts: first, the construction of assessment tasks using Outcomes Based Education (OBE) as a guiding principle; second the analysis and interpretation of results of assessments, which are compatible with OBE, using elementary and advanced principles of modern measurement theory.
Part 1: Principles of Outcomes Based Education and test construction
(More advanced)
OBE focuses on student performance as evidence of competence at some prescribed level. OBE also implies a hierarchy of achievement as a student progresses through levels in a range of educational learning areas. This focus and implication has consequences for how to construct assessment tasks. Concrete examples based on the work of a large monitoring standards project in education in Western Australia will be used to illustrate the key points.
Part 2: Principles of Item analysis and interpretation consistent with OBE
The analysis of assessment results from school level to large system level can be made compatible with the construction of the assessments so that the analysis is informative about both the quality of the assessment tasks, the competence of the students, and the understanding of the learning area construct .
Part 2 will also fall into two parts. The first part will deal with an intuitive and informal approach to analysing and interpreting data. It essentially involves a Guttman framework for item and person analysis. This approach can be carried out readily with a calculator. The second part will deal with a more formal approach to measurement and analysing large scale data. It essentially involves the Rasch framework for item and person analysis.
The audience for the workshop is expected to be professional educators who are embarking on outcomes based education and who are involved in constructing, administering and analysing compatible assessments.
Andrich, David (D3.1)
Outcomes Based Education (OBE)
Outcomes Based Education is either a theme, or is explicit, in many countries where processes are in place to change, improve, or revive education. Taking the perspective that OBE is a worthy goal to pursue, this paper first considers its historical predecessors, in particular the work of Bloom and his colleagues in the 1960s in the development of the Taxonomy of Educational Objectives. This Taxonomy became required reading in most courses in teacher education around the world, and in particular, influenced the construction of tests and other assessment instruments.
Second, the paper considers how the Taxonomy became entangled with the behaviourism as a model of learning, and how it lost favour when "Educational Objectives" became "Behavioural Objectives". One trap in moving from Educational to Behavioural Objectives is that in the latter, the behaviours become ends in themselves rather than evidence of the attainment of Educational Objectives. A second trap is that the behavioural objectives become very small and require a compendium of large check lists assessing minute behaviours. The paper finally considers how OBE might resist falling into the traps into which the Taxonomy fell, and to consider how OBE can be used constructively to define learning goals and to monitor progress towards the achievement of these goals.
Barry, Denise (D3.2a)
WORKSHOP: OBE and classroom reform: How to use performance-based assessment for the implementation of OBE principles within the Continuous Assessment Model - How to do it!
International experience indicates clearly that changing the educational bureaucracy does not automatically change the system of education or the nature of its provision.. Research in international curriculum reform indicates the need for control to be exercised far closer to the individual school than has been the custom previously. The most successful fundamental system changes have been achieved through the change of teacher practice. The fundamental unit of education) namely the classroom and its teacher is the unit that needs to be affected in any reform of education.
Educators in SA realise that to change education in SA there is a need to undergo systemic change rather than merely component change. The policy planners now have to move from the curriculum development mode into the implementation mode. There is an urgent need to train, develop and support teachers in understanding outcomes-based educational aims, methodology and assessment for the successful implementation of curriculum 2005. Herein lies the challenge in education!
This workshop will focus on how the principles of OBE influence assessment and how to plan, design and implement outcome-based assessment in the classroom to ensure positive backwash in teaching and learning. Special attention will be paid to the use of performance assessment in the classroom to ease the tension between effective formative assessment practices to drive positive teaching and learning practices and summative assessment for high stakes decision making and accountability purposes.
Basson, Ray (D2.7a)
Evaluating curriculum development in the community of Boitekong, a consolidation of informal squatter communities in North West Province.
Curriculum development at Boiltekong and taking the form of schooling for children and 'catch-up' and continuing education for adults in face-brick primary and secondary schools and a community college, was initiated by the community in close alliance with apostleates of the Order of Jean Baptiste de la Salle and in loose co-operation with the state, local mines, donor agencies and NGOs. This unsolicited ''connoisseurship" (Eisner 1984) evaluation focusing on describing good practice and disclosure, used a range of ethnographic tools including presentations by the community, observations of Boitekong Education Forum meetings and interviews, it suggests that: curriculum development in this community exemplifies development in rising Africa countries; language usage in Forum meetings is experienced as empowering, thereby enabling the community to take a measure of control over educational provision for the community in response to its mandate; and, that it redefines the meaning of the term 'curriculum developer' bringing into focus the disempowered and under-educated as key actors in this process at Boitekong.
Bellis, Ian (D3.6)
Fixed points that don't stand still: issues of content, process, product and praxis in Assessment and Curriculum
1. Outcomes-based, competency-based approaches as the current context of assessment in education, training and development.
2. Identifying and describing the intended destinations of learning; the significance of verbs.
3. "That which is to be assessed" as the starting point of curriculum and assessment design,
4. The implications for assessment when
- (outcomes) learning and performance are in cognitive, affective and psychomotor domains,
- full achievement is the change issue,
- interim or stepped achievement is the change issue,
- movement is the change issue.
Berkhout, Sanie (D2.2a)
From matriculation to articulation - on the possibility of quality assessment
Apart from the formal and intentional transformation of the South African education system it should also been seen in the wake of a global transition. Testing, evaluation or assessment can be viewed as one of the most powerful mechanism of social control and as such plays a salient role in any transformation process. The 'matriculation examination' (the colloquial name for the school- leaving examination) is also targeted for restructuring. This examination is seen to be one of the symbols reflecting the historic imbalances of the South African education system. Within the context of the National Qualification Framework it is expected to be transformed to promote life long learning by enabling mobility, progression and articulation. This paper aims at analysing the impact of some international and local contextual developments on the possibility of realising quality assessment.
Chamberlain, Jeff (D2.4a)
Developmental assessment in education - an Australian Perspective
The paper is introduced with a brief outline of basic concepts in educational assessment that have been widely (i.e., internationally) discussed during the past century, such as: a definition of measurement; the qualities of a good test; norm- and criterion-referencing. The implications of the concept of developmental assessment and its relation to the profiles for Australian schools is outlined. Finally a number of developmental assessment resource kits for assessing English language that were developed at the Australian Council for Educational Research will be discussed with the main emphasis being on the Developmental Assessment Resource for Teachers.
Chamberlain, Liz (D1.1c)
Workshop: Curriculum assessment using outcome/level frameworks
Course objectives / outcomes are mapped on to prepared frameworks, giving an indication of coverage of the required outcomes. At present the frameworks we have only apply to science, teaching and technology education, but the process is easily transferred to other learning areas.
Clarke, Nikki (D2.4b)
Implementing criterion-based assessment
In order for a criterion-based assessment system to work, the criteria selected for assessment need to be clearly measurable and testable. Teachers and learners must be able to easily recognise and without hesitation and uncertainty know that the criteria have been attained or not attained.
To define criteria so narrowly and specifically, however, is not the intention of Curriculum 2005. In fact, the Curriculum Framework Document warns against reducing outcomes to "narrow statements of measurable behaviour" for fear of South Africa ending up with a mechanistic education system that doesn't lead to the development of the critical outcomes.
And so we find that the specific outcomes, assessment criteria and range statements that have been developed, are at every level broad, generic and open-ended. And we are left with the question, "How does one assess such broad, generic and open-ended outcomes?"
In this paper, three suggestions will be made as to how the broad, generic, open-ended criteria can be assessed:
1. Using level descriptions to assess process outcomes.
2. Using context-specific performance indicators to assess outcomes which have a wide range of requirements in terms of knowledge, skills and attitudes.
3. Using formal, continuous assessment when outcomes need to be assessed differently in different contexts.
These three suggestions will be explained and examples for each provided. The supporting reporting and recording forms for these suggestions will also be included in the paper.
Coats, Maggie (D4.1)
Learning-as-assessment: an outcomes-based approach
Attitudes to assessment are changing. The move to outcomes-based assessment has implications for all aspects of curriculum design, as well as delivery. Introducing an outcomes-based curriculum can lead to a more controlled approach to teaching and learning, or, alternatively, to a positive and creative enhancement of the learning process. It is the assessment methodology and its implementation that influences the learning experience.
This paper argues that an outcomes-based curriculum provides an opportunity to move the emphasis from the traditional uses of assessment for measurement and judgement, for classification and selection, to a greater appreciation of the importance of assessment to the learning experience. It takes the discussion further in considering assessment as an integral part of the learning process - assessment-as-learning - in a way that is compatible with an outcomes-based curriculum.
In this approach assessment is seen as a process in which teacher and learner both play an active part, in preparation, monitoring, self-assessment and reflection. It uses an adaptation of the learning cycle to encourage reflection-in-action and reflection-on-action, thus developing learner autonomy and the skill of learning how to learn. The paper will address both theory and practice, drawing on work in Australia and the USA, as well as our experiences at the UK Open University. It will present a framework that can be implemented in any course, at any level and in all educational sectors.
Dommisse, Joan (D1.3c)
Workshop; Effective assessment in English teaching - the essential role of setting criteria.
Teachers' energies an time are being stretched to the limits in the present teaching situation and it is a matter of urgency to use our personal resources productively.
In line with the principles of OBE where the individual's development is addressed it is important to examine methods of assessment that achieve the best possible results. An integral part of assessment is the setting of criteria. The pupil should be aware of criteria beforehand.
The most prevalent method of assessment of original writing in most schools is probably summative. Such summative assessment is unavoidable but there is need for greater interplay between process and product in these assessments. The mark given is final and there is little incentive for the pupil to apply the constructive advice given in the assessment. Studies show that a process writing approach achieves better results. The workshop will cover the above areas of concern and investigate different forms of assessment.
Drost, Andries. W. (D3.5b)
Pruning a bamboo bush: Some thoughts about assessment of quality in education
In the evolved cultural climate of the twentieth century Western thought the quest for excellence has become characterized by a focus on quantity: More, quicker and bigger have been the default modes of the drive towards progress.
In the backyard of my sub-urban garden I planted a bamboo stalk in 1973. Since then we could observe how this bamboo plant steadily grew into a dense bush, spreading wider each year and progressively producing thicker, longer and healthier stalks. By 1993 a strong, thick, straight and healthy stick of record length (9 m long) was harvested from the bamboo bush, which by that time covered the largest portion of the backyard. During the last 5 years, however, the quality of the new sprouts declined each year in spite of good rain and ample supplements of organic fertilizer.
During my winter vacation of 1998 I decided that it was time to prune our bamboo bush, in order to create scope for new growth and sprouts of a better quality. While engaging in this pruning process, some interesting correlatives between the growth of my bamboo bush and the development of our education system came to light, which I would like to share as the topic of a paper at this conference on assessment in education.
Farhangpour, P (D1.2c)
Workshop: Creative ways to assessment of attitudinal objectives/outcomes
The workshop consists of two parts:
Part I: Theoretical (15 minutes)
A. Introduction: Short introductory explanation on Objectives and Outcomes. The relationship between them and their differences.
B. Types of Objectives & Outcomes: The variety of outcomes are discussed with reference to Outcome Based Education and Curriculum 2005.
C. Attitudinal Outcomes: What and why of attitudinal objectives/outcomes are explained briefly and a few ideas for implementing and assessing them are suggested.
Part II Practical (60 minutes)
A. Two different activities which aim to improve attitudes are introduced. The first activity consists of a case study, while the second is a game.
B. The activities are assessed by the individuals and the groups.
Fourie, Magda and Hay, Driekie (D3.2b)
Assessment challenges as a result of the massification of Higher Education
Massification of higher education has amongst other things, led to larger class groups, particularly at the undergraduate level, without an attending increase in lecturing and tutorial staff. This has lead to the increasing use of multiple choice test and exam questions for assessment, and a decline in the use of essay-type questions. The pro's and cons of the two types of assessment ( multiple choice vs. Essay-type) and the effect that they have on student learning and development will be addressed.
Massification has further lead to increasing numbers of students from historically disadvantaged educational background entering higher education. These students are mostly second or third language speakers of the institution's medium of instruction, and they often have problems in developing communicative competence in subject discipline discourse or to become literate in the subject specific terminology of academic disciplines.
This paper further contends that the over-use of multiple choice questions to the detriment of essay-type questions will not promote the development of academic discourse skills of students, which is prerequisite to function adequately in their job environment. Possible alternatives to remedy the situation will be proposed.
French, Edward (D3.3b)
Assessment and educational reform in the United States
This paper reflects on a study tour which focussed on the use of assessment in educational reform in the United States, and particularly in Kentucky.
Assessment plays a major role in South Africa's plans for the transformation of education and training. The pressures to do this have partly been global, but are concentrated in the Anglophone world. However, the models on which SA have drawn are largely from the (British) Commonwealth. This paper looks at what we might learn from the American experience of using assessment in educational reform. The range and variety of this experience is overwhelming, and its contexts differ widely from ours. As in other countries, the experience is too fresh to have been evaluated persuasively. But the debate is intense.
Particular issues touched on in this paper are:
1. public and educator perceptions of assessment and their impact on reform
2. the promise, contradictions and problems in using formative (authentic, performance) approaches for a range of purposes: eg summative, systemic evaluation
3. the distortions and dilemmas of "low stakes", "high-stakes" and disciplinary uses of assessment, and
4. the cost of forms of assessment that promise much more than scores.
Gerber, Robert E (D4.2a)
Viewing continuous assessment from a different perspective
The National Qualifications Framework (NQF) stresses the importance of evaluating a learner's competence/ability to demonstrate learning outcomes related to knowledge, skills and attitudes appropriate for job related ventures. The South African Qualification Authority (SAQA) proposed seven critical cross-field outcomes and added five guidelines to the critical outcomes. The Department of Education's Outcomes-Based Education (OBE) policy also underlines the idea of teaching and learning to be focussed on attaining certain outcomes as a demonstration of accomplished competencies in preparation for a job and citizenship.
Continuous assessment (CA) is singled out as one of the important vehicles to assess learning and teaching as a demonstration of specific outcomes as accomplished competencies in preparation for a job and citizenship. Current literature and guidelines mainly focus on CA as an evaluation strategy.
To accomplish the critical outcomes, guidelines and the aims of the National Department of Education, the focus should be on CA as part of a learning-teaching-assessment-feedback strategy and not merely as an assessment strategy.
This paper will endeavour to state the necessity of applying CA as a part of a learning-teaching-assessment-feedback strategy by:
(1) pointing out the current views on CA;
(2) indicating what the main goal in using CA should be;
(3) differentiating between assessment and evaluation;
(4) discussing the learning-teaching-assessment-feedback cycle; and
(5) discussing CA as a part of a learning-teaching-assessment-feedback strategy.
Gerber, Robert & Lombard, Elsa (D1.3b, D1.4b)
Workshop: The development of Assessment grids and rubics as assessment tools
The focus that the outcomes-based education (OBE) approach places on continuous assessment necessitates knowledge of skills in and attitudes towards the development a variety of criterion-referenced assessment tools. These tools should also facilitate feedback to improve learning and teaching. Assessment grids and rubrics are criteria referenced assessment tools that supply feedback to the learner and teacher about the level of outcome attainment.
This training session will inform participants about:
(1) the rationale for using assessment grids and rubrics,
(2) the structure of assessment grids and rubrics, and supply an opportunity to develop skills in generating assessment grids and rubrics in variety of learning areas.
Green, Sylvia (D3.3a)
The Cambridge Primary Assessment Scheme (CamPAS) Project
This paper will report on the Cambridge Primary Assessment Scheme (CamPAS) Project, which is part of a research programme being undertaken by the Primary Assessment Unit. The aim of the project is to develop support assessment materials for Primary teachers which can be used flexibly within the classroom context, providing valid and reliable information about pupil attainment. The development of materials has been informed by research which has identified key elements which contribute to effective assessment at primary level. The model for the research and development in Mathematics and Science will be described. The focus of the CamPas project is currently Literacy' and there will be information on this latest phase, which is at an early stage of development.
Hatting, Mariette (D3.2d)
Poster presentation: Assessment of outcomes in food practical
A question marks hangs over the issues of objectivity, reliability and validity when looking at the assessment of outcomes in food preparation. A variety of training establishments offer courses in food preparation and it seems that the assessment practices differ one from another. Unit standards are therefor in question. It seems that each trainer applies his/her own sets of criteria when assessing food practical examinations. When employers want to appoint a candidate, they are unsure about the method of assessment and the level of outcome. The students are also unsure of what is expected of them, because of the difference in criteria. The issues were followed up in perspective by doing a pilot study. Trainers were asked to identify their methods of assessments and problems were confirmed. Research is therefore planned to address these limitations and to propose a solution. The first step would be to start analyzing and evaluating assessment practices of as many as possible cooking schools. The research will be done by attending practical sessions and by sending questionnaires to students, employers, trainers and evaluators. An alternative evaluation system that will support outcomes-based education, is seen as the end product of the research. An evaluation model will be designed to be used by all trainers and evaluators in the Hospitality industry.
Hay, Driekie and Fourie, Magda (D4.5b)
An action research experiment with continuous assessment and portfolios
The implementation of Curriculum 2005 and the shift to Outcomes-Based Education have significant implications, not only for school education, but also for teacher training in South Africa. One of the prerequisites for the successful implementation of the new teaching-learning strategies is the development of new assessment competencies and practices by teachers, which includes continuous assessment.
By continuously evaluating pupils' learning oven an extended period of time, continuous assessment assesses a range of skills and abilities much wider than those normally evaluated by conventional tests and examinations. This paper not only reviews a number of continuous assessment methods, but also argues that, in teacher training, it is essential to provide opportunities for student teachers to actually experience continuous assessment methods, instead of simply hearing about them.
The primary focus of the paper is to report on a small experiment with continuous assessment and portfolios conducted with student teachers in second language teaching and learning. An action research approach was followed, and valuable perspectives, particularly on the use of portfolios as part of continuous assessment, were gained.
Herbst, Dorothea (D2.2c)
The development of a dynamic assessment instrument for use in South African Schools
Dynamic assessment is considered useful in cognitive education where the focus is child-centered and special attention is given to cognitive processes and assessment of learning potential. Dynamic assessment is a procedure where the examiner interested in a child's responsiveness to instruction or learning ease, generally uses a test-teach-test format.
The early identification of learning potential of learners especially between the ages 6 to 8 years old necessitates the development of a dynamic assessment instrument. The class teacher needs to be adequately equipped to identify learning problems and to take remedial action at a very early stage. Dynamic assessment methods are used to identify students who are likely to experience academic problems, and with providing descriptions of those learner's strengths and weaknesses in such a way that remedial programmes can be developed.
The HSRC and the Group: Human Resources regard the development of a dynamic assessment instrument that focuses on the process of learning and cognitive functions as a priority. The approach holds promise as a non-discriminatory approach to the assessment of the learning potential of culturally different, handicapped, language different and learning-impaired persons.
Howie, S J (D4.2b)
South African final-year students' performance in TIMSS: The value of international comparative surveys
The paper discusses the value of international surveys of students' achievement for South Africa and pays particular attention to some of the lessons that can learn from the Third International Mathematics and Science Study. It reports on some of the mathematics and science results of the South African component of TIMSS and the comparisons drawn with a number of selected countries. It highlights the fact that the overall performance in South Africa was exceptionally weak and investigates some of the possible causes of this, including the influence of language and the resultant disadvantaged position of most South African students tested in the study. Finally, some overall conclusions are made from TIMSS which could serve as a contribution for improving mathematics and science education in South Africa.
Kanjee, Anil (D2.1)
Assessment - the South African challenges for the next decade
The issue of assessment in the education and training sector is one of the most crucial aspects of the current transformation initiatives. Since 1990, and especially post 1994, this field has expanded to levels never seen before in South Africa and has presented all stakeholders, especially researchers, with exciting challenges. This paper traces the development of assessment practices in the educational and training sector in South Africa with specific emphasis to post-1990. Second, some of the current developments in the area of assessment are discussed. These include: implementation issues in the context of OBE and the NQF, policy issues, admissions, international links and profile, financial implications, certification and computerised testing. Third, the challenges facing assessment practitioners in South Africa, both long and short terms are noted.
Specifically, the use of assessment for the improvement of "dysfunctional schools" will be discussed. Other challenges include equity and legal issues, the social, political and economic consequences of assessment, implementation of a national assessment system, application of technology, the development of a relevant research agenda, implementation of training programs to meet the increasing demand of assessment expertise, and the issue of a professional assessment organisation. Finally, comparisons to international trends and developments will also be discussed.
Kivilu, J M (D2.4c)
Improving The Quality Of Inferences Drawn From Test Scores: The Issue Of Test Bias and Fairness in Assessment Practice
Background.
There is historical evidence all over the world that assessment tests and methods in education and psychology have been abused and used against certain groups by political establishment and even professionals to deny members of such groups certain rights especially the right to education and to employment opportunities.
Aim.
The issue of bias and fairness in the use of test scores will be discussed. Some computer programs used to identify biased items will be explained and the interpretation of the indices given.
Content.
The distinction between test bias as a statistical characteristic and fairness as a value judgement will be highlighted. Importance of both concepts in making valid inferences in educational and psychological assessment will be emphasised.
Conclusion:
There is no one method of identifying biased item that is adequate. There is a need to consider a number of procedures available in both classical test theory (CTT) and the item response theory (IRT) especially in 'high-stake' situations. Recommendation is made that assessment procedures should be developed or chosen so that inferences drawn about the knowledge, skills, and behaviours possessed by students are valid and not open to misinterpretation.
Lenaghan, Mike (D2.5b)
Lifelong Learning - RPL - Assessment and Accreditation
The pain of change is with us in the field of education and training.
The major problem that this part of Africa has is the fact that the need to give credit to work-based, experiential and lifelong learning of the people is in the hands of a few - who because of the privileged positions they have held, has kept them from glving the opportunlty tc the rnajority of the people
LEARNING FOR ALL PEOPLE is a right The assessment and acceditation of their LIFELONG LEARNING is also a right. Although now a part of both the Bills on Education and Labour it is the world of formal education that is not prepared to join the initiative The private education field has accepted the importance of RPL.
In thls paper the value of assessment and accreditation is taken to its natural end - its value to the countries that accept RPL and use it correctly, not only to adjust the past but also to place those countries who accept the importance of assessment and accreditation of lifelong learning into the correct position (in the productivity rating m Africa and the world.
The importance of the LEARNING nation - community - organisation and the individual will be looked at in depth
Those who attend this presention will leave with a process that works
Lombard, B J (D2.5a)
Utilizing assessment centres in teacher education
In this paper, it is envisaged to explore the implementation of assessment/appraisal centres with regard to teacher education and development. By using the critical outcomes specified by SAQA as basis, the Outcomes for Teacher Education as proposed by the Technical Committee on the Revision of Norms and Standards for Teacher Education (November 1997) will be investigated in order to justify assessment/appraisal centres as a means of evaluation in teacher education and development.
Long, Clive (D1.4c)
Assessing Process Skills in Learner-centred Science
The session will explore practical approaches to the assessment of using process skills to investigate phenomena related to the Natural Sciences' (Specific Outcome 1), in a learner centred programme.
Research findings indicate that the high expectations science educators have of the outcomes that can be achieved using traditional science practical work are seldom reached in practice. During this workshop participants will discuss what it means for a learner to be better at Natural Science Specific Outcome 1, and examine techniques for increasing the number of learners who achieve this outcome at these higher levels. Participants will try assessing the level at which learners achieve the outcome, based on descriptions of the learner's work and sample criteria.
Lubisi, R Cassius (D3.5a)
Teachers' Judgements in Classroom Assessment: Lessons from Afield
One of the central tenets of outcomes-based education in South Africa is that teachers are supposed to assess students' work and actions with reference to explicitly stated outcomes. An assessment framework which includes assessment criteria, range statements and performance indicators has been adopted in official curriculum documents as the one that would help teachers assess students' attainment of specific learning outcomes. In this paper, I shall cite research from other countries where similar ideas had been tried, in order to make sense of the challenges that face South African teachers in applying the official assessment framework. The main argument in this paper is that the official framework is likely to be determined by different teachers depending upon their views of what constitutes competence in their relative subjects and on material conditions in their schools and classrooms. The role of non-cognitive criteria in teachers' mediation of official assessment criteria will also be explored. Most of the illustrative examples used in this paper come from mathematics classrooms.
Mack, Psaswana Modiba (D2.7c)
Assessment of Pupils in Grade 12 through the South African School leaving examination in the Northern Province
Throughout the South African democratic years viz. 1994 t0 1997, assessment of pupils in Grade 12 in the South African Senior Certificate examination (final school leaving examination) in the Northern Province has resulted in a high failure rate of pupils compared to the other provinces.
This paper focuses on the assessment of pupils in the Northen Province by the provincial Senior Certificate Examination since the democratic years. It starts with a brief historical background of the South African School Leaving examination, setting of question papers; the marking of scripts; related processes as well as the required quality assurance.
Finally, this paper aims at giving an account of the performance of Grade 12 pupils and recommending strategies for improving results.
Mohlala, Mpho (D3.4c)
Challenging common misconceptions regarding outcomes-based assessment strategies
The successful implementation and operation of Outcomes-Based Assessment (OBA) strategies depends, amongst other things, on common understanding of concepts related to such strategies. However, a reflection on our assessment training experiences point to common misconceptions by various stakeholders, and the misconceptions threaten to undermine the success of the new education system. It is therefore imperative to provide clear explanations about these misconceptions so that educational change initiatives may proceed smoothly. Furthermore, the clarifications will undoubtedly help practitioners, in particular, to handle assessment issues in the classroom with relative ease and confidence. Following are some of the common misconceptions: the notion that Continuous Assessment (CASS) consists of highly frequent formal assessment tasks; teaching and assessment are separate entities, formative assessment is good and summative assessment is bad; Outcomes-Based Assessment/education (OBA/E) means the lowering of standards; overall learner performance against fixed / "objective" standards does not change; learner-centered teaching and lots of group work is exactly the same as Outcome-Based Education (OBE); assessment consists of ticking off outcomes as the teacher deals with them; "good" teachers have always been practising OBA/E.
Moodie, Peter (D2.2b)
A curriculum evaluation: Primary-school teachers' reactions to innovation in outcomes-based assessment, and their adaptations to policy
This evaluation explores the possibilities for introducing the intended OBE curriculum in Natural Sciences in the light of its dependence on extensive innovations in assessment. The project at present focuses on Grades 5, 6 and 7.
One main aim is to describe teachers interpretations of OB Assessment as they encounter it in a pilot programme-. The assumption here is that teachers never implement policy: they adapt policy in terms of their own conceptions of teaching and in terms of reducing administrative and management conflicts.
The other aim is to investigate the extent to which children can use process skills and deal with unfamiliar problems, because there is a lack of information available for writing levels statements of outcomes which teachers can use to decide whether children are "getting better at the outcomes".
Through interviews and frequent interaction in classrooms we are building up a picture of principals' und teachers' concerns. We are reaching the sensitive stage where schools will issue reports to parents on the children's process in science outcomes. We are finding that the ways in which teachers mediate tasks is crucial to assessing children's abilities. We are finding that children cope better than their teachers expected with some tasks, and we arc finding that the children cope much worse with other tasks which we were taking for granted.
Moore, Bob (D3.4a)
Administering experiential learning programs
An exploration of the foundations of experiential learning and its importance for professional education provides for an program designed to integrate university level learning and experience. Differentiating between "experiential education" and "training," a model is presented that creates a hierarchial structure integrating experience and learning from entry level through mastery.
Moore, Bob (D4.5a)
Assessing learning: the Capstone Course
Following a professional curriculum integrating cognitive and experiential learning, this final course provides for formal assessment at the end of the students' course of study. The Capstone Course is designed to assess cognitive, affective, and psychomotor learning and to do so in a student-centered and student-directed manner that requires the command of analysis and synthesis of knowledge and skills. Expectations in the course require students and faculty to foster interdisciplinary partnerships and cultivate business and industry alliances.
Moore, Bob (D2.3)
Curriculum planning
The presentation is based on the actual curriculum development plan in the Department of Communications at Elizabethtown (PA) College (USA). A sequential process, the plan includes development of definitions, mission statements, curriculum model, integration of philosophy to actual goals and objective, operationalizing goals and objective through courses, and developing an outcomes assessment plan.
Nel, B R (D2.5c)
Quality teaching, quality assessment, quality learning. Which one comes first?
This paper will argue that one cannot discuss quality assessment and quality learning without discussing quality teaching. It will argue further that quality teaching ought to be the first step in the process that culminates in quality learning.
At the same time it will attempt to unpack what is meant by quality teaching, as well as call for the need for training in the scholarship of teaching, especially in higher education where there appears to be a worldwide (global) movement to establish structures that aim for excellence in teaching.
This will also argue that the evaluation of teaching, learning and assessment is a crucial process that must be in place in order to ensure the element of quality.
In conclusion it will state that quality learning is aided and directed, not through quality assessment, but also through quality teaching and evaluation, both essential skill requirements for the 21st Century in education and training.
Nku, Samuel (D3.5c)
Explorations in local government career pathing using assessment techniques
VISION, POLICY AND IMPLEMENTATION: AN EVALUATION OF THE SCIENCE EDUCATION PROJECT
by Isaac Ntshoe and Charles Potter
Vista University and the University of the Witwatersrand, Johannesburg
In previous evaluations of projects in engineering and science education, Potter (1991) and Potter and Moodie (1992) used an evaluation model which focused on a number of levels. The first was the level of vision and intentions of the stakeholders involved in the project, the second the level of curriculum conceptualisation, policy and planning, and the third the level of implementation, or action.
In this paper, the authors report on an evaluation of the work of the Science Education Project (SEP) using a similar transactional framework, based on Decker Walker's (1972) conception of the curriculum as a curriculum as a dynamic between the vision of those involved in the curriculum development process and the deliberation which occurs around the development (usually centring around a notion of materials-in-use), Stenhouse's (1975) distinction between curriculum as intention and curriculum as action, and Hall et al's (1975) conceptualisation of curriculum implementation in terms of "levels of use".
The evaluation focuses on SEP's work in four regions of South Africa from 1980 to 1995. The project has subsequently ceased operating in these regions due to lack of funding. As the data cover the period leading up to the cessation of the project's work, it has been possible to draw conclusions concerning the value of the project's work in relation to policy issues in science education in South Africa, as well as some of the reasons why institutionalisation of the project's approach did not take place. The evaluation is thus relevant both to the development and implementation of other curriculum development projects in post-apartheid South Africa, as well as the development of science education under the new educational dispensation.
Olivier, Cas (D2.6)
How to educate and train Outcomes-based
Re-engineering learning towards the outcomes-based method is a major attempt to build the country into becoming an international role-player. The shift to outcomes learning versus content- and competency based learning, is analogous towards the total quality movement in business and manufacturing. Outcomes-based learning reflects the notion that the best way to get where you want to be is to first determine what you want to achieve. Once the end goal (product/outcome) has been determined. strategies, techniques and other ways and means will be put into place to achieve the goal.
The outcomes-based learning approach intends to focus equally on mastering of knowledge, skills, processes of learning as well as the achievement of the final outcome/result/product/output. In this sense the process of achieving outcomes through learning can be directly related to the way in which outcomes are achieved in the world of work.
Potter, Charles (D3.2c)
Evaluation, Knowledge and Human Interests
Being an applied field, evaluation has traditionally drawn its methodologies from other fields of social science research, and is essentially an applied, contextually based and holistic field of research, concerned with social development. With respect to the evaluation of social programmes, the designs used by evaluators are based in three broad traditions, namely, positivist, interpretive or critical-emancipatory. Each of these traditions reflects different assumptions concerning science and knowledge, and the role of the evaluator in relation to participants in the programme.
In the paper, the development of these different traditions in evaluation is traced, in terms of an historical account of the development of evaluation as it has evolved from its origins in the testing movement. Contextual issues influencing the development of evaluation are described, and related to the characteristics of evaluation models developed at different stages in the development of evaluation as a field. The paper concludes with comment on current trends in evaluation, and suggestions as to emerging and future trends in evaluation as a field of inquiry.
Samuels, Joe (D4.3)
The Principles and purposes of SAQA
Scheiber, Jackie (D1.1b)
Continuous Assessment, and what it means for them in the classroom
Continuous Assessment, and what it means for them in the classroom, is a concept that is of concern to many teachers. This is a workshop on Assessment for Senior Phase teachers (Grade 7, 8 and 9) that was developed under the auspices of the Ikaheng project, a joint venture between the Gauteng Education Department and AMESA (Association of Maths Educators in South Africa) to try to address this need.
In this workshop we look at how to assess group work and the pupils participation in the group, self assessment, peer assessment, assessment through classroom observation, cumulative checklists and student work portfolios.
Although this workshop was developed for maths teachers, only the first activity is specifically maths related. All the other issues discussed are relevant to teachers of all the other school subjects.
Shaw, Fred (D4.4)
Quality Assessment, Quality Learning: Standards
The author looks at the advantages and disadvantages of different approaches in assessment and learning. The author tries to reconcile the different principles with one another and attempts to draw conclusions from the current theoretical and practical situation in education and training, while taking into account the desired outcomes in the field.
Shingles, Christine (D 1.2b)
Workshop: Assessment for foundation phase teachers
The presenter will facilitate a workshop for Foundation Phase teachers [Grade 1 - Grade 3] in the basic a) principles, b) principles of procedure, and c) strategies for accountable assessment of learning outcomes to meet the requirements of Curriculum 2005. The focus of this workshop will be strictly practical and 'hands-on', involving group work and sharing. The
theoretical component of the workshop will consist of a brief overview of the principles of assessment for early childhood development [ECD] and how these principles relate to the departmental prescriptions circulated to schools. An attempt will be made to look at the practical implications of the various functions of assessment in the Foundation Phase and how these culminate in the end-of-term report.
Steenkamp, Karen (D4.2c)
Implications of Outcomes Based Assessment for a transforming workplace
Twalo, Thembinkosi (D3.3c)
Process writing in a multilingual setting
After an analysis of the education system in South Africa and the kind of students it produces, I propose in this paper that one probable solution to the unsatisfactory academic writing produced by the students would be a continuous and developmental form of writing. In this kind of writing, the learner and the ducator communicate and address the learners' problems through process writing and quality assessment. This is based on the assumption that writing is not an event but a process. I used Rhodes University Linguistics I students as subjects for my research and found out that students (mother tongue and non-mother tongue English speakers) have, for various reasons, serious problems with academic discourse. As a result, their essays fell below the standard set for academic essays. This paper argues that quality feedback from educators can greatly improve students' understanding of academic discourse. In the research students wrote two drafts of essays using feedback given to them on the first draft to write a second draft. The results showed general improvement and understanding of academic discourse.
Van der Vyver, John (D3.4b)
The judicious and effective use of rating scales to promote quality learning in outcomes-based courses
Soweto College of Education Currently seconded to GDE - Teacher Colleges Unit
The shift from subject-based to outcomes-based curricula in general, further and higher education in South Africa has necessitated an increased focus on criterion-referenced assessment instruments and strategies. This shift has been accompanied by the creation of a National Qualification Framework that requires national performance standards on stated outcomes. The description of such standards will of necessity, in the short term, have to accommodate the disparities in the various sectors with regard to facilities, resources and educational experience.
A number of projects have been initiated at a variety of institutions to define and describe outcomes for a range of, amongst others, teacher education courses and to attempt a description of acceptable performance levels for such outcomes in terms of appropriate criteria, range statements and performance indicators.
This paper reports on three projects currently being undertaken at a college of education, a technikon and a university in which rating scales have been developed for a range of outcomes in order to: describe performance requirements with an acceptable degree of specificity acquaint learners with respect to their current status with respect to the outcomesprovide learners with appropriate feedback to increase their level of proficiency with respect to the outcomes determine provisional standards of performance for such outcomes assist institutions to justify and/or defend learner proficiency in terms of a range of practitioner, learner and institutional variables.
The paper provides a provisional set of findings that impact on effective and efficient learning at the institution. It also indicates a possible way forward for institutions to address context variables in order to increase the quality of learning and to promote a measure of consensus around appropriate and possible national standards.
Van Koller, J F (D2.7b)
Objective assessment in social disciplines at a technikon at the end of a millennium
This paper reports on an investigation to determine to what extent objective questions were still used in social disciplines at Technikon A S. The aim was to establish to what extent developments such as the following affected the use of objective questions in social disciplines at Technikon S A.
The introduction of the outcomes-based approach in teaching by Department of Education. This approach places the emphasis more on skills than on the mere reproduction of learned facts.
A global trend towards the exploration and implementation of alternative and more authentic forms of assessment. Even the main proponents of objecttive assessment, such as the US and Japan, have also joined this trend.
The knowledge explosion of the 1990s (information systems, technology and so forth).
The author used a questionaire to test the extent to which objective questions were still used by Technikon S A staff and arranged a few focus group interviews to determine whether the attitude of staff members has changed regarding the use of objective questions in the past few years.
