Assessment in a Transforming Higher Learning Institution : A Case for Open-Book Examinations

The society continues to ask pertinent questions to universities in a rapidly changing world. The current era is one of knowledge and innovation; the advent of technology and global advancement require critical students who will be able to transform society. Arguably, suitable assessment in higher education will produce the citizens that every country requires. This qualitative study explored the use of open-book examinations in one higher education institution in the Eastern Cape area, South Africa. Four lecturers and thirty students in a school management program were observed and interviewed. In an attempt to build students who are critical and inquiry driven, the lecturers had a program that used course outcomes to lead the examination’s questions. The results illustrate that although there were many pockets of success, the students are challenged by the novel ways of assessment. They were still beholden to the traditional closed-book examinations, and it was quite an effort to explore a paradigm shift. Some of the lecturers also argue the tension that occurred when new forms of assessment were introduced. Despite these hindrances, it is clear that the changing university needs various forms of assessment that would cultivate inquiry-driven students who will be able to be critical when reading texts. The future requires citizens who are innovative and would be able to be part of a creative workforce. Soon, some universities will be irrelevant and obsolete if the emphasis is on certification rather than knowledge cultivation.


Introduction: Assessment and Critical Outcomes
Critical thinking is often cited as one of the necessary non-negotiables in today's higher education institutions.The rapidly growing technological systems in the world call for individuals who will be ready for the 21 st century challenges where technology and other forms of learning would be supreme.Today's higher education student also needs to be selfregulated.Ackerman and Leiser (2013) explain that self-regulated learning (SRL) is an active, constructive process whereby students regulate the effort they invest in learning so as to achieve self-set learning goals.Self-regulation is part of the paradigm shift needed to improve student performance.Learning styles such as heutagogy and transformational pedagogy needs students who will assume a more active role in the learning process.This article's focus is on the implications of the use of open-book examinations as a form of assessment in a higher education institution.It is an empirical case study conducted at one higher education institution.
Open-book examinations need the maximization of student creativity.Makala (2011) argues that open-book examinations evaluate the creative and critical facilities of students.When students are given an open-book examination to write, the assessors want to establish how they apply knowledge and not to merely regurgitate the text.Chaudhary and Dey (2013) also contend that the traditional assessment practices have changed to meet the needs of the contemporary society."Content-based testing has shifted to performance-based assessment.Assessment is no longer used for grading and certification, rather it has linked with learning and skill development of students" (Chaudhary & Dey 2013, p. 207).
In an age when professors complain about cheating of students in examinations, it might be best to consider critical open-book examinations than closed-book examinations.Some authors have highlighted the challenges in both kinds of examinations.Li (2013) points out that in open-book examinations students might use textbooks to search for pre-built answers "rather than synthesising concepts from class".Li also points out that tutors setting exams need to write complex or scenario-specific questions.Others though have emphasized the role of open-book examinations.Liska and Simonson (2005) assert that open-book examinations ensure that instructors set questions that require interpretation, analysis and critical thinking.The students also tend to be more relaxed than when they write closed-book examinations.Jalal, Fadhil and Hasini (2014) argue that open-book examination is not always positive because it gives a false alarm and hope to the students as they tend to assume that all questions can be solved by referring to the text book during the examination time.Moreover, Jalal et al. (2014) cite a survey conducted in Singapore which established that almost twothirds of the students preferred open-book examinations compared to closed-book examinations "with intention to have less time for preparation, require less memorizations on the subject matter and left more room for logical thinking".Kuhs, Johnson, Agruso and Monrad (2001, p.1) point out that "flash back thirty years ago, one would see teachers assessing student learning utilizing end of the chapter exercises, worksheets, tests and the occasional project".One has to concede that some of these forms of assessment have pervaded until the present.Kuhs et al. (2001) also contend that in today's classrooms, students keep journals, make presentations, work in groups, assemble portfolios of their work and have constant individual conversations with their teachers to demonstrate what they know and are able to do.Whilst the multiple choice items on standardized tests are still used, the shift currently, is towards the use of more open-ended assessment.The employees in all aspects of work need to be able to use their critical analysis when looking at and solving problems.Mothata, Van Niekerk and Mays (2003, p.84) cite Mays who captures what many speakers in one education conference (in the year 2000), highlighted as important qualities of effective and relevant assessment practices.Among these were that assessment should be: • outcomes-oriented: assessment should measure the significant learning that students should actually be able to demonstrate; • validity: the assessment procedures should assess what they are designed to assess; • comprehensiveness: assessment should cover the content, contexts and outcomes adequately; • authenticity: assessment should require demonstration of learning in congruent real-life contexts.All these are crucial elements that are explored in various contexts in the ensuing discussion.In this article the focus is on the use and relevance of open-book examinations at a higher education institution.The main question posed was: What impact will open-book examinations have on candidates registered in a school leadership university program?
The sub question is: How do the open-book examinations impact on academic standards?Boud and Falchikov (2007, p.3) have edited a collection of chapters that focus on the changing nature of assessment at universities.These authors argue:

Reinventing Assessment in Higher Education
Assessment affects people's lives.The future directions and careers of students depend on it.There are risks involved in changing assessment without considering the consequences.This has meant that there has been very slow movement in the development of new assessment ideas and changes in practice.We face a system of assessment that has been subject to slow incremental change, to compromise and to inertia.We are afraid to change the system because of the risks, but we also avoid looking at it because doing so might entail major effort.
Traditional teachers are afraid to take chances by risking with new assessment practices.I have observed how some colleagues have looked at open-book examinations with disdain, not as credible as real traditional examinations.The main mistake done by many teachers is to teach and forget about assessment.Yet effective teaching will be linked to student achievement and assessment.Boud (2007) argues that assessment is a process where various tasks will be designed to accomplish a number of goals with respect to teaching and learning.The challenge with the traditional forms of assessment is that they tend to view the students as passive blank slates who cannot engage with the content.Yet effective assessment will ensure that the student becomes an active learner.Race, Brown and Smith (2005) mention open-book examinations as one possibility of assessing students while making use of their active learning.However, these writers suggest that examiners should approach these examinations with caution.They list five suggestions for examiners to consider when designing open-book examinations.These are: i. Deciding whether to prescribe the books students may employ; ii.Considering the compilation of a source collection for the particular examination; iii.Setting questions that require students to do things with the information available to them; iv.Making the questions particularly clear and straightforward to understand; and v. Planning for short answers.Race et al. maintain that these suggestions ensure that the students maximize the advantages of open-book examinations.These authors also stress the importance of various aspects for effective assessment and these include, student peer assessment, students formulating criteria of assessment, self-assessment and open note examinations.
In South Africa, central to assessment objectives was the need for assessment to provide indications of the students' achievement in the most effective and efficient manner and ensure that students integrate and apply knowledge and skills (DoE, 2002).This DoE publication also highlights that the principal purpose of assessing students should be to enhance individual growth and development, to monitor the progress of students and to facilitate their learning.This new approach to assessment is different from the assessment employed in the past, under apartheid education.Vandeyar and Killen (2003, p.122) refer to these as pre-(outcomes based education) OBE schools and they state, "Prior to the introduction of OBE, most South African schools adopted an approach to learning and assessment that placed a strong emphasis on the accumulation of isolated facts and skills".
The purpose of assessment that utilizes OBE principles differs, as pointed out above.Assessment is currently meant to put an emphasis on high quality and effectiveness.The DoE (2003) posits that an OBE-based framework uses assessment methods that are able to accommodate divergent contextual factors.Moreover, the DoE states that assessment should help students make judgements about their own performance, set goals for progress and provide further learning.Assessment has also become more transparent with the student able to track own progress.The current assessment in many higher education institutions reflects efforts to ensure validity and reliability by using (among others), continuous assessment.Vandeyar and Killen (2003) argue in the pre-OBE system, reliability of assessment was rarely considered because much of the testing was done "once off"."Validity was not considered beyond content relevance and representativeness.Fairness was always lacking, because most of the examinations were "once off" and for many students, these were in a language other than their home language" (Vandeyar & Killen, 2003, p.123).
The focus in this article is more on the open-book examination as a form of summative assessment."Summative assessment gives an overall picture of students' progress at a given time, for example, at the end of a term or year, or on transfer to another school" (DoE, 2002).Sieborger and McIntosh (2002) also define summative assessment as a form of assessment which is used to report to others about the achievements of a student and unlike formative assessment which points to the future, it points to past performance.

The Open-Book Assessment Experience
Changes in assessment have been recognized as an important means of achieving curriculum change (Sieborger & McIntosh, 2002).These writers also add that transforming assessment does not require us to dispense with the current approaches but rather to make more reflective and relevant use of those that exist.A number of higher education institutions and some schools have experimented with the open-book examinations.In this type of examination candidates are allowed to bring their books and references into the examination room.Pritchard (1973) points out that although it is useful to have information stored in memory, in many situations later in life candidates who have been tested are more likely to need to know where to find information.The open-book examination requires candidates not only to find the answer, but also required to evaluate it and then apply it.This needs higher-order thinking.Candidates ask themselves certain questions when they attempt open-book examinations.Some of the most vital questions are: Where do I find these facts?How do they relate to my experience?
Open-book examinations have attracted much interest from supporters and skeptics alike.Educational researchers, lecturers, students as well as policymakers have all looked at this form of assessment as progressive and necessary.However, some skeptics maintain that it may favor those who can borrow the greatest number of books.Some people also argue that where private notes are permitted the risk arises of finding some answers have been copied from lecture notes or from assignments written during the course of the study (Pritchard, 1973).When it comes to the use of lecture notes, Pritchard (1973) argues that students merely copy directly from these.Furthermore, students are likely to bring in essays that might fit some of the questions in the paper.It is for these and other similar reasons that care should be taken when setting the open-book examination questions and when marking.The latter is important when one considers the validity of the examinations.
However, some lecturers have observed in open-book examinations a strong incentive for students to study for the ability to use knowledge rather than for the ability to simply remember it (Ebel & Frisbie, 1991).Furthermore, they argue that such examinations also encourage lecturers to avoid recall type test questions in favour of interpretation and application types.The open-book examination finds much support in certain sectors of academia.Like Pritchard above, some writers have stressed the disadvantages of open-book examinations.Kallish (1958) points out that some disadvantages of the open-book examinations are: • study efforts may be reduced; • efforts to learn sufficiently to achieve full understanding may be discouraged; • note passing and copying from other students are less obvious; • more superficial knowledge is encouraged.However, Race et al. (2005) contend that open-book examinations have the following advantages: • Less stress on memory; • Can be high on validity; and • Slower writers can be helped if this is coupled with time flexibility.Williams and Wong (2007) write about (and contrast) the open and closed-book examinations models.They state that the open-book examinations represent a serious attempt to engage students rather than alienate them and that academic dishonesty practice is less because of the structure of the examinations.Furthermore, the students relate to the task that lies before them as they see its point."By contrast, the closed-book, invigilated examination encourages a strategy of 'cramming' the night before and 'data dumping' on the day with little knowledge retention thereafter (Williams & Wong, 2007, p.1079).However, Agarwal, Karpicke, Kang, Roedeger and McDermott (2007) state that it might be argued that closed-book examinations have a potential of enhancing learning more than open-book examinations.They also aver that one theory that supports closed-book examinations holds that tests require a more challenging retrieval and produce greater benefits for long-term retention.Moreover, the conditions that require more difficult and challenging processing may slow initial learning but ultimately enhance long-term retention (Agarwal et al., 2007).There are many educators who use effective closed-book examinations that transcend mere testing of memory or regurgitation of knowledge.Good quality closed-book examinations challenge the intellect and ensure that the candidates apply the knowledge gained in new situations.Some research has shown that there is no significant difference in examination achievement if the question paper is drawn carefully (Vyas & Vyas, 2009).However, in most cases, the temptation and the snare are still there for examiners not to move beyond recall items.

Research Methodology
This study was a follow-up of a study conducted in 2004 where the National Professional Diploma in Education (NPDE) students from one university were studied.In this study a qualitative investigation was conducted among the Advanced Certificate in Education -School Management and Leadership (ACE-SML) students.Purposive sampling was employed in selecting 30 students in one South African university.Sometimes referred to as judgemental or theoretical sampling, this method is based on the judgement of a researcher regarding participants that are typical or representative of the phenomenon being studied (Brink, 2000).In choosing purposive sampling the researchers usually use their previous knowledge of a population.Researchers assume they can use their knowledge of the population to judge whether or not a particular sample will be representative (Fraenkel & Wallen, 2000).
Fifteen of the participants had just completed their first year of study in the two year part-time program meant for school leaders.The other fifteen had just completed their second year in the program.All had written open-book examinations in their previous examination.Apart from these student participants, four lecturers in the program were also part of the sample.The lecturers were also selected from a population of 10 academics teaching in the program.Unstructured interviews were conducted with all the participants.The majority of these interview sessions took a minimum of one hour for each participant.Unstructured interviews are freer flowing with structure limited by the focus of the research.Unstructured interviews are conducted more like a normal conversation but with a purpose (Brink, 2000).The examination results of the participants were also analysed, while the lecturers showed their material such as study guides as they explained their assessment strategies.The questions mainly focused upon open-book examinations as form of assessment in the program.
The nature of the program is itself very revolutionary in that it utilizes the experiences of the candidates and teaching is guided by cases studies.Furthermore, the materials are very interactive and expect all the candidates to be actively involved in class discussions.The programs used mixed-mode delivery which includes tutorial letters, contact sessions and some school visits.The four lecturers in the sample had been in the program for more than four years and were responsible for a number of modules that included Mentoring, Leading People, Assessment, as well as Schools and External Relations.All the participants were each interviewed separately and the interview notes were recorded during the course of each interview.

The findings
The four lecturers concurred that the open-book examination has many advantages, especially for distance education students.All of them but one were never personally exposed to open-book examinations as students and had to be trained to use these.They also admitted that not all of them as lecturers supported the idea of open-book examinations and there were colleagues who were still using closed-book examinations.Among the potential benefits highlighted by the ACE-SML lecturers were the following: • That the examination mode will "make the students critical and sharper"; • That they will be able to use their experiences in engaging with the text; and • It would minimize memory learning because there would be no facts to recall; Although the lecturers highlighted these benefits, they also pointed out that there were a number of challenges and the biggest was that students would merely reproduce huge portions from texts without being critical.
The ACE-SML candidates have the opportunity of using all the material that they are familiar with during the examination period.They use materials they have been using during the course of the year.However, the lecturers also cited a number of challenges regarding this form of assessment.They reiterated that many students fall into the trap of "writing what is in the study guide and textbooks verbatim".
A number of students both from first and second year the open-book examinations had some challenges.The temptation for them was to repeat what was in the text without any critical reflection.

Discussion
Many teacher-learners in the study, by their own admission, did not appear ready for open-book examinations for they expected one-answer questions that promote rote learning.They did not expect to take the content of their modules beyond the theoretical level.The concept of them having to apply their knowledge to other situations appears challenging and a few use their mature age as a scapegoat for being uncritical.In the examinations many stated that they gave the examples from the study material even when the questions stated clearly that they need to give their own original examples.There is a strong need to untrain the students from what they have learnt in the past.
The study also showed that there is a strong need to deliberately create conscientious educators who will lead the transformation of education.The education system will experience serious challenges if the educators in a teacherupgrading program are not able to apply new knowledge in a number of new situations.Educators are the people who are required to develop the new kind of learner envisaged by the current system of education in South Africa.The findings of this study are discussed under three themes: • Autonomy and inquiry for learning; • Linking learning outcomes to assessment; and • Promoting quality and academic integrity.

Autonomy and inquiry for learning
For many students it was difficult to adopt a culture that they were not used to.Almost all of them reiterated that they came from a culture of traditional examinations which required them to cram knowledge or facts.The idea of a changing university or changing assessment strategies were not well understood by the candidates.The lecturers themselves concurred that they were trying to internalize the novel ways adopted by the ACE-SML program.They also stated that they were trying some of the strategies learnt in the program's assessment strategies including the open-book examination.They found open-book examination crucial for the practicing school leaders.The examinations did not expect the candidates to regurgitate knowledge but to apply their experiences as they engage with literature.The latter is akin to what Ovens (2011, p.41) refers to as action inquiry to learning.This author says about action inquiry: It is about improving practice, which is what we think learning is, fundamentally.It involves looking carefully at the experience of learning, thinking reflectively, using ideas and knowledge to think better, to practice more wisely and much discussion.We ask students to do action inquiry into their practice as learners as the main way in which they learn how to learn at university.
From the interviews it was clear that students in the program did not have the grasp of the assessment objectives as they talked about how daunting it was to bring sources into the examination room.They also talked about the unpredictability of their answers because "the questions tended to be too open-ended".The students were "struggling learning how to learn".Almost all had many years in school management but preferred to write closed ideas from prescribed texts.They clearly wanted standardized answers that would leave no room for creative or critical engagement.The lecturers also talked about previous and current concerns in using open-book examinations.Among their questions are: • Won't the open-book examinations drop standards?
• How critical will the students be in answering the questions?
• Will they be able to avoid reproducing the texts?
• How easy would it be to shed the culture of the traditional assessment?
In addition to these questions, the lecturers also stated that they found it challenging to teach critical analysis of the text during formative assessment during the year.Even when I observed the examination scripts, it was generally difficult to find the actual student experiences as school managers.Student experience is used to imply the value of studentcentered view of teaching which encompasses more than the transmission of knowledge (Ovens et al. 2011).
The active nature of the ACE-SML requires the school leaders in the program to learn through constructivist approaches hence the form of assessment is crucial for the program.Critical reflection, active inquiry and sharing of contexts are part of the program.This is all about learning to think and thinking to learning which Kirkwood (2007, p.169) opines: An important way in which this is manifested is through prompting participants to put new knowledge to the test in the classroom, which requires them to adopt systematic strategies: for example, identifying appropriate sources of feedback and gathering and analyzing evidence…The learning outcomes are indicative of the broad range and nature of module activities, which require authentic productions and collaboration with peers, both important aspects of effective participation within a community of practice.
The open-book examinations seek to achieve these aspects.However, both groups the first years and the second years demonstrated their reservations about open-book examinations.Only four candidates in the second year group were at ease with the open-book examinations, whilst only three stated no objections to these examinations in the first year group.

Linking learning outcomes to assessment
The open-book assessment is linked to the general purposes and outcomes of the ACE-SML program.The program is based on various learning principles which underpin the teaching and assessment of the program.The following are the learning principles: • Directed and self-directed learning in teams and clusters; • Site-based learning (dependent on content); • Variety of learning strategies; • Parallel use throughout of individual and group contexts of learning; • Problem-focused deliberation; • Critical reflection on group processes; • Critical reflection and reporting on personal growth; and • Research and experimentation.
(DoE, 2006) All these learning principles seek to ensure that the school leaders will be able to critically engage and be self-reflective practitioners are developed.All these are relevant to open-book assessment.Race, Brown and Smith (2005) argue about a fit for purpose assessment.They cite the work of John Biggs who wrote about constructive alignment.Biggs describes constructive alignment as making sure that intended learning outcomes connect with the students' achievement.Furthermore, Race et al. (2005) point out that appropriate assessment criteria need to be used to assess the students' achievement of the outcomes.These should also allow "students to receive direct and useful feedback on the extent to which they have demonstrated their achievement of the outcomes" (Race et al. 2005).
When lecturers use constructive alignment, they ensure that their students make the right decisions when it comes to assessment processes.The ACE-SML program is meant to solve complex management problems in South African schools today.The candidates in the program are taught to explore various learning experiences that would make them understand their schools better.The open-book assessment is an attempt to see the critical acumen of the students as school managers.Dochy et al. (2007) also contends that one of the important aspects of assessment is the belief that it has a crucial impact on teaching and learning.These authors stress the idea that the alignment must enhance the power of assessment as a stimulus for learning.Aligning learning, instruction and assessment has led to changing insights in assessment."The traditional view that the assessment of students' achievement is separate from instruction and comes only at the end of the learning process is no longer tenable" (Dochy et al. 2007, p. 87).
Open-book examinations are one of the ways used to engender quality students in an age of technology and development.Barnett (2007) writes about how complex assessment has become in higher education institutions because these institutions find themselves in a world of uncertainty and complexity."Assessment practices in higher education are not always yielding the educational potential that they might" (Barnett 2007, p.29).Yet a number of authors underscore the need for authentic assessment (Schwartz & Webb, 2002;Murphy, 2006;Barnett, 2007).Schwartz and Webb (2002) refer to authentic assessment as integrated assessment that would be superb for complex situations.Furthermore, Barnett (2007) argues that the idea of authenticity embraces a prospect that enables a student to genuinely clasp the potential values of assessment.
As highlighted above, there is much changing in the present society and so do universities.In fact, Barnett (2000) has elucidated on how universities die and then evolve.Yet there are still numerous institutions that are stuck to traditional forms of assessment.This does not help in a changing society where universities (to use Barnett's words), have become "supercomplex".Bryan and Clegg (2006, p.3) also postulate that modern society is demanding and complex although many forms of assessment are very basic in their nature.These authors capture this well: It is time that we recognized the changing nature of society and acknowledged that quality is a more complex concept than traditional assessment criteria suggest -quality cannot be reduced to a set of easily qualified learning outcomes.The widespread use of the phrase 'anticipated learning outcomes' is in part recognition of individualized, personal perceptions and reactions to learning situations.
Assessment has many innovative faces today.Higher education should not avoid this.In an age of technology, lecturers can avoid cheating by asking valuable questions whilst the students have materials at their disposal.Today's students are exposed to much technology and they should be allowed to use this in a responsible fashion even in examination rooms when necessary.With the right assessment items, examination cheating can be an aspect of the past.

Promoting quality and academic integrity
The lecturers in the ACE-SML program were trying to produce better quality in the students by introducing the open-book examinations.They acknowledged that they were teaching experienced school leaders and did not want to pressurize them through assessment.The paradox though was that the students were stressed by assessment strategies they were not used to.They felt so uncomfortable to oppose any of the ideas in their texts although they know exactly what worked and what did not work in their own schools.The lecturers explained though that they did not train their students well for the open-book examinations strategies.Students went to the examinations not sure of the best approaches.Sadler (2002) highlights the importance of setting out a number of criteria for assessment to achieve quality.Sadler mentions the following criteria; relevance, comprehensiveness, coherence, logical reasoning and presentation.It is crucial for students to know exactly how they would be judged.
However , Sadler, (2002) adds that telling the students about assessment requirements can be abstract and lecturers might need a more detailed description.Sadler gives a practical example from his experience when he states, "I now try to show them as well.I realize that the same pedagogical devices that I use with respect to the subject matter in the courses I teach make equally good sense with respect to communicating my expectations about the quality of students' work" (Sadler 2002, p.136).Boud (2007) also affirms that there is a need to reframe assessment so that it magnifies learning.Furthermore, Boud contends that assessment should enable students have the capacity to evaluate evidence, appraise situations, draw sound conclusions and act in accordance with this analysis."This is the idea that focuses on learning centrally -learning to form judgments -as well as on the act of forming judgment about learning" (Boud 2007, p.19).This will build the quality of the students.Again this shows the need for higher education to shift from traditional forms of assessment that stressed rote learning.Kvale (2007) argues that lifelong learning is a necessary condition for a successful changeover to a knowledge-based economy and society.This author adds by stating that traditional forms of assessment may be counter-productive to learning that fosters lifelong motivation for learning.

Conclusion
Assessment in higher education continues to add complexity in many universities.Quality and standards are usually judged through assessment methods used.However, this concept of quality has itself become relative in its use.Quality in one institution may not be in another.This study though, supported by literature has demonstrated that there is much need to revolutionize assessment in this knowledge base society.Society requires lifelong learners and open-book examination is one solution in a myriad of other choices.However, this study has also shown that for assessment to be successful, it needs to be communicated well to students, for them to see how it links not only to the outcomes but to their lives outside the institution.In an age of massification, the previously disadvantaged should also see the relevance of education through assessment methods.Assessment should never be about certification only, but also about knowledge and application.