Changes to assessment through the use of portfolios and digital diaries in teaching languages and reading

 

Dra. Isabella Leibrandt

German Lecturer

Universidad de Navarra

ileibrandt@unav.es

 

Abstract: In this paper, I provide a description of assessment through the use of portfolios. Firstly, I give a brief overview of the principles of formative and reflective, as opposed to summative, evaluation. In line with a metacognitive methodology, the portfolio is presented as a very worthwhile instrument for a self-evaluation process; examples are given from the areas of language learning and reading comprehension. The final part of the paper takes into account possible difficulties in the provision of adequate valuation of learners´ portfolios, and offers some useful evaluation criteria as a response. Some supplementary links to different Web sites are given so as to shed further light on the concept of the portfolio, as well as to provide further information and related material.

Key words: portfolio, formative evaluation, metacognition, self-evaluation, language learning, reading comprehension

Resumen: El artículo ofrece una descripción de diferentes aplicaciones de la evaluación con portfolios. En primer lugar se presentan brevemente los principios de la evaluación formativa o reflexiva en oposición a la sumativa. De acuerdo con la metodología metacognitiva el portfolio es considerado un valioso instrumento para fomentar el proceso de autoevaluación. Algunos ejemplos de la enseñanza de lenguas y de la comprensión de lectura avalan su utilidad. Finalmente también se toman en consideración posibles dificultades a la hora de evaluar los portfolios ofreciendo algunos criterios para su evaluación. Algunos enlaces ejemplares a páginas web relevantes complementan esta presentación añadiendo más información sobre los diferentes aspectos relacionados con el portfolio.

Palabras clave: portfolio, evaluación formativa, metacognición, autoevaluación, aprendizaje de idiomas, comprensión de lectura

 

 

 

1. A brief introduction to formative and reflective evaluation

 

In modern pedagogy, the methodology of evaluation, one of the most important areas of research, dedicates special attention to the how and why of evaluating learners during their whole experience of learning through various forms of assessment. A particular emphasis is placed on self-evaluation and the so-called formative or reflective forms of assessment in contrast to a single evaluation which experts consider worth a third of the whole assessment at the end of a term. In contrast to this extended assessment procedure, formative evaluation is expected to be a combination of different parts such as projects, tests, exams and other possible individual or collective contributions which are designed to show and reward the learner’s activity over the course of a whole semester or year. What are the principal pedagogical reasons for adopting this new and also more demanding form of evaluation?

Essentially, the main distinction between formative evaluation and the normal marking system consists in their different objectives. The former aims to achieve the continuous improvement of skills by obtaining data about the process of learning systematically. In the first place, formative evaluation, including also some self-evaluation, requires the learner to interpret and to take decisions regarding his development, performance and control of his whole learning plan. These points demonstrate that formative evaluation is not a concluding activity at the end of a process but a continuous necessity justified by the fact that it can contribute to the learning process which is to be corrected and improved during the ongoing process and development.

Without doubt, formative evaluation also contributes to innovations in teaching and learning practices in two important ways: in the first place, it seeks to help the learner to be more aware of his learning process and, on the other hand, also aids to change the image of the teacher from an examiner to a guiding coach who helps and accompanies the learner. In these terms, formative evaluation may be regarded as an irreplaceable process which offers an adequate solution to possibly problematic aspects in learning, due to specific difficulties with a topic or other constraints such as time organization, motivation, etc. After analysing and interpreting the collected data, the learner may be helped to overcome possible obstacles or fears and reach his objectives.

In order to understand better the meaning of the concept of ‘metacognition’,  Meichenbaum and Biemiller (1998: 21) offer the following definition which might help illuminate the cognitive function of this skill:

The term metacognition refers to the cognitions you have about your cognitions, your thinking about your thinking, your ideas about ideas. It refers to your thinking about yourself as a problem solver who is able to voluntarily plan, monitor, evaluate, and regulate your own cognitive processes. Metacognition reflects your ability to “read” your own state and to assess how that state will affect your present and future performance. In this self-assessment process, you bring to bear a good deal of knowledge about specific strategies, tasks, and cognitive processes, as well as knowledge about yourself as a learner.

 

Without doubt, this concept of evaluation radically changes the processes, methods and obejctives of the general institutional instruction which in most cases still follows the traditional examining system, but gives little importance to the process of learning because, in general, at its end point, nothing can be done any more to correct failures.

Due to these reasons, as teachers, we should differentiate between the benefits and drawbacks of each form of evaluation without excluding any of them: the formative one offers constant reflection on the process and the development of a course, whereas the summative one shows us whether and how we have obtained a certain result. In other words, marking is a fullstop and a finished process at a particular moment; reflective evaluation is an open process. Both complement each other as they encompass different types of performance, and therefore can together make evaluation as complete as possible.

Exploring this new formative concept in more detail, I would like to point out other related aspects such as learner autonomy, a major participation and responsibility by learning to take charge of their important role in self-assessment and not to rely only on the teacher. On the other hand, obtaining these important data and the individual reflections of the participants can be decisive for the teacher in the further selection of material, contents or a redefinition of some objectives which can be modified in time based on the information acquired. Another important aspect concerns the previous knowledge and other learning experiences in the learner’s biography, an important source for taking necessary steps in course planning. As well as showing our effort to address each learner as an individual, we get to know their starting point, previous experiences, difficulties, needs and interests. Due to the continuous self-reflective process and the interpretation of experiences and perceptions, this process is also known as ‘transformative learning’, and is regarded as indispensable for obtaining significative learning results: “The goal of transformative learning is to understand why we see the world the way we do and to shake off the constraints of the limiting perspectives we have carried with us into the learning experience.” (Palloff and Pratt 1999: 129)

These authors, following constructivistic principles and learner-centred procedures, attach great importance to the learner’s participation in the task of formative evaluation, considering it an integral part of the learning process in order to enhance a process of reflection on one’s own learning.  For this reason, Palloff and Pratt, provide an illustrative example by comparing the learner’s skill of self-assessment to the intellectual practice carried out continually by professionals in different fields who improve their efficiency through constant self-assessment and self-adjustment:

The evaluation of student assignment in a (...) course should not be the job of the instructor alone. Students should be asked to evaluate their own performance and to receive feedback from each other throughout the course. Developing skills in (...) self-assessment can be useful in the promotion of collaborative and transformative learning. (...) the receipt and use of feedback must be an ongoing, routine part of assessment. The reason for making feedback concurrent with performing is that this is the only way students can learn to self-assess continually and then self-adjust their intellectual perfomance, just as musicians, artists, athletes, and other performers continually self-assess and self-adjust. (Palloff and Pratt 1999: 148)

Furthermore, Palloff and Pratt refer to a crucial and desired outcome in realtion to self-assessment: to transmit a sense of responsibility for learning and the capacity of intellectual growth through self-observation. For the purpose of learning how to engage critically with our daily activities, both authors propose an extensive list of questions as a starting point for self-assessment:

What was most useful to me in my learning process? What was least useful? Did I achieve my learning objectives in this course? If yes, what did I  achieve? If no, what got in the way of achieving those objectives? What did I learn about my own learning process by taking this course?

How did I change as a learner through my involvement with this course? Do I feel that what I learned in and through this course will have application in other areas of my life? If so, where will I apply this knowledge? How well did I participate in this course? Am I satisfied with my level and quality of participation? Did I see myself as an active member of the group? Did I contribute adequately to collaborative assignments? How would I evaluate my performance in this class overall? (Palloff and Pratt 1999: 127-128)

Training in reflective evaluation is also an important step towards the investigation of teaching and learning, by making students co-responsible for the entire planning process, and at the same time, reflecting  a change in the evaluation culture. Needless to say, learners as well as teachers might require some introduction and time to get used to this tool. Without doubt, they have to familiarize themselves with this procedures and become accustomed to expressing their thoughts and feelings and learning to regard the portfolio or diary as a window which allows the teacher to look into the learners’ heads. In starting to use a portfolio as an assessment tool a number of questions may arise: “Where do I start? What software should I use? What strategies seem to work well?” In reference to the necessity of becoming used to this innovative procedure, Paloff and Pratt address the following comment to teachers: “We cannot encourage our students to engage in a transformative process if we are unwilling to do so ourselves.” (Palloff and Pratt 1999: 143)

For these reasons, Meichenbaum and Biemiller propose a wide range of questions to stimulate teachers to be more reflective with regard to their own teaching methods:

How do I view myself as an instructor? Do I see myself as an expert? Am I open to the views and opinions of others? How do I process those views when I encounter them?

How much do I feel I need to learn about teaching and about my subject matter?

How do I generally run a class? Do I rely on learn on lecture and discussion methods?

In the traditional classroom setting, do I empower students to pursue knowledge on their own? Do I routinely incorporate collaboration exercises and assignments into my courses?

How do discussions go in my courses? Are they dominated by a few? Are my classes truly interactive?  

How comfortable do I feel with the concept of promoting self-knowledge in learners? Do I honestly feel that this should be the work of someone other than a teacher, such as a counselor or therapist?

How comfortable am I when students disagree with my point of view? How would I feel if a student suggested that I read material they have discovered in their learning process?

Do I feel that I need to maintain control of the learning environment? How comfortable would I feel in giving over that control to the learners and being an equal participant?

How comfortable am I with receiving material from students that is not grammatically correct and well written but is nevertheless an expression of self?

How do I define learning? What do I hope to see as learning outcomes from an (online) class? (Meichenbaum and Biemiller 1998: 264-65)

Concluding this brief overview, we can state that the use of self-reflection strategies, by students as well as teachers, is a recent trend in constructivistic learning, inasmuch as personal growth through participation in a course and interaction with others is regarded as an important way pf personal development through continuous self-evaluation. Futhermore, becoming more reflective about learning offers an opportunity to connect the object of study with one’s personal life, and therefore bring about more positive and longlasting results through a form of learning for which the learner himself is largely responsible. Thus, self-reflective process is to be regarded as a decisive component of learning, in order to revise or interpret experience, as well as a way to guide further action. In relation to this brief introduction, the Web site ‘Managing Assessment Information’ gives more extended and useful information about techniques concerning how to monitor development and make informed decisions about teaching, learning and the evaluation of learning.

 

2. Assessment through portfolio or learning diaries in teaching languages

 

Experts argue that all learning processes are basically determined by the assessment types adopted. Therefore, they conclude that changes to assessment procedures are an important aspect which will also have a positive effect on the transformation of the learning process amounts to an expression of hope for improvements in the quality of learning and teaching.

Current research is therefore focused on self-evaluation through a portfolio or diary. The main reasons which support their use come from a constructivistic paradigm which values metacognition in relation to continous evaluation. Assessment by portfolio has gained widespread acceptance especially in the USA, which is shown by the considerable number of publications and Web sites various examples of which are given below. This dissemination of portfolio development also shows the key ideas about teaching and learning which underlie a new culture of learning.

For these reasons, the use of a learner’s portfolio facilitates reflection on his continuous learning process, his opinion and reflection or other things related to the assignment which shows the individual’s progress in different skills through the collection of commentaries. The principal objective is thereby to increase the capacity for self-reflection so that the student becomes more conscious of his responsibility by learning to organize his time or other factors that might influence the quality of the learning outcome.

Helen C. Barrett in ‘The Electronic Portfolio Development Process’ gives a useful introduction to different kinds of levels of digital portfolio software in teaching English. In order to highlight the self-reflective procedure, I would like pointing out just three simple questions she poses: What?, So what?, Now what? The first question constrains the learner to sum up the work he or she has done. Secondly, so what?, he must explain what he has learned from a specific task, project, text, etc. Finally, now what?, points at the future objectives and implications which might involve taking certain steps as to improve results. These types of self-reflection might be especially helpful when some chapters, topics or units are closed so that the students can revise and considerate their progress. This example shows that portfolios constitute an important innovative tendency in the assessment and guidance process of learning by giving importance to metacognitive skills. In any case, it seems important to underline that the portfolio does not have to be a unique collection of activities and tasks but it aims primarily to develop the capacity of metacognitive interpretation; in other words, the learner learns to observe his own learning in order to learn how he learns and thereby to draw important conclusions about further steps in learning. A collection of further documents related to portfolios is provided on the Web site: Managed Environments for Portfolio-based Reflective Learning.

For teaching English as well as for other languages, a very good introduction and presentation of how and why to use a portfolio is given in Kemp and Toperoff‘s GUIDELINES FOR PORTFOLIO ASSESSMENT IN TEACHING ENGLISH which along with key characteristics also provides some helpful ideas about how to introduce a portfolio into a class, exsamples of how to implement portfolio work in the classroom and other portfolio exsamples for different language skills: speaking and listening, reading and writing. As we can appreciate from this document, the portfolio is an assessment instrument which demonstrates the continous process of learning by giving the teacher major access to the individual progress of each student. This gives him the opportunity to get to know potential problems promptly, and therefore to re-direct the student if necessary. A portfolio can be considered basically as an alternative to the conventional assessment, offering a solution to the possible unease produced in many teachers who feel that a final exam does not provide a valid image of the learners’ real competence.

The concept of ‘language awareness’, that is, awareness of linguistic structures and functions, is very closely related to portfolio methodology. Therefore, language teaching focuses increasingly on a reflective interaction with language in the context of other languages. Consequently, learners of a second or third foreign language, who have already gained experience and strategies in language learning, are more aware of the process of linguistic acquisition, and might have developed a more profound language awareness and reflection. For this reason, ‘language awareness’ is used as a didactic concept which enhances multilinguism and serves as the basis for the European Language Portfolio and the ‘The Europass Language Passport’ with a template of the european levels. According to the criteria of the ELP, it has a documentary as well as a pedagogical function. Not only does it provide learners with an overview of their linguistical capabilities, but it also encourages them to engage more intensely in language learning and to be reflective. Among other purposes, the principal objective is to insìre and help learners to:

- reflect on their objectives, ways of learning and success in language learning
- plan their learning and learn autonomously

As this new language enhancement develpos, it may be said that, those skills which help learners to observe their learning process are increasingly in demand. They should learn to assess themselves and to take responsibility for their own language acquisition. Moreover, they should make use of the wide repertoire of strategies and techniques which have also begun to play a major role in teaching. Another useful document to introduce teachers to the portfolio philosophy is ‘The European Language Portfolio: a guide for teachers and teacher trainers’ by David Little and Radka Perclová, who designed this guide as a support for the use of the ELP in the classroom. They pay particular attention to important aspects such as: classroom activities that can be used to introduce work with the ELP, making time and space for the ELP in language teaching, understanding and using the common reference levels and descriptors, motivation, reflection and learning how to learn, approaches to assessment, as well as other challenges posed by the ELP.

By taking into consideration the portfolio’s usefulness for the reflective process, I would like to draw special attention to its function in relation to reading comprehension. In the teaching of literature, metacognition strategies also offer an important instrument for reflection and self-evaluation during the process of reading, which point, incidentally, to the importance of the reader’s activity in reader-oriented methodology. Self-observation during the reading process, its demonstration and exposition by taking into account textual constraints, the reader’s knowledge and strings, as well as his reaction during interaction with the text, correspond to the abilities to perceive, recognize, interpret and understand. Reading is regarded, therefore, as an act of knowing how to understand and interpret, how to establish a personal opinion, valuation and judgment. In order to reach these results adequately, it is necessary to learn certain strategies which allow the student to integrate his individual contribution, (knowledge, experience, feelings), so as to reach a rounded understanding, and thus, based on the text, develop a personal interpretation. In this sense, the receptor must perform a double task: evaluate the discursive components in order to reach, through personal reception, an adequate understanding and interpretation of a text. The restructuring of spontanous reading to with metaliterary knowledge may be regarded as the most challenging step, as the learner must learn some interpretation techniques. The more automatically these are produced, the faster the learner will formulate, reject and re-formulate his hypothesis and be in a better position to reach a final understanding.

Reading, in this sense, as an action directed by the reader consists of a series of cognitive activities (anticipation, identification, association, conclusion and valuation) which the learner must learn to carry out while he is reading. The central puprpose of these formative activities is metacognition, which will offer the reader means of self-observation during the reading process; while the receptor not only pays attention to the textual constraints, but also to his skills as a reader (abilities, strategies, knowledge). This cooperation between reader and text is reached in a definitve way when an interpretation encompasses both the intentionality of the text and personal valuation.

What are the characteristics of a competent reader? Here are some mentioned by Mendoza Fillola (1998: 175): to make logical co-relations, establish coherence, identify clues, orientations, macro-structures, to show metacognition skills of the reading activity, to use them to organize and identify different phases of reading, to execute those strategies which the text suggests, and conclude in a logically adequate statement  of comprehension and interpretation.

To see reading as an active process of construction of meaning from textual  stimuli is to regard it as an individual activity with a personal and intimate character, because it is the receptor who develops the process, who reacts to the  textual stimulus and who contributes with his knowledge depending on the requirements of a text. Therefore, reading is an cognitive activity which invovles association of ideas and evaluation.

Due to the complexity of a conscious and reflective reading process, experts agree that an indispensable condition for the teaching and learning of reading is the enhancement of metacognition skills during the reading activity, as this reflective knowledge serves as a guide before, during and after the process, helping the learner to organize and identify each phase of his reading, and thereby to learn to read more slowly, paying attention to important details.

Consequently, self-observation constitutes an important part of the reception process showing the reader how to appreciate different elements, to perceive his own necessary participative activity in order to establish meaning - in other words, to  learn how to manage the reader’s re-creative activity.

As reading is a complex system of deriving meaning from printed text that requires skill and knowledge, it not only involves the ability to decode (un)familiar words and the ability to read fluently, but also sufficient background information and vocabulary to foster reading comprehension, the development of appropriate active strategies to construct meaning and the development and maintenance of motivation to read. Furthermore, research in reading indicates that there are five essential and inter-related components of effective reading instruction, to enable learners to master skills in order to become proficient, successful readers. One of the areas where explicit and systematic instruction must occur is in reading comprehension strategies for which the Web site ‘Just read now’ offers a range of innovative and effective reading strategies, along with actual classroom examples and associated lesson plans, applicable across academic disciplines and learner levels, subdivided into the following four focus areas: discussion, active reading, vocabulary and organization strategies.  

The reasons why they ought to be applied in the teaching of reading are convincing: to make the reader autonomous and enable him to respond intelligently to texts of very different kinds to be self-refelctive with regard to his understanding, to establish relationships between what he is reading and his personal contribution, to query his knowledge and modify it, as well as to generalise – that is, to transfer what he has learned to other contexts.

A reading diary, in this context, not only serves the learners as an expression of personal reflection about the book but gives, more importantly, the teacher the opportunity to get to know individual ways of thinking and understanding a text. A reading diary offers the possibility of combining the expression of initial reading impressions with more formal text analysis, based on a reading-oriented theory which will encourage the reader’s activity in communication between the reader and the text. In this sense, the diary allows the teacher to see this intimate communication between the reader or learner and the text, making it possible at the same time to guide this process by giving some advice, commentaries or further questions in order to draw the learner’s attention to specific aspects which might be difficult to understand or simply overread in the first reading.

In this sense, the following quotation from the Web site Just read now underlines the idea of writing a reading diary: “In order for students to truly gain understanding while reading, they must strategically interact with the text. Strategic readers think about text, think with text, and think through text.”

In relation to the metacognition and self-directive strategies mentioned above a ‘Strategy Log’ might also be wirthwhile: a note-taking tool designed to help students identify the strategies they use while reading. One of the very useful active reading strategies recommended on this site is how to encourage students to understand the purpose of a response journal by giving them as a first form of help some ‘Reading Response Journal Prompts’:

Explain the functions of the response journal to students. Stress that the journal is personal—a place to express ideas, feelings, questions, and opinions. Point out that there are no "right answers" in response journals. Successful journals capture high-quality student-text interaction.

Provide a model journal for students. Make sure that this model includes observations, questions, critical judgments, opinions, and feelings. Explain that while all of these are appropriate, students should be able to distinguish opinion from observation and critical judgment from feelings.

Provide journal sheets or booklets with prompting questions that will help structure student responses. Encourage students to record as many observations as they can.

From time to time, organize the class into small groups and allow students to share their journal responses with their peers. Stress again the functions of the journal and the fact that there are no "right" or "wrong" answers.

 

3. How to evaluate a portfolio

 

The last aspect concerning the portfolio or diary is its evaluation. What criteria might be useful? Beside the originality and complexity of the learner’s thoughts in reference to his capacity for self-reflection about his learning progress (for example, concerning reading and writing skills), the following questions could be a useful guide both for the teacher as well as the learner in order to give more transparency to the evaluation criteria:

Are the objectives pursued in the portfolio well-defined?

Is learning progress made sufficiently documented?

Is learning progress more transparent and visible by using a portfolio?

Is the learner developing a growing capacity for metacognition through the learning process?

Does the learner show original thought or are ideas just copied from other sources?

How many ideas are presented and how are they connected?

Are the ideas and opinions appropriate in relation to the text?

Does the portfolio show a continous and thorough activity?

These few examples which follow the reflective spirit of the portfolio demonstrate at the same time that the teacher can assist the learner’s progress with some motivating commentaries as: I liked especially…, I liked less…, it has attracted my attention…

In conclusion, we can estimate the usefulness of the portfolio by emphasizing its contribution to the learning processes based on the constructivistic theory which implies a certain distance from educational approaches centred only on results. But without doubt, the implementation of portfolios also requires a great deal of experimentation and practice by the teacher who will necessarily have to question his own teaching activity. Portfolio pedagogy not only refers to the need to change learners’ habits and learning styles, but also reveals new prospects and possibilites in teaching methods which may involve a complete revision of objectives whith regard to processes and not only outcomes. The use of the portfolio opens the possibility of converting evaluation into an inseperable part of this changing process by giving more importance to the teacher as a guide rather than a simple marker of exams. In this sense, this tool responds to the necessity expressed by reformers of the traditional educational system who seeks to substitute testing forms with other methods of evalaution, giving control of exams a different function within formative assessment.

 

4. Other interesting Web sites:

 

“The online Space for Teachers intergrating Technology into Classroom: Using Portfolios”. ALTEC: Advanced Learning Technologies in Education Consortia.  [Documento de Internet disponible en http://www.4teachers.org/profdev/as.php]

“Instructional Strategies Online”. 2004. Saskatoon Public School Division. [Documento de Internet disponible en http://olc.spsd.sk.ca/DE/PD/instr/strats/responsejournal/index.html]

Bafile C. 2003. “Reading Response Journals: Writing after Reading is Revealing”. Education World. [Documento de Internet disponible en http://www.education-world.com/a_curr/profdev/profdev085.shtml]

Reflection Journal Activities in Teacher Created Rubrics for Assessment”. University of Wisconsin, Stout. [Documento de Internet disponible en http://www.uwstout.edu/soe/profdev/rubrics.shtml]

“An Electronic Portfolio to Support Learning”. Canadian Journal of Learning and Technology. [Documento de Internet disponible en http://www.cjlt.ca/content/vol31.3/wade.html]

 

BIBLIOGRAFÍA:

Barrett, H. C. 1999. “The Electronic Portfolio Development Process”. [Documento de Internet disponible en http://electronicportfolios.com/portfolios/aahe2000.html]

“European Language Portfolio”. [Documento de Internet disponible en http://www.coe.int/T/DG4/Portfolio/?L=E&M=/main_pages/introduction.html]

Just read now”. The Beacon Learning Center: The U.S. Department of Education Technology Innovation Challenge Grant Program. [Documento de Internet disponible en http://www.justreadnow.com/strategies/active.htm]

Kemp, J. and D. Toperoff. 1998. “Guidelines for Portfolio Assessment in Teaching English”. [Documento de Internet disponible en http://www.anglit.net/main/portfolio/default.html]

Little D. and R. Perclová. “The European Language Portfolio: a guide for teachers and teacher trainers”. [Documento de Internet disponible en http://www.coe.int/T/DG4/Portfolio/documents/ELPguide_teacherstrainers.pdf]

Mendoza Fillola, A. Coord. 1998. Conceptos clave en didáctica de la lengua y la literatura, Barcelona: SEDLL.

“Managing Assessment Information”. 1996. Technology Education Coordinator: Province of British Columbia Ministry of Education MOE Curriculum Branch. [Documento de Internet disponible en http://www.bced.gov.bc.ca/irp/tech_ed/apdgen.htm]

“Managed Environments for Portfolio-based Reflective Learning”. 2006. [Documento de Internet disponible en http://www.eportfolios.ac.uk/FDTL4/references?display=assessment]

Meichenbaum, D. and A. Biemiller. 1998. Nurturing Independet Learners. Massachusettes: Brookline Books.

Palloff, R. M. and K. Pratt. 1999. Building Learning Communities in Cyberspace. San Francisco: Jossey-Bass Publishers.