Submission Deadline-30th July 2024
July 2024 Issue : Publication Fee: 30$ USD Submit Now
Submission Deadline-20th July 2024
Special Issue of Education: Publication Fee: 30$ USD Submit Now

Portfolio Assessment Strategy _ A 21st Century Education and Constructivists Pedagogic Mediated Students Evaluation

  • Karen T. Odhiambo
  • 575-594
  • May 18, 2024
  • Psychology

Portfolio Assessment Strategy _ A 21st Century Education and Constructivists Pedagogic Mediated Students Evaluation

Karen T. Odhiambo, Ph. D

Department of Psychology, Education Psychology

University of Nairobi

DOI: https://dx.doi.org/10.47772/IJRISS.2024.803040S

Received: 21 March 2024; Accepted: 14 April 2024; Published: 18 May 2024

ABSTRACT

The shift in education to Assessment for Learning (AfL), formative verses summative evaluation in nature has resulted in a need for problem solution nexus between the practice of assessment and demands of the society to align with the 21st Century perspective of lifelong learning. The constant logic model of learning overtime of curriculum-pedagogy-assessment as informed by constructivist approach requires new approaches such as ‘Portfolios’ specifically ‘Portfolio Assessment.’ Conceptualization of the essence of 21st Century skills beyond discipline content is lacking. Consider cognitive efficiency and competencies as well as metacognitive awareness and skills as driven by the constructivist theory. The author would like to address these issues in this publication by harmonizing the thinking, providing knowledge, theoretical basis guiding practice, and products desired into a framework. The author will use the Portfolio Strategy as an avenue to demonstrate and explore attributes and themes alluded to in the write up.

Key Terms: Student Evaluation, Portfolio Assessment, eLearning, Constructive Theory, Lifelong Learning, 21st Century

INTRODUCTION

Over the last two decades, the education system globally has witnessed a shift from summative, product-oriented assessment towards formative process-oriented assessment. Brown (2004) and Gillett and Hammond (2009) describe this shift in terms of the need for assessment pedagogic strategies suitable for the 21st century and for strategies ‘fit for purpose.’ Boud (2000) appeals to the need for ‘sustainable education assessment’ that which leads to lifelong sustained learning. According to recent literature, the way to fulfil such needs is through effective use of formative assessment. The shift desired is from traditional measurement theory approaches to assessment that embraces not only acquisition of discipline knowledge and skills but also strategies that develop cognitive competency skills in learners such as critical thinking and metacognitive skills or awareness of evidenced learning process. The outcome of such a process is independent and democratic learners who are able to construct and reconstruct knowledge and reflect upon it in this knowledge economy even as they learn to adapt to contexts and circumstances beyond school. Strategies that suit this way of thinking become important. This brings to bear assessment as a pedagogic approach to education.

If construction of knowledge other than acquisition of knowledge is the aim of learning today, then assessment as practiced must change because construction is not an end but a process. The congruence of assessment of learning (AfL) an attribute for formative assessment becomes key. It is this that that will be explored in this chapter with a view to provide harmonized information even as the frame of thought regarding the conceptual frame is negotiated guided by the literature. The key thing is all about student evaluation while they learn within a ‘evaluation judgment’ process that which requires learners to develop their cognitive skills even as they learn thus the cognitive aspects and metacognition strategies that results in critical thinking abilities. The process as perceived has an important impact on instruction. Assessment and its evaluative judgement perspective become a process that leads to drawing together activities, learning and assessment strategies and the understanding learner behavior, a process and dialogue that leads to reproduction of knowledge even as the knowledge is constructed (Gibson, 1998). All of this within dimensions of learning hierarchies as found in blooms taxonomy (Gibbs, 1998) intelligence by Gardner, and learning strategies by Gagne (1985).

Today’s thinking of competency-based education (CBE) should be conceptualized and made applicable in a constructivist learning philosophy whereby in assessment terms this new thinking results in the demand for assessment that is formative in nature verses summative, that which results in Self- Regulated Learning (SRL) processes leading to learning to learn and reflection on the how of the learning. This kind of assessment does not lead to addressing distinct objectives but leads to learner’s ability to integrate knowledge, skills and attitudes. This cascades not only through knowledge but also thinking skills of the discipline and reflection of the learning. This assessment today is that which is structured and applied while learning, one that leads to thinking skills that are problem solving in nature that should lead to lifelong experiences of articulating real life situations or authentic. This is a process that needs learners to learn and acquire experiences within the discipline and context even as they transform their learning environment into a core-regulated space of learning. Thus the term reflection, self-regulated and self-reflection processes are important and so SRL. This process arises from an assessment pedagogy with self-driven approaches to learning which works best with learner centered models with the teacher as facilitator. The result is development of knowledge and skills necessary to solve problems and to the development of thinking skills of cognition and metacognition. All this while applying SRL skills towards intellectual skills and knowledge determination skills.  Motivation is a key competent in that if learners are not motivated, they do not stay on the task long enough to develop. So, students are key to the process and teachers should have capacity to drive the learning process. This kind of assessment becomes an “assessment pedagogic approach” that which drives the intended formative assessment process intended to its conclusive end.

Seen as a problem education solution nexus, the shift in learning in assessment terms can be described as an educational solution to an observed mismatch between practices of assessment and expectations of societal demands today that is associated with CBE or competency-based learning (Luhmann 1995). The mismatch is in relation to the fact that learning in the previous traditional way which was teacher centered focused more on examinations based on recall of facts about knowledge. The desire today is a learning process that is constructed by the learner, a product of critical thinking and evaluative judgement desired regarding the thinking of learning globally. Tradition learning approaches regarding assessment relied on learner receiving the answer and not coming up with their own derived response. The move towards learning that is also performance based is to ensure demonstrated performance learning outcome that is authentic and depicting realities of life as perceived. This approach also seeks to establish sustainable living with an idea for autonomous learners who are independent and liberal and not passive while expressive collaboratively.

Whatever is introduced into education to offer solutions in this world which serves as the platform for education problem nexus should address genuine education needs. The education logic seems to be constant, that of, curriculum-pedagogy-assessment guided by theories of learning such as behaviorists and constructivists approaches. Thus the aim of education is to respond through pedagogy as informed by theories of learning more so constructivists. This today is drawn from most appropriate digital technologies to facilitate learning in the current knowledge economy perspective for more effectiveness and efficiency and speed for real time results and feedback. The ICT approach has been experienced since the advent of programmable machines, as connected to the internet that offer infinite possibilities learning. With the availability of new digital technologies, classroom teaching and learning has benefited, however this benefit must be seen as offering useful solutions leading to genuine education needs and goals. Even with technology, new approaches need to be more or less superficial so as to be successful in their role.

LITERATURE

Student Evaluation in a Constructivist Learning Environment: Education Problem Nexus on Changing Learning Context

The new thinking of learning has brought to bear new terminologies and concepts as well as themes that need to be explored and synergies arising explained even as all this is drawn together by the author in a comprehensive whole in the thinking and processes as well as practices. In this section the author intends to provide the basic principles and practice. This section will do so taking into account the challenging nature of student evaluation today more so learning as a process driven by assessment pedagogy within a formative assessment frame. These are captured in the given section below.

Constructivist Learning Landscape_ Education Assessment Pedagogical Driven Perspective

The learning landscape is rapidly changing with learners needing problem solving, critical thinking and reasoning skills so as to deal with problems in unchartered grounds that is ever evolving that is also unfamiliar. Consider attributes already alluded to such as authenticity, lifelong perspectives of life, competency basis of learning and self-regulated learning processes and skills. The question arising is whether learning will lead to learners constructing the authentic realities in which they participate while learning and if this will help mold their thinking. Further, there is need to determine whether this experience will be a lifelong experience. How effective and efficient the learning processes and tools put in place are, is critical. The aim of constructivism is the ability to acquire, share and utilize knowledge within an online social context through reflection while developing discourse that is authentic. The new learning landscape has affected the content taught, the context and the environment taught in and the learner’s outlook. The new thinking has implications on the logic model typical in educating that has been in place for long, that of curriculum-pedagogy-assessment thus the pedagogic tool to driving the learning process.

In a constructivist learning environment this has to do with determining the core-regulatory process that leads to knowledge construction and critical thinking skills development. It is the constructivist learning environment that then leads to the demand of assessment in education that is formative in nature verses summative, that results in self-regulated learning processes leading to learning to learn and reflecting on the how of the learning. Learners learn and acquire knowledge even as they transform the knowledge and skills necessary to solve problems and to apply them resulting learners developing self-regulation skills of metacognition beyond cognitive intellectual skills. Thus the assessment driven learning process. It applies data collected from testing process such as e_ Testing in many circumstances. The frame of thought arising is assessment based on abstraction, intervention analysis of knowledge-based domains, skills and behavior towards lifelong education. The analysis procedure includes the collaboration, interactive process to determine the solution building (verses score determination) using the given models of knowledge domains, problems to solve, their solutions, and student evaluation as dialogue progresses.

The resulting focus on complex skills or professional competencies implies the integration of knowledge, skills and attitudes in its entirety so that it is not just transaction and transmission of knowledge but also transformation of this knowledge as it is constructed. This brings to bear integration of learning processes, assessment as tools and the psychological skills behavioral in nature such as self-efficacy to enhance learning. Educating is no longer learning as guided by objectives, a mere linear process but by ability to apply the arising knowledge to new problem situations authentic in nature, that which is based on real life situations, thus three-dimensional sphere and not unidimensional.

This is a learning process within a constructivist approach to learning and applying competency basis of learning with assessment embedded as key to pedagogic approach of the learning thus assessment and eventually e_Assessment. The author hopes to illustrate this as stated earlier using portfolio education strategy as this kind of assessment pedagogy requires modelling instruction in the most effective and efficient way. This constructivist point of view from assessment point of view, should lead to action that is visible, demonstrated and to some extent measurable. It becomes one that describes recurrent activities between simple, concrete meanings and more complex generalized meaning and vice versa, regulated learning within the frame of student evaluation learning. The process further results in co-regulated spaces that results in science of knowledge, a process that results in large amounts of information even as learning ensues. This is a process that goes beyond assessment pedagogy requiring ICT support for effectiveness and efficiency. It is believed that ICT would improve learning process towards instruction, managing the evaluative judgment process as well as knowledge building process and evidence of learning arising.

The conceptual framework intended is assessment based on abstraction, intervention analysis of knowledge-based domains, skills and behavior towards lifelong education. The analysis procedure bears the traits of inquiry learning, constructivism in nature, that which involves the collaboration, interactive process to determine the solution building (verses score determination) using the given models of knowledge domains, problems to solve, their solutions, and student evaluation through evaluative judgment as dialogue progresses. The hope is for an education system that leads to learners constructing their realities of the world in which they live in. It is because of the mismatch arising in conceptualization and application of assessment pedagogic process and factors in formative assessment processes that the author has chosen to approach the demands and capture the essence of these issues in the given book chapter herein. Also, there is lack of harmonized knowledge in linking the terminologies arising from constructivist approach to the new thinking as they are linked and aligned into the new thinking. The constructivist approach to learning and constructivist environment is still a distance reality more so in developing countries.

Knowledge Construction as Key Product of SE within a Student Assessment Pedagogy Driven Process of Learning:

This involves differentiated intellectual fields, in terms of nature of the epistemic relations of what can be known and how well it can be known more so as a social relation of who can know and differences differentiated in the discourse. It is important to ensure and generate patterns of differences that we can relate to regarding differences in knowledge knower structuring of the intellectual fields. The mode of assessment leads to possibilities of constructing knowledge or not. The meta genres can vary from factual descriptions and reports. It is important to establish and realize a relevant site for exploring and kinds of knowers that are implicated and the nature of their contributions to the knowledge. It is the framing question for analysis of one who gets to say ‘what of the known’ in the process of legitimization that is of real concern. These closely associate around aspects of discipline areas such sciences, mathematics, languages, social sciences, thus legitimate within its intellectual field (Martin & White, 2005).

In terms of who gets to know, while assessment ensues and knowledge accrues from assessments carried out even as discourse on the discipline area is generated, reference is be made to the modelling of interpersonal meaning in discourse of appraisal. Maton (2016) states, thus monglossic where the presenter is the soul knower verses heteroglossic discourse which is the presence of other voices. Where this is given voice whether quoting or reporting, the way these are represented and the nature of what is projected matters in academic discourse (Hood, 2010). These become the outcome of thoughts arising as ideas that transform into know for the presented and hopefully for the future. Even as the desire is for the future and for the information given is in terms of the ‘locale’ of one, interest is in their particular ‘social gaze’ on the world and ‘social space’ and time (Maton, 2016). The ideal way they challenge the naïve conceptions are important. The experience as a result of the assessment system set up together with the formative assessment tools applied and the e_Testing or e_Assessment should lead to opening up a wider set of possibilities in terms of ways of knowing and reflecting. The way in which projecting voices are represented in the discourse, that generates a continuum of visibility should lead to a maximum, thus, the flow of the discourse, with elaboration on their locus and disposition, hence the view they take on the world that which leads to desired discourses of legitimization towards realities of their world, that which is demonstrated.

This brings to bear social relations between education knowledge and its author or subject, one making claim to the knowledge. The result is strong social relations giving emphasis to the attitudes and dispositions of knowers of frequent multiple contributions to a domain of knowledge as referenced. Thus ability to challenge naïve conceptions and to develop understandings.

From Cognitive Psychology Perspective to Metacognitive Processes: Cognitive Factors in Acquisition of Knowledge, Cognitive Skills and Metacognition

Cognitive factors have a role to play in acquisition of learning as attained and is continuous even as it is improvement oriented, transferable across domains. The end game is not necessarily expertise but induction retrieval, installation of schematic knowledge structures other than acquisition, use of production rules, skilled performance, parallel integration of multiple sources of information rather than serial information. According to Stoyanov and Kirschner (2004), both novices and experts need cognitive factors support to increase their performance. The load in knowledge processed and the complexities arising in knowledge management as well as the multiplicity and the multi-processes that determine learning and learning strategies, it is acknowledged that even as learning becomes learner centered, there seems to be limited capacity of working memory, an important cognitive factor to deal with (Hambrick & Engle, 2003; Kirschner 2002). This brings to bear cognitive load and cognitive capacities to deal with the load. Challenges of limited capacities also arise in storage and processing of knowledge the basis of learning.  However, several theories do emphasize long term memory as well for it is the long-term memory that results in unlimited capacity to store information and it’s retrieval. This is important as it separates novices against experts. Such functions such as fixedness, dominant thinking patterns, routine expertise and negative transfer can then be explained by the ‘paradox of knowledge structure’ or ‘organization of knowledge’ in patterns and schemas that are absolutely necessary for success in learning in the new thinking. These are characterized by useful solution options, platforms for interpretation and communicating as well as presentation of new solutions. Possibility for inappropriate schema exists but may be detrimental in this day of new thinking towards authentic and lifelong implications.

In planning the learning one must appreciate and consider the fact that intelligence an attribute of learning is not uniform across learners. Wrong strategies or approaches within the education logic nexus such as assessment methods that are not appropriate to build on cognitive capabilities while learning is a risk to livelihood for it leads to inappropriate learning. It is important not to create a system of CBE that advantages some over others. Dominance arises from lack of skills and resources, and also creativity in designing learning activities more so SRL process. Care should also be taken even while pushing for ownership that dominance in the learning approaches does not arise. This is what competency-based education intends to change, a change that allows for possible alternative solutions. With insufficient knowledge structure, one is likely to fall into this trap of dominance leading to inability to look at a problem in a new way.

Possibilities of inappropriate approaches and insufficient knowledge structures has led to greater emphasis on metacognitive knowledge and strategies for regulating, monitoring and control even while self-managing and self-appraising performance (Paris & WInograd, 1990). Self-management is the metacognition action that is referred to as ‘analysis of problem situation, idea generation, idea selection and solution implementation, a process of assessment better yet e_Assessment. This is the activity that is so desired that assessment for learning becomes the strategy to enhancing and ensuring metacognition is entrenched in the process of learning, a derivative for competency-based education processes. The metacognitive process operates on the internal representations of the problem solver, in this case the learner. Paris and Winograd (1990) states that self-appraisal here refers to self-reflections on cognitive and affective processes in problem solutions and determination. Thus the enabling parts of knowledge structure is enhanced while suppressing the restrictive part. Thus learning is not only about domain knowledge in the learning but ability to transform this knowledge into articulating and demonstrating performance even while transferring knowledge experienced to transforming ones context of real life circumstances. feedback arising entered into the learning loop, thus ‘student evaluation’ in the new perspective within the frame of ‘assessment for Learning (AfL).

Constructivists Authentic Learning: Beyond Behaviourism

As a result of the challenge of real life situations the world faces and regarding the typical model of learning oriented towards behaviourist approaches, learning objectives are not appropriate approach to go by as they are not authentically referenced, nor do they conform to realistic real life situations. This has a bearing on how instructional methods are selected to accomplish the chosen authentic learning outcomes. The learning outcomes are highly integrated with reference situations that ask the learner to demonstrate the authentic objectives set out. The broader context of today’s approach to objective determination puts learning into a broader context thus making learning more meaningful. It is to this end that learners apply acquired knowledge, skills and attitudes that enable them to perform authentic tasks leading to transfer of knowledge and transformative circumstances. To be successful, what is to be learned must have clear implications for instruction. This is in sync with the new approaches of thinking in education regarding learning. However, this requires well-structured learning process, tasks and applications suitable to the regulated processes desirable of self-regulated learner process as well as embedded ICT regarding eLearning.

The problem is that education systems still lag behind as practice still reflects past behavioral approaches that addresses recall other than construction and engagement with knowledge. Thus the learning processes and the learner lack experiential learning designed in real or simulated task environments arising from intended traditional assessment of learning (AoL) verses assessment for learning (AfL). There is need for sequencing of such experiences, into ways to scaffold the learning, that which appeals to domain general skills, that which sustain critical thinking skills. It is this kind that appeals to varied solutions, which then lead to the development of expert approaches. In the same mode, these are the kind that ask for metacognitive skills that lead to and allow independent learning leading to autonomy. In assessment terms this what leads to demand of assessment in education that is formative in nature verses summative, that then instigates activities and resources of ICT that results in self-regulated learning process towards regulated spaces of learning.

Student Evaluation and eLearning Platform for Efficient and Effective Real Time Self-Regulated Process

ICT as electronic learning (eLearning) encapsulates a wide range of areas of learning from using the internet to research small items to participating in programmes taught through an electronic medium.  While the value of this learning is recognized it further shines a light on the use of technology in the classroom.  Research indicates that students are increasingly becoming proficient in the use of Information Communication Technologies (ICT) in advancing their learning.  However, the potential of ICT to advance student learning has yet to be fully realized in the design and technology classroom.  There is little evidence to suggest ICT is being used by learners in reflecting, reviewing, critiquing and evaluating their ideas as they regulate their learning.

Lewis argues that there are a variety of generative cognitive processes that are more likely to occur in technology education than elsewhere in the curriculum.  Design based Technology Education (TE) supports the development of higher cognitive thinking and problem-solving skills, where students have the capacity to express their creativity and innovation through the completion of the artifact they design, manufacture, and report on.  For effective higher order cognitive learning to occur students are encouraged to record and evidence their design evolution and development as the design journey progresses. One avenue through which this can be demonstrated is e_Portfolio.

An electronic education portfolio or e_Portfolio is a modern variation of the traditional paper-based education portfolio model.  Lorenzo and Ittelson defined an electronic portfolio as a digitised collection of artefacts including demonstrations, resources and accomplishments that represent an individual, group, or institution.  Similar to traditional paper-based models, e_Portfolios provide students with the opportunity to engage with their learning and critically analyse their thinking.  Further, an e_Portfolio portfolio has greater scope to allow students represent their learning through the full range of digital media.  This scope enables student to select a form of media which best allows them to portray their learning.  This multimedia scope presents students with the opportunity to capture, construct and evidence their capability and learning throughout the design activity.

A central aim of an e_Portfolio should be the critical engagement of students with the learning task they are completing.  It allows learners the opportunity to track and highlight their learning as it takes place.  This unprompted capturing of the activity allows learners to document their learning as it happens.  Reporting on the learning process in real time and on demand removes the “post-task” burden on learners enabling them to reconstruct and sanitize the account of the process even as it emerges. One aspect of the student portfolio is a record of student learning, and the ability of students to represent these abilities through a portfolio assessment strategy, an aspect which is central to the learning cycle. The frame that arises results in interaction with the objectives and learning outcomes with self-regulated process and technology e_Systems, constructivist in nature.

Lately, besides technological possibilities learning strategies in ICT include diffusion and connection towards networking people, groups, communities and services thus making eLearning systems even more diverse. Assessment tools have also become diverse to the extent that it is possible to apply concept maps, concept tests, that are multiple choice in nature, knowledge surveys, as well as, exams, oral presentations, peer reviews, written reports, anchor rubrics. There are also possibilities of concept sketches, case studies, seminar style courses among others. The main output besides desire for constructive knowledge and cognitive skills, is the operationalization of assessment pedagogy that is interactive that serve multiple learning and multiple perspectives and dimensions of discipline area.

METHODLOGY

Foundational Needs that Inform Conceptual Framework Understanding

The purpose of chapter is to discuss grounding presentations that inform current understandings of the synergies created arising from assessment as pedagogy as a learning tool driven process, a constructivist approach.  The literature that arises is a plethora of the development of the field of assessment as a pedagogic tool within a learner centered approach that is constructive in nature while giving directions for future research and practice. The numerous challenges arising is in using learner centered environments that result in ambiguity that tend to lead back to teacher centered practice, thus the tendency to fall back to behaviorist approach to learning. This may limit the learning environment potential and effectiveness towards the learning task as a core regulatory process within an evaluative judgment approach that supplies assessment processes. The literature shows that there is a challenge in conceptualizing and aligning these conceptually rich domains that requires learners to monitor and regulate several parts of their learning. Thus there is need to further contribute to this area by harmonizing knowledge and providing frameworks of the thinking herein and guidelines for practice.

Contextual factors apply in that there could be variations regarding contexts that have resource deficiency and human resource numbers and experience. Only then will learners be able to modify plans, goals, strategies and efforts in their learning in relation to dynamically changing contextual conditions within their contexts as situated locally and globally since learners and their contexts beyond school do become a community of practice. There is also modification related to fluctuations in learners motivational and affective states as a result of the process of evaluative judgment which require determination as well as how much social support, if any, may be needed to perform the task. Depending on learning contexts, one may need more to modify their cognition, metacognition, motivation and affect. Thus within the given frame arising in this write up, it can be argued that metacognition and self-regulation play a critical role in learning and should be explored more empirically.

On Portfolio as a Strategic Avenue to Illustrate ‘Student Evaluation’

One tool that has been recognized and is favored across board in the new thinking is portfolio Strategy as a key tool for learning within a competency-based system of education, that which applies assessment within the frame of evaluative judgement in the new education thinking of formative assessment thus ‘Portfolio Assessment (PA) learning.’  This is because the process desired such as metacognitive processes are best negotiated within portfolio strategy as an alternative assessment and learning tool. PA is a systematic and organized collection of student selected work which serves as a basis for ongoing students’ evaluation (Vavrus,1990). Portfolios reflect actual day-to-day learning activities of students and their reflections based on their work. A portfolio also documents student development regarding strengths and weaknesses in their learning while focusing on growth as perceived in the new ways of thinking in ensuring cognitive competence and metacognition (Hamm &Adams,1991). Thus it was thought Portfolio as a tool in education assessment becomes the most appropriate avenue to demonstrate the problem nexus of assessment in education as perceived.

The hope is to put all this together into a comprehensive whole and to propose a conceptual framework cum model that will harmonize the process of student evaluation from a constructivists paradigm while applying FA within a AfL format.   This chapter will also be a contribution to the literature on education assessment today with portfolio as the mediation of the process in this chapter as presented. This will be done by providing modalities, commonalities, while giving assumptions within a constructivist theoretical basis. The idea is for a framework cum guide of teaching and learning with assessment as a pedagogic driving tool, one which will contribute towards high quality learning outcomes. That a better understanding of the way learners learns and can learn and make sense of their experience in the manner put across does contribute to designing learning programs, more so student evaluation assessment attributes specifically that which will lead to authentic sustainable lifelong learning and performance that is authentic, a 21st Century perspective. It is with this in mind that this chapter on student evaluation as a pedagogy to learning is put together even as the epistemology is still evolving.

Student Evaluation Conceptual Framework: A Guide to Evaluation Thinking and Practice

The idea is for a portfolio assessment framework of teaching and learning through with an embedded eLearning platform as a contributing factor for achieving high quality learning outcomes, that which is lifelong emulating real-life situations. It is with the realization that better understanding of the way learners learn and make sense of their portfolio does contribute to designing learning programs that promote academic achievement more so in performance-oriented thinking such as competency-based learning that applies assessment driven pedagogy within an FA system. That conceptual change is a central principle in the development of meaning, cognitive reorganization that arise as learners engage in learnings activities set out for the portfolio learning experience that is competency based. The change envisaged most to development of formal constructs of meaning. Here teachers create situations whereby learners actively participate in activities that enable them to make their own individual construction of individual, group, problem solving and open discussion, as a product of recall, memory and critical thinking. That assessment is not only reproduction constructivists alignment of a typical educational model, but one that leads to enhanced learning activities that lead to expert learners and teachers as facilitators. It is with this in mind that this conceptual theoretical framework is conceptualized, with portfolios as the avenue.

Core regulatory construction learning spaces is the essence of a good e_Portfolio assessment model.  This is the spaces where the interactions from pedagogic assessment engagement occur regarding portfolio as a tool for learning, when the assessor informs the essence of their performance. The one assessing which could be the teacher or the learner, but preferably the learner, informs the one assessing about their leaning performance while the one being assessed internalizes the assessors’ regulatory actions and evaluative judgment. Three characteristics do arise that influence appropriation of such as learning process. The authenticity of learning outcome, the quality of feedback level as related to tasks, processes for self-regulation strategies for improvement of planning and student judgment strategies. What is required is connection between assessment expertise through self and peer assessment and therefore conceptualization as the development of evaluative judgment process

The conceptual guide is the conception as perceived from this author guided by the existing literature which as was stated earlier, the literature is not yet harmonized and is based on research that is yet to provide a comprehensive whole. The frame will be guided by the following process and knowledge:

  1. Synergies of education assessment problem nexus determined
  2. Principles of education student evaluation of evaluative judgement, e_Systems, assessment application and processes,
  3. Applications of assessment pedagogic approaches of self-regulation
  4. Education assessment principles arising as a link between the education attributes
  5. eLearning planning dimensions and applications

On the Education Portfolio Assessment Strategic Model

The goal of the chapter is to provide a framework for the new education problem nexus in place today and to provide knowledge as a guide for practice. The ‘Portfolio Assessment Conceptual Framework’ is intended to reflect synergies that result in core-regulated constructivist learning spaces. This is the environment where the interactions from pedagogic assessment engagement occur regarding portfolio as a tool for learning where the assessor informs the essence of their performance. The one assessing which could be the teacher or the learner, but preferably the learner, informs the one assessing about their leaning performance while the one being assessed internalizes the assessor’s regulatory actions and evaluative judgment. Three characteristics do arise that influence appropriation of such as learning process.

These are:

  1. The quality of feedback levels as related to tasks, processes,
  2. Self-regulation strategies for improvement of planning, and,
  3. Student judgment strategies.

What is required is connection between assessment expertise through self and peer assessment and therefore conceptualization as the development from the evaluative judgment process.

Coupled with the background profiled above one should be able to follow the tables below and be guided. The system as well as e_System works like an engine whereby many perspectives of student evaluation are implied and only certain attributes are manifested through the assessment pedagogic learning of SRL process. The SRL process cannot be seen but can be evidenced through the outcome of its manifestation in terms of learner behavior and award or performance. Rubrics criteria seems to be the most appropriate and has been explored in one of the tables. What is visible and visualized is articulated and evidence arising is alluded to in the table details. It is with this in mind that the attributes in the table below are founded. The given is the basics of this chapter on “Student Evaluation” herein. The idea was to put together and organize the knowledge on student evaluation into a consolidated comprehensive whole. The framework may not be the ultimate frame of thought and guide, but it does contribute to better harmonization the knowledge and does well to guide education professionals and teachers alike.

The tables represent what the author of the chapter has determined as the conceptual framework and represents a Portfolio Strategy as the avenue with given thematic areas sectioned below. The literature or the background has presented the discourse that describes the tables and their role and function and will not be repeated. The essence of what each table is alluding to is in the title of the section and as attributed as the subtitle of each table. The four sections are given as follows:

  1. Basic principles and practice of student evaluation process
  2. Formative Assessment Strategy and Self-Regulation Learning Perspective
  3. Student Evaluation and Formative Assessment Strategy and Self-Regulation Learning Perspective
  4. Portfolio Education Assessment Strategic Model

RESULT

Framework for The Basic Principles and Practice of Student Evaluation Process _ A Portfolio Assessment Strategy

The author sort to put together the thinking of “student evaluation” within a constructivist approach and embedded within an eLearning platform that will result in core-regulated spaces regarding a ‘portfolio strategy.’ The goal was a frame of thought that would guide resulting in enabling teachers and education practitioners to be able to have reference which is a one-stop kind of guideline towards assessment that is authentic in nature that will result in development of the discipline knowledge as well as cognitive skills of critical thinking and metacognition. As stated, this will be done within a portfolio assessment strategy as the avenue for the framework as envisaged. This has been conceptualized and presented in tabular format as given in 3 tables, with Background Tables formats and finally the “Conceptual Framework” regarding Portfolio Student Assessment Strategy. The framework has been articulated within the given tables below:

  1. Attributes of student evaluation guiding philosophy and practice
  2. Student evaluation assessment facilitation, tools and resources
  3. Rubrics performance criteria_ Principles and practice
  4. Portfolio student assessment strategy

Attributes of Student Evaluation Guiding Philosophy and Practice

The section has been constructed to give a background on student evaluation and includes synergies and attributes of:

  1. Constructivist theoretical basis
  2. Student pedagogic assessment process
  3. Evaluation Judgement perspective 

TABLE 1: Student Evaluation Guiding Philosophy and Practice

  Guiding Philosophy and Practice Built on Student Assessment Guiding Philosophy_ Constructivism within CBE model
1. Constructivist Theoretical Basis_

 

 

Principle: Based on the belief that knowledge is constructed and mutually built in the social context leading to authentic knowledge construction and cognitive development.

Authorities:

Piaget (1890-1980)

Vygosty(1896-1934)

Dewey (1859-1952)

Practice: Teacher acts as coach supporting understanding and not answers helping learner build on their own understanding. This is by reflecting on own experiences while constructing ones understanding.

Proposition: Results in construction of meaning that arises from Education Assessment process evaluative judgement of knowledge from the learning process arising. Learning relates to real life tasks, active techniques, learner active vs passive; Develops learner cognitive efficiency and metacognition awareness

2. Student Pedagogic Assessment

 

Coordination, Regulation and reflection as well as critical analysis of learning content, strategies and processes

Principle:

Self-Regulated Learning (SRL) Models:

 Zimmerman (2000)_ cognitive motivation and emotions

Winnie (1996,2011): Cognitive and metacognitive SRL process

Practice:

Assessment pedagogic driven learning using Formative assessment practice within a self-regulated learning process

Self-Regulated Process and Skills:

·       Content

·       Dialectics or questioning

·       Reflection

·       Collaborative Learning

·       Evaluative Judgment*

Proposition:

·       Knowledge Construction,

·       Cognitive Efficiency,

·       Critical Thinking Skills,

·       Metacognitive Skills,

·       Community of practitioners.

·       Social Dimension _ lifelong and sustainable

3. Evaluative Judgement Scientific Process_

 

Student engagement and interaction with the content, learning outcome instructional activities

Principle:

An Assessment for Learning (AoL) perspective carried out while ensuring a core-regulated Learning spaces. It involves determining value and worthwhileness and critical judgement for effective learning and feedback:

·       Context or Adequacy of performance relative to what is required and how;

·       Quality and Standards or specific level of quality and attainment level desired;

·       Assessment Criteria or specific level of quality and attainment desired

·       Expertise or What is needed to understand and appropriate quality and internalize the criteria together with the interpretation.

 

Practice:

Applications that integrates ‘Evaluative Judgement” while assessing students:

·       Self-Assessment

·       Peer collaboration_Teamwork

·       Feedback loop

 Proposition:

Ability to judge learning of the self and others;

Ability of one’s own learning assessment capability,

Influence on self-regulation of learning while assessing,

Enables core-regulation of learning spaces for effective learning

Develops self-regulation skills thru developed evaluative judgment skills of FA and SRL

Learning becomes strategic in their approach as well as shaping the learning experience

FA: Formative Assessment; SRL: Self-Regulated Learning

Dr. Karen T. Odhiambo (2023)

a. Student evaluation and formative assessment (FA) strategy and self-regulation learning (SRL) perspective

The section has been conceptualized to include the following:

  1. Formative assessment tools
  2. Data or learning and learner outcome collection and collating tools
  3. Psychometric approaches
  4. Rubrics performance criteria

TABLE 2: Student Evaluation Assessment Facilitation, Tools and Resources

Attribute: Education Assessment Perspective Built on Student Assessment Guiding Philosophy_ Constructivism within CBE model
Assessment Strategic Tools

_Determinants of SRL process through learning outcome, measures, performance and quality of learning_

Principle:

Reliable and valid

Practice:

Formative Assessment (FA) tools_

Scaffolding:

·       Mind maps,

·       fishbowl

Education Portfolios*

Data Collection and collating tools: Tests

Classroom Tests_ Essays, Multiple Choice,

Psychometric Approaches_

Test analysis, interpretation and statistical presentation; qualitative and quantitative

Rubrics*

Criteria for learning. Determination of learning from multi-dimensional performance criteria

Proposition:

Effective and Efficient system considering context, context, tools, SRL process

e_Learning Platform

 

_ ICT, Computer Programmed and ICT novel practices of virtual world and networking

Principles:

_Connects learners to peers, content and others_ knowledge and skills.

_Provides support and continuous access.

_Dynamic on-to-one support.

_Introduces Test codes built into programming.

_Retrieves and updates learner activity in real time and speed.

_Reviews, collects, sends feedback even ONLINE

 

Practice:

Supporting Tools:

·       Tech. Tools: IT, ICT, Virtual-Online, internet, web, bulletin, webcasts, multi-media

·       Proliferation Methodologies for Virtual World_ face book, You tube, WhatsApp, Blogs,

·       Networking_

Proposition:

Blended model with eLearning pedagogic and e-System:

Tech. Tools for communication_

·       Synchronous/Asynchronous

·       Communication tools

·       Course management tools

·       Programmable tools_ Technology

Sustainable Learning:

Methods: narrative, genre, discourse analysis, evaluative judgment, hermeneutics

Process: expanding subject matter within an eLearning format

ICT Novel Practices: eLearning platform, Internet, Web-Technologies, Bulletins, Webcasts, Online Education

 

Dr. Karen T. Odhiambo (2023)

 b. Rubrics as a criteria for determining the performance

Rubrics seems to be the most appropriate criteria for determining performance, a constructivist learning approach within competency-based education. The four sections of the model are given as follows and they provide the concept of the Portfolio Education Assessment and so e_Portfolio Assessment Strategic Model. These are related to but not limited to these:

  1. The principle and concept
  2. Types of rubrics_
    1. Analytical
    2. Holistic
    3. Standard Based
    4. Check List
    5. Reflective
  3. Quality Levels _ Proficiency, novice, beginner
  4. Award Criteria_ rating: A-B-C-D-E; 5-4-3-2-1; Number Value: A (5) (80-100%)
  5. Determining value and worthiness

TABLE 3: Rubric’s Performance Criteria _ Principle and Practice

Education Assessment Perspective: Built on Student Assessment Guiding Philosophy_ Constructivism within CBE model
Rubric:

Competency Based Performance Criteria and Measures

Principle:

Criteria for determination of learning from multi-dimensional performance criteria. They offer a chance for reflection, having sliding scales that enables learner to visualize and perceive clearly areas of attainment and growth. They could be applied to various learning areas and tasks but should be relevant for they serve different purposes including assessing the rubrics and every other process therein.

  Practice:

 

Types of Rubrics:

Analytical_ Skill based, detailing goals and indicators and specifying the evaluation criteria for each learning outcome

Holistic_ Single scale consisting of a measure giving a value of range an overall judgement

Standard Based_ Shows Advancement in a given prescribed area

Checklist_ A matter of completion than proficiency. It is about completeness of learning task and not necessarily quality.

Reflective: Provides feedback for transformative learning action

Measurement Criteria:

Quality Levels:

 

Outstanding_

·       Effective

·       Adequate

·       Ineffective

Expectation_

·       Meeting

·       Approaching

·       Not meeting

 

Accomplishment_

·       Advanced

·       Proficiency

·       Novice

·       Beginner

 

Award Criteria:

Score: 5: 4 : 3 : 2 : 1

Grade: A: B : C : D : E

Descriptor_

5(A) Excellent

4(B) Good

3(C) Could do better

2(D) Below criteria

1(E) Under Performer

 

Quantifying_

A (5) (80-100%)

B (4) (60-79%)

C (3) (40-59%)

D (2) (20-39%)

E (1) (10-19%)

 

  Proposition:

 

Regarding quality_

·       Specific criteria reflected _ e.g., Bloom Taxonomy hierarchy of learning and Gardner’s intelligence mental processes

·       Learners know what to expect

·       Levels of performance relevant to learning outcomes intended

·       Consistency in awarding performance

Dr. Karen T. Odhiambo (2023)

c. Portfolio Education Assessment Strategic Conceptual Framework

The four sections of the model are given as follows and they provide the concept of the e_Portfolio assessment strategic model. These are:

  1. Portfolio basis for assessment _ A strategy for learning
  2. Portfolio assessment principles, practice and tools
  3. eLearning and assessment embedded process within Portfolios’
  4. Determination of Learning and learner competencies _ Rubrics Criteria*

TABLE 4: Portfolio Student Assessment Strategy _ Conceptual Framework

Portfolio Assessment Strategy _ Knowledge and Guide
Portfolio Basis for Assessment_ Strategy for Learning Portfolio Assessment Practice eLearning and Assessment Embedded Process Learning and Learner Competencies
Portfolio Purposes:

Show Case,

Growth in Knowledge skills,

Stages of development of project,

Reflective ness portfolio

 

Pedagogic Leverage:

Space and Platform

Design, Evidenced

Goals, Assignment and contextualizing while constructing of knowledge;

Reflective practice;

Strategies for engaging; Schedules for review; application of digital spaces

Co-Regulated Spaces tools: Dynamics of self-regulation learning

Portfolio Applications_ Assessment Practice (SRL)

Participatory, anchored instruction, cognitive apprentice,

Generating Learning,

Computer support,

Discovery, Learning

Interpretation constructs,

Mind tools,

Encounter authentic,

Project method approach

Portfolio Assessment Strategies:

Problem question

Conceptual Model

Theoretical basis_ IF-THEN

Formative analysis _ revision

Summative analysis

Interpretations

Assessment Formats:

Written Assessment

Performance Tasks

Projects

Portfolio specific

·       Work Samples

·       Records

·       Written work

·       Demonstrations

·       Artifacts

·       Journals

ICT Novel Practices_ e_Systems

 

e_Learning platform

Internet

Web-Technologies

Bulletins

Webcasts

Online Education

_synchronous and asynchronous

Tech. Tools: IT, ICT, Virtual-Online, internet, web, bulletin, webcasts, multi-media

Virtual World_ Proliferation Methodologies of

face book, You tube

Sustainable Learning:

Methods: narrative, genre, discourse analysis, evaluative judgment, hermeneutics

Product: Knowledge construction vs descriptive recall knowledge; demonstration in visual, oral, written, audio; Demos, illustrations

Process: expanding subject matter

 

Learner Competencies _

Cognitive skills:

Critical Thinking understanding; Reasoning; Problem solving; Analytical

Academic Skills: writing, problem solving, inquiry based, aspects of

Multiple modes: demonstration in visual, oral, written, audio

Author: Dr. Karen T. Odhiambo

CONCLUSION

The task here was to come up with a harmonized literature and a Conceptual Frame of Thought ‘to guide and inform practice on “Portfolio Education Assessment Conceptual Framework. This awareness of process and structure of the paradox of knowledge structure and of critical thinking and metacognitive knowledge is what is supposed to be enhanced in the new thinking of educating assessment. It is believed ‘portfolios’ is the best approach thus far. This is what competency-based education is all about, that which has assessment as philosophy first pedagogy, a process driven by formative assessment approaches within classroom learning as assessment is determined and applied, and feedback arising entered into the learning loop, thus ‘student evaluation.’ The chapter has explored this construct within the frame of Assessment for Learning (AfL) with e_learning platform embedded. It is not only about domain knowledge in the learning but ability to transform this knowledge into articulating and demonstrating performance even while transferring knowledge learned to transforming one’s context of real-life circumstances. This is a situation that both novices and experts should attain as well as the community at large. Since there are still challenges arising as shown in the literature, the author in contributing to the new thinking has come up with a ‘Conceptual Frame of Thought’ to guide and inform practice, thus “Portfolio Education Assessment Conceptual Framework”. It is only when education fraternity become experts in this process as well as the learners and community, then, claim can be made towards sustainable learning that is lifelong will hopefully be attained. Before then, more guidelines and models will have to be designed to develop the area further. This publication is a response to the challenges that still plague the field as the world embraces 21st Century Education towards lifelong learning.

Funding:

The research received no external funding.

Conflicts of Interest:

The author declares that there is no conflict of interest.

Informed Consent Statement:

This was obtained from all subjects involved in the study.

REFERENCES

  1. Anderson A. (2004). Towards a theory of online learning. Theory and practice of online learning
  2. Biggs, J. (1998). Assumptions underlying new approaches to assessment. In P. Stimson & P. Morris (Eds.), Curriculum and assessment in Hong Kong: Two components, one system, (pp. 351–384). Hong Kong: Open University of Hong Kong Press
  3. Black, P., & William, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy and Practice, 5(1), 7-73.
  4. Brown (2004) and Gillett and Hammond (2009)
  5. Boud (2000). Sustainable Assessment: Rethinking assessment for the learning society
  6. Boud, D. (1995). Enhancing Learning through Self-Assessment.  London: Kogan Page.
  7. Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education, 22, 2, 151-167.
  8. Boud, D. (2010). Studies in Continuing Education. Redesigning assessment for learning beyond higher education. Pages 151-167 | Published online.
  9. Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives, handbook I: Cognitive domain.Portfolio Assessment Strategy _ A 21st Century Education and Constructivists Pedagogic Mediated Students EvaluationNew Yolk, NY: David McKay Company.
  10. Boud, D., Dawson, P., Tai, J., & Ajjawi, R. (2018). Creating an agenda for developing students’ evaluative judgement. In D. Boud, R. Ajjawi, P. Dawson, & J. Tai (Eds.), Developing Evaluative Judgement: Assessment for Knowing and Producing Quality Work (pp. 186-195). Abingdon: Routledge.
  11. Dewey, J. (1910). How we think.Boston, MA: D.C. Heath and Company.
  12. De Fina,A. A. (1992). Portfolio assessment: ^^etting started. New York: Scholastic.
  13. Doğan, N., Kıbrıslıoğlu Uysal, N., Kelecioğlu, H., & Hambleton, R. K. (2020). An overview of e-assessment. Hacettepe University Journal of Education, 35(Special Issue), 1-5. doi: 10.16986/HUJE.2020063669
  14. Gagne, R. (1985). The Conditions of Learning (4th Ed). New York: Holt, Rinehart and Winston
  15. Gibbs, G. (1998) Learning by Doing: A Guide to Teaching and Learning Methods. Oxford Brooks University, Oxford.
  16. Gillett A., Hammond, A.(2009). Active Learning in Higher Education, Mapping the maze of assessment: An investigation into practice. journals.sagepub.com
  17. Glazer, S.M. & Brown, C.S. (1993). Portfolios and beyond: Collaborative assessment in reading and writing. Norwood MA: Christopher-Gordon.
  18. Hamm &Adams (991). The impact of portfolio evaluation method on the development of children’s reading skills. Author links open overlay Panel Yalda Delgoshaeia KamālKharrāzib Mahmoud Talkhabi. Procedia – Social and Behavioral Sciences. Volume 32, 2012, Pages 356-360
  19. Hambrick, D. Z., & Engle, R. W. (2003). The role of working memory in problem solving. In J. E. Davidson & R. J. Sternberg (Eds.), The psychology of problem solving(pp. 176–206). Cambridge University Press. https://doi.org/10.1017/CBO9780511615771.007
  20. Kirschner, P. A. (2002). Cognitive load theory: Implications of cognitive load theory on the design of learning [Editorial]. Learning and Instruction, 12(1), 1–10. https://doi.org/10.1016/S0959-4752(01)00014-7
  21. Luhmann. (1995). Social Systems. Stanford University Press, 1995
  22. Mayer R., Sunga, E., (2003). Five facets of social presence in online distance education. Computers in Human Behavior 28 (2012) 1738–1747.
  23. Mohammad Awad Al-Dawoody Abdulaal (2020). A Shift from a [23] Monoglossic to a Heteroglossic View: Metalinguistic StegoTranslanguaging Lens Approach. Arab World English Journal (AWEJ) Volume 11. Number 4. December 2020 Pp.461-473 DOI: htttps://dx.doi.org/10.24093/awej/vol11no4. 29
  24. Martin, P. & White, W. (2005). The Language of Evaluation: Appraisal in English; January 2005; DOI; Publisher: Palgrave Macmillan [Peter Robert Rupert White]
  25. Maton, (2016). A review of Karl Maton, Susan Hood & Suellen Shay (eds.), Knowledge Building: Educational Studies in Legitimation Code Theory, Abingdon and New York: Routledge, 2016. xvii + 264 pp.
  26. Maton, K. (2016). Legitimation Code Theory: Building knowledge about knowledge-building, in K. Maton, S. Hood & S. · LCT publications
  27. Mayes, T. and de Freitas, S. (2004) Review of e-learning theories, frameworks and models. London: Joint Information Systems Committee
  28. Neto, R.D & Henderson, J.L, (1994). Portfolio Assessment: An Alternative in Evaluating Student Performance. NACTA Journal 38, No. 1 (March 1994), pp. 26-28 (3 pages)
  29. Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education31(2), 199-218.
  30. Paris, S. G., & Winograd, P. (1990). Promoting metacognition and motivation of exceptional children. RASE: Remedial & Special Education, 11(6), 7–15. https://doi.org/10.1177/074193259001100604
  31. Pond, W. K. (2002, Summer). Distributed education in the 21st century: Implications for quality assurance. Online Journal of Distance Learning Administration5(2). Available online at: http://www.westga.edu/~distance/ojdla/summer52/pond52.html
  32. Rief,L. (1990). Finding the value in evaluat^ion: Self-assessment in a middle school classroom. Educational Leadership. 47(6), 24-29.
  33. Sadler, D. R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment & Evaluation In Higher Education, 34(5), 535-550.
  34. Schat, S. (2014). Social and emotional learning: A critical appraisal. [Review of Social and emotional learning: A critical appraisal, by N. Humphreys]. Journal of Invitational Theory and Practice, 20, 61–63.
  35. Stoyanov, S., & Kirschner, P. (2004). Expert Concept Mapping Method for Defining the Characteristics of Adaptive E-Learning: ALFANET Project Case. Educational Technology Research and Development, 52(2), 41–56. https://doi.org/10.1007/BF02504838,
  36. Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018). Developing evaluative judgement: Enabling students to make decisions about the quality of work. Higher Education, 76(3), 467–481. doi:10.1007/s10734-017-0220-3
  37. Teaching, Learning, and Technology (TLT) Group. (2011). Rubrics: Definition, tools, examples,vreferences. Available onlineat: http://www.tltgroup.org/resources/flashlight/rubrics.htm
  38. Tiemey,R.J., arter,M.A.,&Desai,L.E.(1991). Portfolio Assessmentin the reading-writing classroom. Norwood,MA: Christopher-Gordon.
  39. Vavrus,L. (1990). Put portfolios to tllie test. Instructor Magazine, pp.48-53.
  40. Wright, P.M. and Kehoe, R.R. (2008) Human Resource Practices and Organizational Commitment: A Deeper Examination. Asia Pacific Journal of Human Resource Management, 46, 6-20. https://doi.org/10.1177/1038411107086540
  41. Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 13- 40). San Diego, California: Academic Press

Article Statistics

Track views and downloads to measure the impact and reach of your article.

3

PDF Downloads

[views]

Metrics

PlumX

Altmetrics

Paper Submission Deadline

Subscribe to Our Newsletter

Sign up for our newsletter, to get updates regarding the Call for Paper, Papers & Research.

    Subscribe to Our Newsletter

    Sign up for our newsletter, to get updates regarding the Call for Paper, Papers & Research.