The Assessment Model

ABSTRACT CHAPTER 4 The Assessment Model Celina P. Sarmiento, Levi E.Elipane, Brando C. Palomar, Marie Paz E. Morales Fundamental to STEAM education is quality assessment in that it promotes student learning and confirms students’ conceptual understanding, learning progress, and achievement throughout the teaching-learning discourse. It is a dynamic and cyclical process wherein teachers learn about their students, as they also learn with their students, that may provide greater positive impact on students’ learning. This chapter presents the details and features of the Philippine STEAM Assessment model developed through an exploration study, observing a three-tiered analysis of interview transcriptions, observation notes, and existing documents from sampled HEIs (SUC levels 1 and 2, LUCs and non-autonomous private schools) all over the country. Furthermore, a two-tiered validation process by experts, administrators, and practitioners was done to establish the suitability and appropriacy of the model for STEAM education and its alignment to the dimensions of the Technological Pedagogical and Content Knowledge (TPACK), domains and strands of the Philippine Professional Standards for Teachers (PPST), and the Policies, Standards, and Goals (PSG) set by the Commission on Higher Education (CHED). Two models were derived from the validation process, the Validated and the Emerging Models of STEAM assessment. The Validated Model represents the exact and observed assessment practices that transpired in the collected data. The Emerging Model was created to incorporate the suggestions of the validators, many of whom come from SUC levels 3 and 4 and autonomous private schools. In both models, the first three variables: Enablers, Drivers, and Processes of STEAM assessment encapsulates the fourth, that defines the target STEAM outcomes: Critical Thinker (21st Century Skills in the Emerging Model), Productive Citizen, and Innovative STEAM Professional or Learner. The models represent the framework that ensures the quality of assessment in STEAM education. It may guide the different educational stakeholders in grasping the many aspects of assessment in STEAM. It also offers a series of TPACK aligned indicators that would guide different institutions in developing, implementing, evaluating, and internalizing policies and guidelines that ensure quality assessment. Lastly, it defines the attributes of teaching competencies, insofar as it establishes the requirements for advancement in each career stage (Beginner, Proficient, Highly Proficient, Distinguished) of a STEAM educator Keywords: model variable, quality assessment, teaching and learning discourse, teaching competencies 4.1. The Model Defined 4.1.1. Why the Assessment Model (Rationalize the Need for the Model) Key-technologies propel industrial revolutions that result in societal changes. The 4th Industrial Revolution (IR4), characterized by high level of complexity and the incorporation of total network of product and production process (Dombrowski & Wagner, 2014), blurs the barriers between the physical and digital worlds (Kazançoğlu & Özkan Özen, 2018), its vision prompted by technological notions and solutions to attain a blending of the economy of scale with the economy of scope (Dombrowski & Wagner, 2014). These advancements are led by the emergence of modern disciplines like robotics, Artificial Intelligence (AI), Internet of Things (IoT), biotechnology, nanotechnology, autonomous vehicles, 3D printing, quantum computing, material science, and energy storage (Diwan, 2017). The impact of IR4 is felt not only in business, governance and the people, but also affects education, thus the term Education 4.0 was born (Sinlarat, 2016). Education 4.0 addresses the necessities of IR4 where human capabilities and technological innovations are aligned to permit new opportunities (Hussin, 2018; Harkins, 2008). Interestingly, Fisk (2017) and Goldsberry (2018) noted that the new goals of learning encourages learners to develop both knowledge and skills required, and to recognize the sources of information to become lifelong learners able to acquire knowledge and skills on their own. Education is built around the learners as to where and how to learn and tracking their performance is done through data-based customization. In this connection, peers become very vital in the acquisition of learning. Considerably, they learn together and from each other, while the teachers assume the role of facilitators in their learning. A countrywide movement to promote the viewpoint of Education 4.0 is stipulated in the Philippine Development Plan 2017-2022 (National Economic and Development Authority (NEDA), 2017), has driven all sectors of the government to innovate for progress. Specifically, significant changes in all levels of education to attain internationalization, globalization, IR4, and the country’s economic development through technological advancement, research and innovation, and the acceleration of human capital emphasize developing attributes and qualities of STEAM (Science, Technology, Engineering, Agri-Fisheries, Mathematics)-skilled professionals (National Economic and Development Authority (NEDA), 2017). Since then, government agencies, such as the Commission on Higher Education (CHED), have made concentrated efforts to foster successful STEAM education. For instance, the CHED has clearly articulated the value of STEAM education in the national curriculum to ensure its inclusion in the planning of classroom instruction. Ironically, however, little is known about how STEAM education is actually implemented in school. In particular, we barely know of how teachers, as a key agent of policy implementation, valorize and practice assessment in STEAM education. 55 Chapter IV Willis and Cowie (2014) view assessment as a ‘generative dance’ wherein assessment is ‘reimagined as a dynamic space where teachers learn about their students, as they learn with their students, and where all students can be empowered to find success and, in turn, develop learner agency’ (p. 23). Assessment can provide indicators of learners’ progress according to defined standards or through certain norms within a period of learning, as well as performance and achievement at the end of the learning period. Quality assessment takes into consideration both cognitive and affective domains, and must be informed, purposeful, authentic, valid, and reliable (Teachers' guide to assessment, 2016). Studies show that quality assessment may have better impact on student learning than any other intervention (Davies, Herbst, & Reynolds, 2012). Furthermore, data exhibit that all students benefit from quality assessment practice (William, 2011). Thus, a model that can underpin the components involved in the STEAM assessment process is necessary for the STEAM education community. It will provide various STEAM education stakeholders a structured conceptual blueprint involved in the practices of STEAM educators in executing assessment along with the details of the different factors that influence its implementation. 4.1.2. The Assessment Model is… The STEAM assessment model provides a holistic picture of the major considerations of STEAM educators in delivering quality assessment. It makes use of three key domains: (1) variables of the assessment model, (2) dimensions of the assessment model, and (3) quality indicators of the assessment model; in which we find the blending of core practices, the cyclical and dynamic process, and the crosscutting dimensions central to the assessment process. The model that comes in two forms, validated and emerging, serves as a framework wherein the implications are clear for what STEAM stakeholders, most especially what the teachers must do to deliver a quality assessment. It also equips them with assessment literacy that could advance their career stages. 4.1.3. The Assessment Model is NOT… The model far from reflecting the ideal assessment practices, but rather mirrors the most dominant and best practices of Philippine STEAM teachers as documented in the gathered data. It does not either aim to be a prescriptive model of what should be done, though mirrors the actual situation and reflects the insights of STEAM stakeholders. Furthermore, the model echoes the assessment process and practices from an investigation that takes STEAM as an allinclusive discipline; thus, it may not necessarily indicate features specific to individual STEAM areas. The Assessment Model 56 4.1. 4. What is new about the Assessment Model? The development of the STEAM assessment model followed five consecutive steps: (1) development of the first version from the codes and memos derived from the gathered data; (2) validation of the first version with STEAM experts, coordinators, and administrators; (3) adjustment of the model on the basis of the first validation; (4) validation of the adjusted model through a capability building program with practitioners (2nd validation); and (5) adjustment of the model based on the 2nd validation. The final output features a Validated Assessment Model, which captures the confirmed assessment practices of Higher Education STEAM teachers, as derived from the data; plus an Emerging Assessment Model that reflects additional facets and inputs that expert validators shared. Both the Validated and Emerging models embody the assessment process and the best practices unique to the Philippine STEAM education. Moreover, it is aligned to the dimensions of the Philippine Professional Standards for Teachers (PPST), the Policies, Standards and Goals (PSG) of CHED, and the components of the Technological, Pedagogical, and Content Knowledge (TPACK). Lastly, the model defines the attributes of teaching competencies, as well as establishes the requirements for advancement in each career stage (Beginner, Proficient, Highly Proficient, Distinguished) of a STEAM educator. 4.1.5. Salient Features of the Assessment Model The model developed clearly represents the collective idea of the Philippine STEAM assessment process. Specifically, the assessment model: 1. Identifies the support system that enables quality assessment 2. Documents the drivers of assessment in STEAM education 3. Ensures unified assessment process 4. Captures the best STEAM assessment practices in the Philippines 5. Highlights the target STEAM outcomes 6. Incorporates the dimensions of PPST, PSG, and TPACK 4.1.6. What is the Assessment Model? The Validated STEAM Assessment Model (Figure 4.1) makes four prominent variables, represented by the four layers in the figure, that comprise thirteen dimensions influencing the overall framework of assessment in the Philippine STEAM Education. The first three variables from the outermost layer going inwards are represented as concentric circles encapsulating the fourth (the innermost layer). 57 Chapter IV Figure 4.1. The Validated STEAM Assessment Model The first variable includes the (A) “Enablers” of STEAM Assessment which occupies the outermost layer of the model. This variable has two dimensions, (1) Institutional Affordances and (2) Sustainability, and considered crucial as it highlights the capabilities, forces, and resources that contribute to the success of the assessment process. The first dimension refers to the properties, facilities and policies of educational institutions or an aspect of its environment that describes and aids their STEAM assessment process. The second dimension values the efforts and practices exerted to secure, maintain, and improve the quality of the STEAM assessment process; involving the various research initiatives that aim to oversee and enhance assessment. The connection between the two dimensions indicates the significant linkage between the two and how one influences the other. The second variable–the (B) “Drivers” of STEAM Assessment–is displayed as the next layer of the model. It enumerates the key factors and main considerations in the STEAM assessment process and direction. These factors are categorized into three dimensions: (3) Equity and Diversity, (4) Collaboration, and (5) Modality. By and large, these three dimensions ensure the inclusion of all types of learners, accommodate the context and locale of the students, and make certain that each has a fair and equal opportunity during the assessment process, maintain the dynamic and engaging interactions that exist between various key players in the assessment process, and bestow the use of varied and appropriate tools and methods for various purposes of assessment in the STEAM learning-teaching discourse. The third variable enumerates the (C) “Processes” of STEAM Assessment, located in the third inner layer of the model. This variable identifies five stages which depict the last five dimensions of the model: (6) Planning and Preparation, (7) Implementation, (8) Rating, (9) The Assessment Model 58 Reporting, and (10) Reflection. They represent the different phases of reflective instruction where assessment principles are observed and practiced. The arrows pointing from one stage to the other symbolize that the STEAM assessment follows a specific order and the cyclical nature of the process. Furthermore, the Process of STEAM Assessment, with its corresponding indicators (correlated with the Drivers of STEAM Assessment) commands the assortment of STEAM assessment practices. It also defines the attributes of teaching competencies, if not, establishes the requirements for advancement in each career stage (Beginner, Proficient, Highly Proficient, Distinguished) of a STEAM educator. The last variable, appearing at the kernel of the model, specifies the desired (D) “Outcomes” of STEAM Education. It sets forth the intended trait and characteristics of STEAM learners and graduates, categorized into three dimensions: to be (11) Innovative STEAM Professional Learner, (12) Critical Thinker, and (13) Productive Citizen (members of the society); that also serves as a metric of a successful delivery of STEAM education. Figure 4.2. The Emerging STEAM Assessment Model The Emerging STEAM Assessment Model (Figure 4.2) resembles the Validated STEAM Assessment Model, with minor extension in some areas, one of which incorporates Innovation as a driver of STEAM assessment. This additional dimension (a total of 14 in the Emerging Model) seeks to apply creativity and problem-solving skills in utilizing and maximizing resources in the STEAM learning-teaching discourse. Another difference between the validated and emerging model lies in expanding the outcome “Critical Thinking” into “21st Century Skills”, that requires a gamut of abilities that a STEAM graduate should possess. Aside from critical thinking, 21st Century Skills also foster problem solving capacity and higher order thinking skills, as sine qua non in this information technology era. 59 Chapter IV 4.2. Alignment of the Assessment Model to TPACK and Other Theories (PPST and PSG) One of the major considerations in developing the STEAM assessment model lies on its alignment to the dimensions of TPACK, the domains and strands of PPST, and the indicators of the CHED’s PSG. The evaluation process with experts and stakeholders was also observed to ensure the validity of the assessment model and its adherence to the aforecited constructs; results indicate that they were incorporated in the developed model. Setting forth three components such as Technological Knowledge (TK), Pedagogical Knowledge (PK), and Content Knowledge (CK) and its combinations, the TPACK framework addresses the demands of Education 4.0 (Nurhadi, Purwaningsih, Masjkur, & Nyan-Myau, 2019). All these components are reflected in the assessment model (specifically in the processes variable of STEAM assessment) since the latter pinpoints the role of technology in STEAM assessment as well as focuses on how the content is integrated in the process. The alignment of the model to the TPACK ensures that teachers are able to properly assess the current set of students, expected to have acquired skills for collaborating, problem solving, innovative thinking, and the ability to utilize information and communication technology to the fullest (Valtonen, et al., 2017). On a similar note, significant changes are about to happen in light of the new PPST that was recently institutionalized in the Department of Education (DepEd, 2017) and eventually by the Commission on Higher Education to bring about greater attention to assessment of learning and even the appraisal of programs in the of STEAM education. The alignment to the PPST and to the PSG of CHED of the STEAM assessment model is then deemed to facilitate the process, especially that we are still in the nascent stages of implementation. Rather than be very positivist – the purpose is to engender deeper rationalizations on how the agency could still be upheld and developed, given the different contexts of each institution. The assessment model appears far being thought of to align assessment forms with the drivers identified and thus recalibrate learning among students, as it is considered to influence the teaching cultures in the educational institutions. Besides, the PPST specifies standards for teaching, as they align with the subject/course contents. Therefore, the alignment feature of the model represents a rigorous initiative to ensure that the PPST are met, while at the same time, challenges Philippine education – such as quality, equity, and relevance in the light of STEAM – are addressed in such a way that assessment practices are influenced. However, in being able to do so, support must be in place to adopt and implement the STEAM aligned with TPACK, PPST, and PSG to be achieved/done. Considering that institutional affordances and support have been a very important element of the model, the implication is that there ought to be valid, reliable, fair, equitable, and relevant system for assessments. The Assessment Model 60 4.3. Purposes of the Assessment Model A practicing STEAM educator at any level or discipline does not need to reinvent new ways and processes when looking for effective and efficient ways to ensure quality assessment implementation. After all, numerous educational models could serve as their compass towards meeting goals set for students’ learning. Simply put, models offer ways in which instructional experiences and learning environments can be created, organized, or delivered (Wilson, n.d.). They offer instructional or theoretical scaffolds, patterns, visualizations or illustrations for various educational components. The developed model is an instrument that can be used by teachers, administrators, and other stakeholders concerning assessment practices, as observed in the instructional planning and delivery, because it can help: A) Teachers: 1. follow a logical and systematic assessment process; 2. conceptualize either a more uniform or varied assessment strategies, guided by targeted content or subjects; 3. become reflective practitioners who continuously improve assessment tools and delivery; 4. gain insights about various assessment methods, purposes, tools and techniques in relation to students’ learning; 5. understand the many factors that drive assessment practices and processes; and 6. radically adjust and reconfigure existing assessment practices and instructional delivery to better meet the needs of the target STEAM outcomes. B) Administrators and educational authorities: 1. provide technologies and facilities that aid the delivery of quality assessment; 2. develop and implement policies and programs that secure the successful delivery and sustainability of the assessment process; 3. furnish appropriate and updated assessment trainings and tools to STEAM teachers; and 4. promote, encourage, and assist research initiatives that oversee and enhance assessment. Also, the developed STEAM assessment model projects a coherent image of the components and factors that guarantee quality assessment implementation, to assist teachers and other stakeholders in further developing assessment literacy. From a sociocultural perspective, Willis and colleagues (2013) define assessment literacy as follows: “… a dynamic context dependent social practice that involves teachers articulating and negotiating classroom and cultural knowledge with one another and with learners, in 61 Chapter IV the initiation, development and practice of assessment to achieve the learning goals of students.” (p. 242) The assessment model can serve as a shared language that may enable teachers to engage in critical inquiry of their assessment practices, enough to lead them to re-evaluate and adjust their principles and understandings of the assessment process. Through a roadmap that directs towards assessment literacy, teachers can be supported in developing the required skills to attain proficiency in assessment and equip them with the appropriate environment and technology to successfully deliver assessment, vital in quality STEAM education. The technology integration model may yet address the teacher quality and the students’ learning, particularly cognitive and affective components. The model intends to provide opportunities for the STEAM educators to update or adapt to new technology used in the classroom and to provide opportunities to capacitate STEAM educators. Furthermore, their effort might probably lead to integrate technology in innovative teaching strategy and thus further improves the teaching. More pointedly, the model outlines the cogency of technology integration to produce quality STEAM learners. 4.4. The Model Explained This section discusses the domains and the corresponding components of the Assessment Model. 4.4.1. Domain Overview The developed model comprises three major domains: (1) Variables of the assessment model; (2) Dimensions of the assessment model; and (3) Quality indicators of the assessment model, presented in detail below. 4.4.1.a. Variables of the Assessment Model A variable is a characteristic or quality, magnitude or quantity that can undertake transformations and that is subject to analysis, measurement, assessment, or control during a research endeavor (Arias, 2012). In terms of STEAM Assessment, the study adapted the definition of a variable as a characteristic that expresses the feature of the practices of STEAM teachers in terms of assessment. Four variables are reflected in the integrative model developed for Assessment in STEAM education. The first variable is the Enablers of STEAM Assessment, so crucial that it embraces the capabilities, forces, and resources that contribute to the success of the assessment process. The second variable refers to the Drivers of the STEAM Assessment, which enumerate the key factors and main considerations in the STEAM assessment process and direction. The third variable, Process of STEAM Assessment, describes the procedure and progression of STEAM teachers’ practices in conducting The Assessment Model 62 assessment. Lastly, the fourth variable, labeled as Outcomes of STEAM education, reflects traits and characteristics of STEAM learners and graduates. The last variable is considered as one of the metrics of success in STEAM education that contributes greatly in pedagogical planning, instructional implementation, and assessment considerations of teachers and other stakeholders. 4.4.1.b. Dimensions of the Assessment Model The dimensions specify the route of the actions and cover the distinctive feature of the whole, as an integrated piece (Butter, Aguilera, Quintana, Pérez, & Valenzuela, 2017), each of which catches a single aspect of STEAM assessment, but when fused together offers a holistic picture of the entire assessment process. The validated model proposed in this study contains 13 dimensions, while the emerging model has 14. The first two dimensions of both the validated and the emerging identify the components that enable STEAM Assessment, as the next three dimensions in the validated and the next four in the emerging make it possible to recognize the considerations of STEAM teachers in conducting assessment. Whereas the last five dimensions in both itemize the stages of the STEAM assessment process. The dimensions and working definition for each is presented in Table 4.1. Table 4.1. STEAM model dimensions and corresponding working definition Variables Dimensions A. Enablers of STEAM Assessment Dimension A1: Institutional Affordances Dimension A1.1 Curriculum Development Dimension A1.2 Institutional Identities Dimension A1.3 Agency and Empowerment Dimension A2: Sustainability Dimension A2.1: Quality Assurance Dimension A2.2: Research Undertakings Dimension A2.3: Policies and Programs 63 Working Definition Refers to the properties or facilities of educational institutions or an aspect of its environment and policies that aids the STEAM assessment process. The various approaches followed by institutions in continuously updating their curriculum for improvement. The unique characteristics and features that define an institution. Purposeful initiatives and actions of institutions that empower those involved in the assessment process. Efforts exerted to secure, maintain, and improve the quality of the STEAM assessment process. The verification procedures implemented whether internally or externally that ensure that the desired level of quality in the assessment process is met. The different research initiatives that aim to oversee and improve the assessment process. The system of principles implemented by the institution as a procedure or protocol that guides the STEAM assessment process. Chapter IV Dimension B1: Ensuring Equity Dimension B1.1 Gender Sensitivity C. Process of STEAM Assessment B. Drivers of STEAM Assessment Dimension B1.2 Monitoring and Feedback Dimension B1.3 Student Interests and Expressions Dimension B1.4 Contextualization and Localization Dimension B1.5 Ethics Dimension B2: Pursuing Collaboration Dimension B2.1 Student-to-Student Dimension B2.2 Teacher-to-Teacher Dimension B2.3 Teacher-to-Student Dimension B2.4 Community Involvement Dimension B2.5 Involvement of other Stakeholders Dimension B3: Utilizing Modality Dimension B3.1 Tools and Technology Dimension B3.2 Types of Assessment Dimension B4 (Emerging Model): Innovation Dimension C1: Planning and Preparation Dimension C2: Implementation Dimension C3: Grading Dimension C4: Reporting Dimension C5: Reflection The Assessment Model Ensuring inclusion of all learners and making certain that each student has a fair and equal opportunity during assessment process. Understanding and taking into account gender equality in the assessment process. Practices in checking the result, progress, and quality of the assessment and providing constructive information for improvement. Considerations in observing the behavior, expressions, and response of students that might affect the assessment process. Factors undertaken in placing and adjusting the assessment process to accommodate the context and locale of the students. Moral principles that govern the assessment process. Dynamics that exist between the various key players in the assessment process. Interaction among students during the assessment process. Interaction among teachers during the assessment process. Interaction between teachers and students during the assessment process. Community participation in the assessment process. Participation of other stakeholders in the assessment process. Varied tools used and methods applied in the assessment process. Various technological tools used in each dimension of the assessment process Different assessment tools or methods used in STEAM for various purposes Application of creativity and problem-solving skills in utilizing and maximizing resources in the STEAM learning-teaching discourse Practices and guidelines observed during the preparation for the assessment process. Practices during the actual execution of the assessment process. Processes of marking students' performance, outputs and tests, as well as manner of analyzing the results. Communicating the results of the assessment process to target clientele. Impressions and actions considered after the analysis of the results of the assessment process. 64 D. STEAM Outcomes Dimension D1: Critical Thinker (Validated) / 21st Century Skills (Emerging) Graduates ability to execute logical, reasoned, and wellthought-out judgments. / Graduates that possess skills, abilities, and attitude necessary to succeed in the 21st century workplaces. Dimension D2: Productive Citizen Graduates that are able and have the proper disposition to contribute greatly to the growth and development of the nation. Dimension D3: Innovative STEAM Professional Well-trained professionals adapted to today’s market needs and societal demands. 4.4.1.c. Quality Indicators for STEAM Assessment Indicators are a set of features or characteristics that allow or establish the description and evaluation of certain dimensions of a variable. They are usually presented in varied ways like a checklist that measures the achieved degree of quality or as guiding questions. The study generated a total of 53 quality indicators distributed to the first three variables, useful in verifying the extent of STEAM assessment in the Philippines. These indicators are divided into two groups, presented in Table 4.2 and Table 4.3. The first nine (9) indicators describe distinctive features of the enablers that greatly contribute to a quality assessment in STEAM. The rest of the indicators enumerate teachers’ practices that ensure successful assessment and correspond to both the process and the drivers of assessment in the validated STEAM assessment model. Table 4.2. Dimensions and indicators of variable 1: enablers of steam assessment Dimensions Indicators Dimension A1: Institutional Affordances Dimension A1.1 (1) Observation of practices and programs to continuously improve and attain the curriculum Curriculum Development Dimension A1.2 Institutional Identities Dimension A1.3 Agency and Empowerment (2) Presence and utilization of appropriate technology that aids the assessment process (3) Appropriations of financial support for improving the assessment process (4) Existence of continuing faculty development programs and activities related to assessment (5) Presence of guidelines for hiring new faculty members Dimension A2: Sustainability Dimension A2.1: Quality Assurance Dimension A2.2: Research Undertakings Dimension A2.3: Policies and Programs 65 (6) Existence and implementation of internal quality assurance practices (7) Existence and implementation of external quality assurance practices (8) Conduct of research projects/programs in improving the assessment practices (9) Existence and implementation of policies and programs ensuring quality of the assessment process Chapter IV Additionally, the TPACK dimensions (TPCK: Technological Pedagogical Content Knowledge, TCK: Technological Content Knowledge, PCK: Pedagogical Content Knowledge, TPK: Technological Pedagogical Knowledge, TK: Technological Knowledge, PK: Pedagogical Knowledge, CK: Content Knowledge) are emulated in the Process of STEAM assessment. The indicators of the aforecited variable were associated with the corresponding TPACK dimension, as shown in Table 4.3. Table 4.3. Indicators of variable 2: drivers of STEAM assessment and variable 3: process of STEAM of assessment and corresponding TPACK dimension Variable 3: Process of STEAM of Assessment Variable 2: Drivers of STEAM Assessment (Dimensions) B1.4; B3.2 B1.4; B3.2 B1.2 B2.2; B2.4; B2.5 Dimension C1: Planning and Preparation B1.4; B3.1; B3.2 B1.1; B1.3; B1.4; B3.1; B3.2 B1.4; B3.2 B1.3; B1.4; B3.2 B1.4; B3.2 B1.2 B1.4; B3.2 B1.3; B1.5; B2.3 Dimensio n C2: Implemen tation B1.2 B1.2; B1.5 B1.2; B2.1; B2.3; B3.1; B3.2 The Assessment Model Indicators (10) Ensures balanced distribution of items in terms of content (11) Includes real life application problems (since the application is usually disciplined specific) (12) Remediates students’ difficulties and misconceptions (13) Involves other experts and stakeholders in the assessment process (14) Uses various reliable references (including online sources) to create assessment tools (15) Considers the different background of students in terms of language, circumstances (some are returnees), learning styles, pacing, etc. and contextualizes the assessment (16) Ensures balanced distribution of items on tests in terms of difficulty and assessment tools (17) Includes questions that provoke HOTS (high order thinking skills) and critical thinking (18) Involves repetition of items/activities for mastery of skills (19) Interprets the result of previous assessment and uses it to design the next (20) Selects appropriate assessment based on the competencies and expected outcome (21) Orients learners about expectations for the assessment and how they will be graded (22) Ensures the quality of assessment (23) Plans rules that students must adhere to (24) Identifies the appropriate type of grouping TPACK Dimension PCK PCK PCK PCK CK PK PK PK PK PK PK PK PK PK PK B1.3; B1.4; B3. 2 (25) Encourages students to create (and improve their output) TPCK B1.3; B1.4; B3.1; B3.2 (26) Utilizes both traditional and authentic tasks TPCK 66 B1.4; B3.1; B3.2 B2.4; B2.5 B1.2; B1.3; B1.4; B3.2 B1.3; B1.4 B1.4; B3.1; B3.2 B1.2; B1.5 B1.2; B1.5 B1.1; B2.1; B2.3 Dimension C3: Rating B1.5; B3.1 B3.1; B3.2 B1.2; B1.3; B1.4; B3.1; B3.2 B1.2; B1.5; B2.3; B3.2 B1.2; B3.1 B1.2; B1.3; B1.5; B2.3; B3.1; B3.2 B1.2; B1.5; B2.2 Dimension C4: Reporting B1.2; B3.1; B3.2 B1.2; B1.5 B1.2; B1.5; B2.3 B1.1; B1.2; B1.5 B1.2; B1.5 B1.2; B2.3 B1.2; B3.1 Dimension C5: Reflection B1.2; B1.3; B1.4; B2.3; 67 B1.2; B1.5; B3.1 B1.2; B1.3; B2.3 (27) Integrates technology to innovate assessment implementation (28) Coordinates with other stakeholders in the assessment process (29) Exercises the art of questioning (rephrase questions that students cannot understand) (30) Observes students’ expressions (31) Uses assessment for/of/as learning (32) Provides clear definite instructions (33) Ensures proper monitoring of the assessment implementation (34) Assigns roles to students (leaders, monitors, recorders, participants, etc.) (35) Integrates technology to innovate rating of submission (36) Uses rubrics (37) Identifies students’ difficulties (38) Rates outputs and performances according to standards (set and agreed) (39) Conducts item analysis (difficulty and discrimination) (40) Ensures the quality of student submission (41) Deliberates the grade to be given to the student (some schools do team-teaching) (42) Integrates technology in reporting the assessment results (43) Monitors the number of students who reached the standards and progress of each student (44) Informs students about the results of the assessment for/of/as learning (45) Practices academic integrity and fairness (46) Maintains confidentiality of results (47) Provides recommendations (48) Evaluates the effectiveness of integrating technology in innovating the assessment (49) Analyses reasons/factors for students’ difficulties and misconceptions (50) Encourages students to reflect on the result of their assessment (51) Evaluates the need to re-teach the lesson or moveon to the next TPCK PCK PCK PK PK PK PK PK TPCK PCK PCK PCK PK PK PK TPCK PCK PK PK PK PK TPCK PCK PCK PCK B1.1; B1.2; B2.2; B2.3; B3.1 (52) Uses item analysis to improve assessment PCK B1.2 (53) Improves classroom practices based on the results of the assessment PCK Chapter IV 4.4.2. Continuum of Practice This section presents the traits and characteristics that STEAM educators must possess in each career stage. Each continuum level of teacher proficiency assumes proficiency at the previous level. 4.4.2.a. Beginner They possess knowledge in using assessment strategies, monitoring and evaluation, and feedback system consistent with the curriculum requirement. 4.4.2.b. Proficient They exhibit effective use of assessment strategies, monitoring and evaluation, and feedback system consistent with the curriculum requirement; they manifest capability of using assessment data to address challenges in implementing effective teaching and learning practices 4.4.2.c. Highly Proficient They participate and cooperate in a collective, complete, and sensible planning, selecting, implementing and monitoring assessment and evaluation of student learning, feedback system and designing of assessment-based programs and plan of actions for better progress in student learning. 4.4.2.d. Distinguished They model, exemplify, and mentor in planning, selecting, implementing and monitoring assessment and evaluation of student learning, feedback system and in designing of assessment-based programs and plan of actions for better progress in student learning. 4.4.3. Suggested Resources Information culled and analyzed from the transcripts of the interview and competencies demonstrated during the classroom observations described the resources of the assessment model into three levels – (1) individual, (2) peer/ faculty, and (3) institutional. These levels of resources are anchored on the assessment methods, tools, guidelines and processes, practiced and aspired by STEAM faculty members and administrators. These resources of the assessment model covers from the regulation and principles, as prescribed by CHED and implemented by the institution, to the actual assessment requirements and practices, as observed by individual and among faculty members. The Assessment Model 68 More specifically, the institutional level of resources highlights the facilities, properties and policies, as described by the “enablers” of the assessment model. It further describes the mechanisms on how the institution responds to the assessment needs, provides physical facilities and creates clear processes and policies in translating the assessment specifications and standards for the delivery and enhancement of STEAM courses. These resources include the institution’s assessment-related programs for quality assurance, curriculum improvement, personnel empowerment, research undertakings and branding. More specifically, it also stresses the intuitional procedures and parameters in maintaining and improving quality of the assessment context in all “processes” of the model. Expectedly, institutions have identified directions and rules concerning assessment from the planning-preparation process up to the reporting and reflection process, as highly reflected on the institutional actions in adhering with PSGs, constructing physical and online learning environments, enhancing course programs and aligning syllabi with standards like the PPST. Conversely, the peer-faculty level of resources accentuates on the practice of ensuring equity and diversity, promoting collaboration and utilizing modalities for assessment. These resources are manifested in the “drivers” of the model, where the faculty or unit of the institution established assessment mechanisms and practices related to students’ interest and expression, gender- and cultural-sensitivity, ethical considerations and matters on contextualization and localization. This level of resources also describes the varied purposes of assessment (as, of, for learning) being observed in the delivery of STEAM courses all shown on the assessment guidelines, tools and technology set and used by the faculty, together with their aspirations on improving and innovating the assessment understanding and practices of their unit. The last level of resources reflects the assessment practices of the individual STEAM educator as demonstrated on his or her daily learning-teaching discourses. Moreover, these resources are the instructor’s unique and contextualized pedagogical assessment strategies and tools in demonstrating the “drivers” in the different “processes” of the model to achieve the “outcomes” of STEAM education. More particularly, these assessment resources are highly utilized by the individual educators in the whole learning-teaching cycle. They cover the insights gained by the educators and translate them to the planning of instruction, enacting of STEAM courses, rating and reporting of STEAM learners’ academic performances, reflecting on the strength and weaknesses of the instructional practice based on students’ feedbacks, then return to the drawing board of planning based on certain insights gained. Besides, such level of resources provides information if these very resources from the institutional level are properly cascaded and translated in the sheer experiences of both STEAM educators and learners. It also captures both the strength and limitations of the resources provided by the institution and the faculty, together with the innovative responses of the individual STEAM educators in the context of assessment. 69 Chapter IV 4.4.4. Illustration of Practice This section provides representations and evidences of the different variables defined in the assessment model. They cover actual practices, aspirations and limitations on the context of assessment derived from the competencies demonstrated during the classroom observations and from the key features culled on the transcript of the interviews with the STEAM educators and administrators. In the context of the enabler variables of the assessment model, the “institutional affordances” are emphasized through the capacity of the institution in providing a conducive learning environment, adequate physical facilities and properties, substantial financial and appropriations, and training programs that support the underpinnings and processes of assessment so as to realize the standards in delivering STEAM education. It also includes assessment practices integrated in the institutional programs to attain STEAM courses, faculty development activities and guidelines, and to utilize technology. Comparably, “sustainability” is represented in terms of the initiated programs and policies concerning assessment practices of the institution, as aligned with the regulations prescribed by CHED and other educational reforms. It also defines the assessment programs of the institution concerning internal and external quality assurances, research endeavors and action plans for improvement. To illustrate, both enablers are markedly pronounced on the program descriptions, methodologies and sets of technology related to assessment, as stipulated on the course syllabi of STEAM disciplines. They are also presented in the context of classroom functionality, facility availability and restrictions, and practices of technology integration in relation to assessment specifications for local quality assurances and for describing intuitional performances. The practices of ensuring “equity and diversity” were demonstrated via the aspirations and positive attitudes by STEAM educators toward their learners with varied cultural backgrounds and academic profiles. It was also presented by practicing ethical considerations and confidentiality along with establishing clear academic expectations and protocols. In addition, it observed using student feedbacks, situational cases, rubrics and real life applications in assessing their understanding and learning, as well as by humanizing the procedure of technology for assessment practices. Conversely, “collaboration” variable was highlighted by the actual practices and yearnings of promoting teamwork and establishing academic relationship among STEAM educators and learners. Equally, it captured the assessment practices, as observed in the community and other stakeholder partnerships made possible through extension programs and service learning projects. The “modality” variable was established with the use of varied conventional and authentic assessment strategies and tools by STEAM educators in evaluating learners’ understanding and related-skills. Lastly, it presented the practice of the use of technology for assessment and research undertakings. These practices under the “driver” variable are commonly observed in all aspects of the “process” variable of the assessment model. For example, during the planning and preparation process, STEAM educators observe appropriate distribution of the content on a particular instruction, identify students’ misconceptions and difficulties, list questions that promote The Assessment Model 70 critical thinking skills, develop competency-based instruction and organize mechanisms for students’ grouping. The appropriate use of traditional and authentic assessment tools, provision of clear assessment instruction and expectations, practice of students’ appraisal and mechanisms for monitoring formative and summative tests were resorted to as assessment practices of STEAM educators during the implementation of instruction process. For the rating and reporting processes, STEAM educators showed good understanding of the use of criteria and rubrics, practices item-analysis, observed academic integrity and confidentiality, provided recommendation and feedback, and measured technology effectiveness related to assessment practices. Moreover, assessment practices observed in the reflection process helped identify students’ difficulties and misconceptions in learning STEAM courses, enhance test construction, and raise the level instructional delivery and learning environment. STEAM educators established these assessment practices at different levels and manifestations of observing equity and diversity, stimulating collaboration, employing modalities and introducing novelties to realize the functions of assessment in the actual learning-teaching cycle. They demonstrated these practices, as anchored on the goal of their course discipline to produce STEAM learners with attributes of being innovative professionals, critical thinkers and productive citizens, despite the constraints and other academic and administrative-related challenges encountered. 71 Chapter IV References Arias, F. G. (2012). The research project: Introduction to scientific methodology. Caracas: EPISTEME. Butter, M. C., Aguilera, E. M., Quintana, M. B., Pérez, L. J., & Valenzuela, E. S. (2017). Quality assurance for postgraduate programs: Design of a model applied on a university in Chile. The International Review of Research in Open and Distributed Learning, 18(1), 267292. Davies, A., Herbst, S., & Reynolds, B. (2012). Leading the way to assessment for learning: A practical guide (2nd ed.). Courtenay, British Columbia: Connections Publishing. Department of Education. (2017, August 11). Department of Education. Retrieved March 16, 2019, from National adaptation and implementation of the Philippine Professional Standards for Teachers: https://www.deped.gov.ph/2017/08/11/do-42-s-2017-national-adoption-andimplementation-of-the-philippine-professional-standards-for-teachers/ Diwan, P. (2017, August 6). Is Education 4.0 an imperative for success of 4th Industrial Revolution? Retrieved April 2019, from Medium: https://medium.com/@pdiwan/is-education4-0-an-imperative-for-success-of-4th-industrial-revolution-50c31451e8a4 Dombrowski, U., & Wagner, T. (2014). Mental strain as field of action in the 4th industrial revolution. Procedia CIRP, 17, 100-105. Fisk, P. (2017). Education 4.0 … the future of learning will be dramatically different, in school and throughout life. Retrieved April 2019, from https://www.thegeniusworks.com/2017/01/future-education-young-everyone-taught-together/ Goldsberry, C. (2018, June 14). What does Industry 4.0 mean for the global workforce? Retrieved September 2018, from Fastener News: https://www.fastenernewsdesk.com/21572/what-does-industry-4-0-mean-for-the-globalworkforce/ Harkins, A. M. (2008, March 28). Leapfrog principles and practices: Core components of Education 3.0 and 4.0. Futures Research Quarterly draft VIII, 1-15. Hussin, A. A. (2018, July). Education 4.0 made simple: Ideas for teaching. International Journal of Education and Literacy Studies, 6(3), 92-98. Kazançoğlu, Y., & Özkan Özen, Y. (2018). Analyzing Workforce 4.0 in the Fourth Industrial Revolution and proposing a road map from operations management perspective with fuzzy DEMATEL. Journal of Enterprise Information Management, 31(6), 891-907. The Assessment Model 72 National Economic and Development Authority (NEDA). (2017). Philippine Development Plan 2017-2022. Retrieved from National Economic and Development Authority: https://www.neda.gov.ph/wp-content/uploads/2018/01/Abridged-PDP-2017-2022_Updatedas-of-01052018.pdf Nurhadi, D., Purwaningsih, E., Masjkur, K., & Nyan-Myau, L. (2019). Using TPACK to map teaching and learning skills for vacational high school teacher candidates in Indonesia. In 5th UPI International Conference on Technical and Vocational Education and Training (ICTVET 2018) (Vol. 299, pp. 38-40). Atlantis Press. doi:https://doi.org/ictvet-19.2019.9 Sinlarat, P. (2016). Education 4.0: More than education (3rd ed.). Bangkok: Chulalongkorn University Press. Teachers' guide to assessment. (2016). Australia. Valtonen, T., Sointu, E., Kukkonen, J., Kontkanen, S., Lambert, M. C., & Mäkitalo-Siegl, K. (2017). TPACK updated to measure pre-service teachers' twenty-first century skills. Australasian Journal of Educational Technology, 33(3), 15-31. William, D. (2011). Embedded formative assessment. Bloomington, USA: Solution Tree Press. Willis, J., & Cowie, B. (2014). Assessment as a generative dance. In C. Wyatt-Smitt, V. Klenowski, & P. Colbert (Eds.), Designing assessment for quality learning (pp. 23-38). London: Springer. Willis, J., Adie, L., & Klenowski, V. (2013). Conceptualising teachers' assessment literacies in an era of curriculum and assessment reform. The Australian Educational Researcher, 40(2), 241-256. Wilson, L. O. (n.d.). Models of teaching. Retrieved March 2019, from The Second Principle: https://thesecondprinciple.com/teaching-essentials/models-of-teaching/. 73 Chapter IV