Chapter 10 Designing and Conducting Formative Evaluations Powerpoint

Chapter 10 Designing and Conducting Formative Evaluations from cdjhaigler

Chapter 10 Designing and Conducting Formative Evaluations Summary

A formative evaluation, evidence of an instructional program’s worth is gathered for use in making decisions about how to revise the program while it is being developed. This is why it is called "formative" evaluation, because the instruction is in its developmental stages and is not yet "grown up". The idea is to find out if your newly developed course works at teaching the objectives you need to teach to the learners who need to learn them, before you present it to your target audience. In any given formative evaluation, you can find out how to make your instruction more:

•      Effective

•      Efficient

•      Interesting/Motivating

•      Usable

•      Acceptable

You do this by carrying out procedures that will provide you with evidence as to the effectiveness of your instruction. The emphasis is on collecting data and revising the instruction.  Formative evaluation of instructional materials is conducted to determine the effectiveness of the materials and to revise them in areas where they are ineffective. Formative evaluations should be conducted on newly developed materials as well as existing materials that are selected based on the instructional strategy. Evaluations are necessary for both mediated and instructor presented materials. The evaluations should be designed to produce data to pinpoint specific areas where the instruction is faulty and to suggest how it should be revised. An iterative process of formative evaluation containing at least three cycles of data collection, analysis, and revision is recommended. Each cycle focuses on different aspects of quality. The first cycle, one- to-one evaluation, is conducted to pinpoint gross errors in the materials. These errors typically relate to both the clarity of vocabulary, concepts, and examples used, and the motivational value of all five components of the instructional materials. Evaluations can also be conducted with content experts and individuals familiar with the characteristics of target learners. One- to- one evaluations must be conducted with representatives of the target population. An interactive interview process is used so the evaluator can learn what was wrong with the materials and why it was wrong.

Chapter 10 Vocabulary

 Designing and Conducting Formative Evaluations

Ø Formative Evaluation – Evaluation designed to collect data and information that is used to improve a program or product; conducted while the program is still being developed.

Ø Summative Evaluation - Evaluation designed and used after an instructional program has been implemented. The purpose is to present conclusions about the worth of the program or product and make recommendations about its adoption or retention.

Ø Subject-Matter Expert (SME) – A person knowledgeable about a particular content area.

Ø Performance Context – The setting in which in which it is hoped that learners will successfully use the skills they are learning; includes both the physical and social aspects of the setting.

Ø Field Trial - The third stage in formative evaluation, referring to the evaluation of the program or product in the setting in which it is intended to be used.

Chapter 9 Developing Instructional Materials Summary

The hallmark of individualized instruction is that many of the instruction events typically carried out by the instructor with a group of students are now presented to the individual student through instructional materials. The authors recommend that you produce self-instructional materials in your first attempt at instructional design, that is, the materials should permit the student to learn the new information and skills without any intervention from an instructor or fellow students.

          Translating your analysis and design opens opportunities to be creative with the instruction. Simply translating your analysis and strategies into prose can yield rather dry and boring instruction. As a teacher and instructional designer, you need to consider ways to add interesting and motivating approaches to the instruction. For example, consider the various introductions to the units used in this course. We have used a variety of styles and approaches to make the units interesting and appealing to the students. Making good instruction is part science and part art, the trick is finding the right mix to challenge and engage your students. As we mentioned, developing new materials "from scratch" is not a matter to be taken lightly. It is time-consuming, may involve the efforts of an entire team, and must be undertaken carefully and systematically. That said, however, developing instructional materials may be highly satisfying to you and your team, and the resulting programs or products serve as tangible evidence of your own skills and knowledge.

There is something exhilarating about seeing users interact with your own multimedia instruction, or get feedback from learners in a course you developed. The type of material you determine needs to be created will often times be determined by how you want the learner to encode the skills or knowledge. Is it something they need to embed in long-term memory? If so, then you'll need to create instructional materials. But sometimes it's more convenient, or more important, to give the learners aids to use to complete specific actions. In this case, the information doesn't need to be encoded deeply in long-term memory, but can be accessed from an external object, processed in short-term memory, and acted upon.  All this gratification assumes you have developed a successful educational or instructional program or product. Seldom will instructional designer be responsible for media production, but knowing the processes helps when communicating with media production specialists.

Each type of product, whether multimedia, print, video, or online, have major production processes and a group of specialized production people who carry them out. Each of the different forms of media demand both careful review of the preproduction documents, and careful evaluation during the production phase. Many instructional projects will incorporate some combination of instructor-led, multimedia, print, video, and/or online components. Practice working on them while in graduate school will better prepare you for job opportunities beyond the university. Although the creation of instructional materials will be time intensive, it will eventually come to an end. But that's not when the work of an instructional designer ends. Next you'll need to determine how the product will be implemented, and eventually evaluated.

Chapter 9 Developing Instructional Materials Vocabulary/Terms

1.  Rough Draft Materials- The development of instructional material in quick, inexpensive media formats for formative tryouts.

2.  Rapid Prototyping- In software development it is also called rapid application design (RAD) and is the process of using prototype approximations of a software design in order to test whether the application meets the design specifications.

3.  Feedback – Information provided to learners about the correctness of their responses to practice questions in the instruction.

4.  Instructional Analysis – The procedures applied to an instructional goal in order to identify the relevant skills and their subordinate skills and information required for a student to achieve.

5.  Instructional materials- print or other mediated instruction used by a student to achieve an instructional goal.

Chapter 9 Developing Instructional Materials Powerpoint

Developing Instructional Materials from cdjhaigler

Chapter 8 Developing an Instructional Strategy Powerpoint

Chapter 8 Developing an Instructional Strategy from cdjhaigler

Chapter 8 Summary



The first instructional strategies would be creating the right classroom climate. This is to assure that your students feel safe, secure and are engaged with their own learning. Learning decreases when students feel threatened or unchallenged (Marzano, et al, 1992). Have discussions with your students on how to maintain the right climate for your classroom. Have them help formulate the way they want the classroom to run and they will help your enforce everyone’s expectations.

The second instructional strategy would be your classroom plan. This will help you foster student involvement and cooperation in your classroom activities and will help establish a productive working environment. To establish a productive working environment you need to make your room stimulating and inviting. Classroom displays and arrangements must be functional and inviting. Your students will be more than happy to be in on the decision making process of how to make your classroom learner friendly.

The third instructional strategy would involve organizing your classroom. You will need to create room arrangements that allow for activities that require movement. Stations or centers can be created for long-term learning and involvement. These stations can be placed so that your students can easily find information, use computers, and access other technologies that would help with their learning.

The next instructional strategy involves developing routines and procedures. You will need to create a limited number of rules that are clear, specific, and stated in a positive manner. Always involve your students in the process of developing, understanding, and maintaining the routines and procedures. You will also need to practice and reinforce these routines and procedures throughout the school year.

The fifth instructional strategy would be assigning and managing work assignments. As a master teacher you need to provide meaningful and relevant assignments. Involve your students in real world scenarios that have a purpose. You will be amazed with your student’s abilities when they have an assignment that involves an audience and a real world purpose.

The next instructional strategy involves preparing for instruction. When your students are actively involved in their learning discipline problem will decrease. Keep your students involved in the planning and preparation of the units of studies. Make them a part of your on-going planning, implementation, and evaluation of units of study. Could your students compose a real proposal that could be submitted to a corporation, city council, or organization that needs fresh ideas?

The sixth instructional strategy is discussing behaviour in the classroom. Always communicate and reinforce class routines and procedures. Have your students enforce these and have classroom discussion on procedures that need to be added, deleted, or edited. You will be surprised how well your classroom will function when the students have a voice.

The last instructional strategy is to keep your classroom running smoothly throughout the school year. Always reflect and evaluate each day or week during the school year. Do not settle for less than your students’ best efforts. Always celebrate success!!!!!

These seven instructional strategies will help you become a successful teacher that students will remember forever. This will also help your students become lifelong learners.

Chapter 8 Developing an Instructional Strategy



CHAPTER 8 VOCABULARY

1.    Motivating Learners – One of the typical criticisms of instruction is its lack of interest and appeal to the learner.

2.  Prerequisite Skills – Another term used to describe entry skills.

3.  Pre-instructional Activities – Techniques used to provide the following three events prior to delivering instructional content:

o    get the learners’ attention

o   Advise them of the prerequisite skills for the unit

o   Tell them what they will able to do after the instruction

4.  Instructional Strategy - An overall plan of activities to achieve an instructional goal. The strategy includes the sequence of intermediate objectives and the learning activities leading to the instructional goals as well specification of student groupings, media, and the delivery system. The instructional activities typically include pre-instructional activities, content presentation, learner participation, assessment, and follow-through activities.

5.    Delivery System- The term used to describe the means by which instruction to facilitate its evaluation. The chart relates skills, objectives, and associated test items, allowing easy comparison among the components of the instructional design.

Chapter 7 Developing Assessment Instrument Powerpoint

Developing Assessment Instruments Chapter 7 from cdjhaigler

Chapter 7 Developing Assessment Instrument, Summary

         An “assessment” is not just a test, but a range of items used to gauge a learner’s abilities and progress as well as quality of instruction in addition to evaluating the instructional mediums used. Criterion (or the plural version of criteria) is defined as: Criterion. A standard on which a judgment or decision may be based learners will  a characterizing mark or trait answer 85% of quiz Criterion-referenced assessments, also known as objective questions reference assessments, focus on measuring performance items correctly to associated with learner performance and instructional integrity.

          Demonstrate mastery of Benchmarks, or specific levels of reference, are used to gauge the lesson change in performance, attitudes, and other measurable items. It is important to promote learners to evaluate themselves – quality of work and performance “Self-evaluation and self-refinement are two of the main goals of all instruction since they can lead to independent learning.”

          A criterion-referenced assessment is composed of items or performance tasks that directly measure skills described in one or more behavioral objectives. Leaner-centered assessments are to be criterion-referenced. This type of testing of test is important for evaluating both learners; a process and instructional quality.

Chapter 7 Developing Assessment Instruments, Terms/Phrases

1.    Goal-Centered Criteria- Test or item criteria used to analyze a goal to identify the sequence of operations and decisions required to achieve it.

 

2.    Criterion-Referenced – Items designed to measure performance on an explicit set of objectives; also known as objective-referenced test items.

3.    Context-Centered Criteria- Test or items criteria used to judge the congruence between the situations used in the assessments and the learning and performance contexts. Authenticity of examples and simulations is the main focus.

4.    Assessment-Centered Criteria- Test or items criteria used to judge item writing qualities such as grammar, spelling, punctuation, clarity, parsimony, and the use of recommended item formatting rules.

5.    Criterion- A standard against which performance or product is measured.