Evaluation

11. Describe the purpose of formative evaluation.

           

A Formative Evaluation aims to assist instructional designers and course creators in tracking how learners progress throughout the course. Formative evaluations are also used to provide feedback to instructional designers or instructors to improve the learning experience for learners. Formative assessments are performed during instructional design. One-to-one formative evaluation involves an instructional designer or educator and three to five learners that will represent the target audience of the projected learners. The designer will pick one learner above the lesson's ability average, one at the standard, and one below the ability average to represent a range of the target learners. The goal is for those learners to check for clarity, impact, and feasibility and provide feedback to the instructional designer. As the designer, you must make the learners feel comfortable and encourage them to discuss the materials. "The first critical hallmark of the one-to-one formative evaluation is its almost total dependence on the ability of the designer to establish rapport with the learner and then to interact effectively (Dick, Carey, Carey, 2022). Simply do not just hand material to your learner but go over the material with them, working one-on-one with each learner. After the learner has finished material, it is time for the designer to start asking questions after each item in the assessment on why they made the selection they did; this can help eliminate any items that are not clear and are defective. Next, you will need to evaluate text directions and rubrics with your learners, again taking anything that isn't very clear or does not belong to the material you are testing. Another advantage to the one-to-one formative evaluation is that it will help to determine how much time you will need to spend on each section of the materials your learners are working on. After analyzing the feedback and data from the student learners, a conclusion must be drawn on the amount of precise information that the instruction provides, its impact on the learner, and the feasibility of the instruction. Suppose certain aspects of the instruction are deemed to be ineffective. In that case, these aspects can be reconsidered for the purposes of planning revisions likely to result in improved instruction for students in a similar situation.

 

11.1 Propose a plan for the one-to-one formative evaluation stage.

               

The setting is a business lab at a local high school where students take the entrepreneurship class. This will be the first year the learners will be taught this field of study. My job as the designer and educator is to produce the most effective, content-rich curriculum and lessons I can create! I will conduct a  one-to-one formative evaluation with three learners, whom I will meet with one at a time. I have selected three types of students for this evaluation. The first student, X, is a gifted student who finishes their work early and is always eager to help other students. This is the type of student who would teach the class if allowed. The second learner, Y, is below the criterion standard of 70% on his mastery and learning goals in this class and will ask, why do I have to do this assignment? The third learner, Z, falls somewhere between the scale of learners X and Y. They are neither hot nor cold about the subject. I am pleased that all three students have agreed to be a part of my one-to-one formative elevation. Each learner is evaluated at different times, making this a one-to-one evaluation.

First, I thank my learners for helping me with this formative evaluation. I let them know that the work they are putting in now will help ensure that future entrepreneurship students are receiving the most effective and appropriate materials for the learning objectives. I want the learners to feel comfortable, so I offer them a water bottle or a snack. We chat briefly about school, sports, work, etc., while the learners finish their snacks. Now I tell them, this is where the real fun begins. You will ease the working out of any kinks, misunderstandings, or irrelevant material in the educational materials the class will use. I am looking for your feedback throughout this evaluation; specifically, I am looking for feedback on the learning materials' clarity, impact, and feasibility. The learners are eager and ready to start, well, most of them!

Learners are instructed to log in to their computers and pull up the district's local LMS, Canvas. Learners then access the Entrepreneurship and Small Business class. This study will only examine module one, All About Entrepreneurs! I start by explaining the concept behind the class to teach students what it takes to run and operate their own business to make a profit with their business. Then, I  start the lesson on entrepreneurship using the motivation type, attention, in the form of a question asking the learners if they know anyone who owns their own business. The learners give various answers, depending on whether they know anyone who owns a business. I then provided an example of someone I know who owns their own business as a photographer/visual media specialist. After that fun little discussion, I give an overview of what the learners will be learning by briefly discussing a few key terms of the chapter, such as Entrepreneur and Entrepreneurship. I then explain to the learners that the module is scaffolded in a systematic way that will guide them through the content effectively and efficiently. I then give the learners an overview of the module and briefly discuss the contest of the module. I point out the module's layout and review each set of instructions for each assessment with the learners. Starting at the top of the module and working our way down. The first assessment starts with a reading assignment, practice tests, research tasks, discussion, an entrepreneur inventory assessment, and finally, wrapping up the module with the post-test and a follow-through assignment in the form of the learners creating a visual aid in the form of a SmartArt. I inform the learners that they are to continue to each lesson in the module until they come to the discussion. We will do that together! I ask the learners if they have any questions, but no questions are asked. I then asked the learners to access their E-book, Entrepreneurship Ideas in Action 6E, linked to the book's name in module one. As students are doing that, I ask them to please select the second link in the module, a feedback questionnaire. I explain to the learners that I have permission to see what they are typing in real time but do not have permission to edit the document. collaboration, if you will. I reviewed how the questionnaire is set up, ranking one as the lowest and five as the highest rank they could give the task. I point out to the learners that there is also a text entry for each question where they can provide detailed information that they want me to know about the materials and if there are any concerns or if the text is confusing or hard to understand what I am asking them to do. I also ask them to please type it and let me know verbally when they encounter an issue or need to ask a question or say something in general. Learners acknowledge that they will highlight the appropriate number for each task and leave written feedback when called for. I encourage learners they can leave positive and negative comments about their learning, what went well, and what went wrong. The first question asks students for feedback and ease of use on getting to the e-book. The learners highlight five across the board. The next task learners get started on is the reading activity, the first assignment in the module. The learners will use evidence-based text strategies to highlight critical terms from the curriculum while reading independently for fifteen minutes. Right away, I noticed learner Z typing in what went wrong. I asked the learner to please point out the issue that they were having. The simple reply was, where is the highlighter? I then point out the tools panel to the right of the screen with the highlighter, flashcards, and notes section. I immediately modify the assignment in Canvas to include that in the instructions of the assignment. In addition to the assignment reads, students will have access to a highlighter, notes, and flashcards in your toolbar in the right-hand corner of your e-book. After the learners have completed highlighting key text, they move to the next assignment in the module. The learners are multitasking. On one side of their screen, they have Canvas open going through the module, and on the other side of the screen, they have their questionnaire open. They are typing and highlighting anytime they see the need to inform me of something they do not like. They also are typing things they are enjoying as well. Learners are now taking their pretests; I have created a pretest using Canvas Quiz templates. The practice test gives the learners immediate feedback for the learner. This enables the learner to review the terms with the correct definitions if they have missed any. I notice learner Y making faces at the screen. I ask learner Y to express his feelings. The learner states that he hates using Canvas to take a Quiz. The learner states he would rather use Kahoot or Gimkit. Learner Y states that he learns better when he has game simulations. I asked him to put what went wrong in the practice test section in his feedback section. I also make a note of it and immediately Google Gimkit. It is exactly as learner Y states: learning in-game simulation! I quickly make an account while learner Y is wrapping up his practice test. The next assignment in the module has the learners utilizing a web search engine and Microsoft Word to research and list out in bullet form at least eight characteristics of a successful entrepreneur. To my surprise, learners produced a list of at least ten characteristics of a successful entrepreneur. I noticed this assignment had fives across the board, which is excellent! Learners are now taking their pretests with immediate feedback. Again, learner Y states he is unhappy with the format I chose for the pretest. I reassured student Y that I had already created an account with Gimkit and that I would be creating some practice tests for the class to do in the future for practice test. Although for the pre and post-test, I will be utilizing Canvas to ensure quality control over the settings and tools that can be used during the test. Learner Y nods and says he guesses one out of three isn't so bad and that he appreciated my taking his feedback into consideration and was happy to hear that I would implement this in the future of this class. Learners are now accessing the assignment named career inventory. The assignment reads: Learners will use classroom computers, the internet, and the link provided to access their interests and aptitudes by taking an entrepreneur inventory assessment. Learners are instructed to save their results to their OneDrive on their computers. During the next formative assessment, students will demonstrate their understanding of a successful entrepreneur's eight characteristics by participating in a group discussion. Of course, for this evaluation, it will be a one-on-one discussion. Learners X, Y, and Z all start the discussion off. They start by talking about the difference between and entrepreneur and an entrepreneurship. Leaner Z states that running their own business seems like something they might want to do! I asked Learner Z to elaborate, and Learner Z replied, I already have a few of the characteristics to be a successful entrepreneur. I think I could do it! I like the idea of being my own boss! I might even want to work by myself! Learners X and Y also do well with this discussion, and I've received two fives and one four for this assignment. I then asked the learner who rated this assignment a four why they did so. The learner let me know they feel uncomfortable talking in large groups. They told me they would feel better if this assignment could be broken down to talk with your nearest neighbor. I like this suggestion. It ensures that no one learner gets left out of a discussion. I quickly opened the assignment in Canvas and amended it to state that learners will turn to their nearest neighbor and discuss their findings on the characteristics required to be a successful entrepreneur. Learners are now on the formative assessment of the module, a post-test with immediate feedback for the students and the teacher to know what areas they have mastered and what areas might need a little more time spent on. Learners X, Y, and Z all score above 80% on the post-test. I am pleased with the results, and so are the learners, as I have just received a five across the board for this post-test with a comment from learner X: The post-test covered the materials learned in the module. Thank you for not adding anything to the post-test not provided in the instruction. I noted that the post-test looks good and will not need any changes. Learners are now on the final assignment of the module, the follow-through assignment, where they will open Microsoft Word, create a list of eight characteristics of a successful entrepreneur, and convert it into a SmartArt, a memory aid they can reference throughout the course. Learners end their feedback with a short response on how they felt about the material, the instruction layout, ease of use, and the feasibility of how likely they will use this instruction inside and outside the classroom. As the designer and instructor, I will look over the feedback and apply any more necessary changes that need to be made to the curriculum or materials. Creating a live feedback document where the instructional designer can monitor the feedback while students give it allows for changes to be made quickly and effectively.