Developing Multiple Choice Tests That Assess Higher Order Thinking Skills
The value of multiple-choice questions lies in their efficiency. Multiple-choice items are a relatively quick way to assess students’ understanding across a wide range of topics and instructors (or automated systems) can grade the items (and provide feedback) even faster. The challenge with multiple-choice items is that they historically target rote and application levels of learning. As such, their value has been criticized for failure to promote or assess higher-order thinking strategies. In this webinar, we will explore how to transform multiple-choice items to promote synthesis and evaluation of course information.
After attending this webinar, participants will be able to design and integrate multiple-choice items that assess higher order thinking skills.
With an increase in online and blended learning, it is important to examine student engagement and assessment strategies that can efficiently and effectively be incorporated into the virtual classroom. While multiple-choice assessment has been a mainstay of education for many years, creative integration of higher-order multiple-choice assessment offers the opportunity to target higher order learning objectives without burdening instructor workload. Multiple-choice items are widely used due to their versatility in assessing a range of learning objectives; well-constructed multiple-choice questions can target factual knowledge, comprehension, analysis, synthesis, or evaluation. Also, the objective nature of multiple-choice items ensures reliable, efficient scoring and allows for rapid feedback. While there are many benefits to utilizing multiple-choice items, the effectiveness of multiple-choice assessments rests on the quality of the test items and the appropriateness of this type of evaluation in relation to the learning objectives. Participants will learn how to evaluate the value of a multiple-choice test for meeting specific learning objectives, design high-quality, meaningful multiple-choice items, and utilize the outcome of multiple-choice assessments to enhance the teaching-learning dynamic. We will also explore how automated multiple-choice assessments can be used to promote student learning through mastery assessments, maximize instructional time in the classroom and improve student preparation for class activities.
- Evaluate the appropriateness of multiple-choice tests in relation to learning objectives
- Design effective, meaningful multiple-choice items
- Enhance the teaching-learning dynamic using the outcome of multiple-choice assessments
- Maximize instructional time in the classroom through automated multiple-choice tests
- Integrate multiple-choice assessments to improve students’ preparation for class activities
- Utilize multiple-choice items to engage students during class while simultaneously providing individualized formative feedback
- 2-year & 4-year institutions
- Academic Affairs/Instruction
- Faculty (full and part-time)
- Dean of Instruction & Department Chairs
- Online Learning Administrators
- Faculty who teach or plan to teach online/hybrid courses
- Online Learning Professionals
- Anyone interested in learning more about designing multiple choice tests & multiple choice assessments
“The value of multiple-choice assessments lies in their versatility; multiple-choice assessments can be used to promote students’ engagement with material prior to class, gauge understanding during classroom activities and measure understanding at the conclusion of a learning unit.”
B. Jean Mandernach, Ph.D. is Research Professor and Executive Director of the Center for Innovation in Research and Teaching at Grand Canyon University. Her research focuses on enhancing student learning through assessment and innovative online instructional strategies. In addition, she has interests in examining the perception of online degrees, the quality of online course offerings and the development of effective faculty evaluation models. Jean received her B.S. in comprehensive psychology from the University of Nebraska at Kearney, an M.S. in experimental psychology from Western Illinois University and Ph.D. in social psychology from the University of Nebraska at Lincoln.
Bio current as of March 2021.
What is a live webinar?
A live webinar is an interactive online training. Participants can communicate with the presenter(s) during the event via a live chat feature.
What is an on-demand webinar and how do I get access?
An on-demand training is a previously recorded webinar available online which can be accessed anytime, anywhere. You can register for on-demand trainings at any time. You will receive the recording the week following the live event. The recording is a campus access license and is available for one year from the purchase date.
How long are the webinars?
Webinars range from 60-90 minutes. Please check each training for the exact timeframe.
How do I register?
You can register online by adding the product to your shopping cart. You can also register by fax (1-866-508-0860), email (email@example.com), or mail by completing the paper-based registration form. The paper-based registration form is unique to each webinar and is available on each product page.
How do I request accommodations?
For captioning or any other accommodation, please contact us 7 days in advance. 303.955.0415 or firstname.lastname@example.org
When do I register?
You can register at any time, even the morning of the live event. If the live event has occurred, you can purchase the on-demand training (recording) of the session. You can register for on-demand trainings at any time.
What is the process for attending a live event?
The process is as follows:
- You will receive the login information within 5 minutes of registering for the webinar.
- The presentation materials will be available before the live event. Please check the link in the login information email.
- On the day of the live event, participants can log in 30 minutes prior to the start time. Once logged in, participants can see the PowerPoint slides, ask questions, and make comments via the chat feature.
- We email participants a link to the recording the Monday following the live event.
Is there a recording available? And how long is the recording good for?
Approximately one week after the conclusion of the live webinar, participants will receive a link to the recording which can be forwarded to all faculty and staff for viewing anytime, anywhere. The recording is a campus access license and is available for one year from the date of the live event.
What are the technical requirements?
Innovative Educators uses Zoom as its web conferencing provider. If you have not previously attended a Zoom event, please Join a Test Meeting to make sure your computer is compatible with Zoom. Be sure to complete this test prior to the live event.
What are the benefits of online training?
Cost-Effective: No travel required. Online Training is an innovative way to provide your entire faculty and staff with a variety of professional development opportunities for one low price! The more you train the more you save, as the registration fee is per institution, not per person.
Easy: You will receive a detailed list of instructions via email. If you run into any problems, we're always here to help.
Practical: Our training sessions focus on the most critical and relevant issues facing educators today. Our primary goal is to provide participants with the information, training, and skills necessary to immediately implement positive change at their institutions.
Top-Notch Presenters: Our presenters are subject matter experts and recognized in their field.
Value Added: When you purchase an online training, you also receive a campus-wide access license to the recording for one year. Faculty and staff can access it as often as they like from any location.
Satisfaction Guaranteed: Our online training is 100% guaranteed. If you are not satisfied, we will give you a credit for a future webinar or on-demand training of your choice (of equal or lesser value).
How can we use these trainings?
- Live: Promote and attend a live webinar and debrief immediately following.
- Hybrid: Distribute the recording to all faculty and staff at the beginning of each month and plan a discussion session at the end of the month to determine how you will implement the strategies presented.
- On-Demand: Distribute the recording to faculty and staff so they can watch anytime, anywhere.
In-Service Training: Plan an in-service around a live webinar or schedule a day and time to show the recording in a lecture hall or large conference room and invite faculty and staff to brainstorm and discuss implications for your institution.
Online Faculty and Staff Learning Communities: Distribute the recording to faculty and staff so they can watch anytime, anywhere.
Staff Recognition: Develop a program around the webinar with monthly themes and recognize the staff members that implement the best idea related to the theme.
Team-Building: Utilize these trainings to develop cross-functional and cross-discipline teams to foster collegiality.
New Employee Training: Include the online training as part of your new employee training program to ensure consistency.
Implementation and Follow-Up: Use the guide and evaluation materials provided by Innovative Educators to plan, implement, and track your progress.
How much does a live webinar or on-demand training cost?
1 Training (Live Webinar or On-Demand Training) - $425 Unlimited connections, campus-wide access license to the recording for one year.
- 2 Trainings - $645 Unlimited connections, recording for one year (campus-wide access)
- 3 Trainings - $900 Unlimited connections, recording for one year (campus-wide access)
- 6 Trainings - $1500 Unlimited connections, recording for one year (campus-wide access)
What type of payment do you accept?
You may pay with a credit card (American Express, MasterCard or Visa), PO, check, or electronic transfer of funds. You can email us (email@example.com), call 303-955-0415 or fax 1.866.508.0860.
Payment terms are net 90 days.
Where do I send payment?
Please mail checks and POs to our mailing address:
3277 Carbon Place
Boulder, CO 80301
What is your cancellation policy?
Below is a breakdown of our cancellation policy.
- 30 days prior: Full refund
- 14 days prior: $100 processing fee
- Fewer than 14 days: Credit toward another IE event. Please note that registration for the live webinar includes access to the recording which you can share, as it is a campus access license. We email the recording the Monday following the live event. For questions, please email us or call 303.955.0415.
What is included in the Go2K Membership?
All live Webinars and over 100 pre-selected, on-demand webinars.
Is the price for Go2K a one-time fee or an annual fee?
The purchase price for Go2K is an annual fee, which includes access to over 150 live events/year and 100+ on-demand trainings.
Are their certificates of completion available?
Your on-demand portal includes automated certificates of completion and can be printed or downloaded after completing the training. The live events do not include automated certificates. However, we can provide you a fillable certificate PDF to share with faculty and staff.
What kind of reporting is included?
For your on-demand trainings, we provide you with a monthly usage report as well as an evaluation report. Upon request, we can provide annual or bi-annual registration reports for any live orders.
How do I register for a live event?
After you purchase your Go2K membership, we will email you an institution specific password that can be used to register for any live event.
Can we share the registration coupon code with everyone at our institution?
Yes. Feel free to share the coupon code with anyone at your institution. Please do not share with anyone outside your institution. We recommend registering 24 hours in advance of the live event.
How do I access my on-demand trainings?
After you purchase your Go2K membership, we will email you a link to your on-demand portal that you can share with your entire institution.
How are colleges using Go2O?
Colleges are using Go2O in a variety of ways:
- As a stand-alone online orientation
- As an online complement to their face-to-face orientation
- As an online orientation for their distance learners
- As an online enrollment checklist to ease the registration process
Does purchase of Go2O include completion reports?
You can receive completion reports two ways:
- Automated delivery (ftp dump, email, webhook)
- Instant access via administrative login
What is the registration checklist and is it included in my Go2O purchase?
The online enrollment checklist is designed to walk students through your enrollment process ensuring they make it to the first day of class! The registration checklist is one of the modules available with Go2O. Based on your purchasing level, the module can be a selection or will be included.
Are there various pricing options available for Go2O?
Yes, we have 4 pricing levels available.
Is Go2O ADA compliant and responsive?
Go2O is ADA accessible and fully responsive on any mobile device, tablet, or computer.
How are colleges using SL?
- Career Readiness
- Learning Centers
- TRIO Programs
- IX Training
- IV Grants
- With Advisors & Counselors
- Extended Learning Institute
- FYE & Student Success Courses
- Online FYE Seminar
- Extra Credit
- Financial Aid Appeal
- College Level English Requirement
- Supplemental Instruction
- Target Online Learners
- Student Disability Office
- Writing Center
- Can be required by faculty
- Student Success Course
Which workshops are your top rated?
We take quality seriously, and so all of our workshops are top notch in terms of content and look. However, if we know how you plan to use StudentLingo (FYE, Retention, Probation, TRIO, etc.), we can make workshop recommendations.
Funding is a little tight, how many workshops can I get for $XX?
You can buy the entire StudentLingo package (50 workshops) or we offer several a la carte options. Unlike a lot of other software companies, we don’t charge per user, so we can work with any budget. We recommend talking to other departments on campus to increase your purchasing power and to share the cost. Also, we do have a pay model whereby students pay directly. If you are interested in learning more about this option, give us a call.
Is there an admin view?
This is a feature of our platform, but comes at an additional cost. With the admin view, you get access to more advanced reporting, and you have it in real time. With the standard membership, you get monthly reports.
How have other schools leveraged reporting?
- One of our partners, College of Southern Nevada, used StudentLingo to measure learning outcomes. in a study, one class had access to StudentLingo and the other cohort did not. 92% of the cohort with access to StudentLingo earned a whole letter grade higher than the cohort without StudentLingo.
- A lot of our partners use the data to justify funding for grants, operating dollars, or for new program development.
- The evaluation data is particularly useful in identifying what type of learning students prefer and the qualitative takeaways from viewing the workshop.
- Some of our partners use the data to connect with their student information system, like Banner or Peoplesoft.
- I think a great way to get started on how you want to use the data is to ask yourself some questions:
- How will I know if StudentLingo was successful?
- What do I hope students will gain from access to StudentLingo?
- What data is important for me to know in order to show that StudentLingo is a worthwhile purchase?
What is TutorLingo?
TutorLingo is a series of 9 online training modules that prepare tutors by providing the basic skills necessary to support their peers in becoming more independent and strategic learners.
Should the TutorLingo trainings be viewed in any particular order?
No, each training is a stand-alone workshop, and they can be viewed in any order.
What is the CRLA?
The CRLA is a group of student-oriented professionals active in the fields of reading, learning assistance, developmental education, tutoring, and mentoring. In 1989, the CRLA created a training program that is considered ‘best-practice’ for tutor training. A fews years ago, Innovative Educators partnered with the CRLA to create supplementary training modules. Our training supports learning centers in achieving level 1 certification through CRLA.
Do CRLA members receive a discount on TutorLingo?
Yes. CRLA members receive a 10% discount on TutorLingo.
Is there an admin view?
Yes. The admin view is an upgrade and costs $500, but gives you the ability to access completion data more efficiently and in real time. If you don’t want the admin view, we provide monthly usage reports.
What is ParentLingo?
ParentLingo is a go-to resource that institutions can easily share with parents to encourage student independence, growth & success. ParentLingo is a shared resource that recognizes and includes parents as partners in the educational process.
What is the cost?
Please contact us for pricing information. Email firstname.lastname@example.org or 303-955-0415.
What features are included?
- 8 Targeted Modules For Parents Branding (Logo On Every Page)
- ADA Compliant & Responsive Design
- Promotional Materials
- Registration (First Name, Last Name, Email)
- 8 Targeted Modules For Parents Branding (Logo On Every Page)
- ADA Compliant & Responsive Design
- Promotional Materials
- Completion Reports & Standard Evaluations
- Custom Branding (Logo & Institutional Photos)
- 3 Additional Registration Points
- Add & Edit Custom Content
- Customizable Evaluations & Surveys Advanced Reporting
- Designated Instructional Designer
- Flexible Pricing (College Payment Option Or Parents Pay Directly)