Developing Multiple Choice Tests That Assess Higher Order Thinking Skills
The value of multiple-choice questions lies in their efficiency. Multiple-choice items are a relatively quick way to assess students’ understanding across a wide range of topics and instructors (or automated systems) can grade the items (and provide feedback) even faster. The challenge with multiple-choice items is that they historically target rote and application levels of learning. As such, their value has been criticized for failure to promote or assess higher-order thinking strategies. In this webinar, we will explore how to transform multiple-choice items to promote synthesis and evaluation of course information.
After attending this webinar, participants will be able to design and integrate multiple-choice items that assess higher order thinking skills.
With an increase in online and blended learning, it is important to examine student engagement and assessment strategies that can efficiently and effectively be incorporated into the virtual classroom. While multiple-choice assessment has been a mainstay of education for many years, creative integration of higher-order multiple-choice assessment offers the opportunity to target higher order learning objectives without burdening instructor workload. Multiple-choice items are widely used due to their versatility in assessing a range of learning objectives; well-constructed multiple-choice questions can target factual knowledge, comprehension, analysis, synthesis, or evaluation. Also, the objective nature of multiple-choice items ensures reliable, efficient scoring and allows for rapid feedback. While there are many benefits to utilizing multiple-choice items, the effectiveness of multiple-choice assessments rests on the quality of the test items and the appropriateness of this type of evaluation in relation to the learning objectives. Participants will learn how to evaluate the value of a multiple-choice test for meeting specific learning objectives, design high-quality, meaningful multiple-choice items, and utilize the outcome of multiple-choice assessments to enhance the teaching-learning dynamic. We will also explore how automated multiple-choice assessments can be used to promote student learning through mastery assessments, maximize instructional time in the classroom and improve student preparation for class activities.
- Evaluate the appropriateness of multiple-choice tests in relation to learning objectives
- Design effective, meaningful multiple-choice items
- Enhance the teaching-learning dynamic using the outcome of multiple-choice assessments
- Maximize instructional time in the classroom through automated multiple-choice tests
- Integrate multiple-choice assessments to improve students’ preparation for class activities
- Utilize multiple-choice items to engage students during class while simultaneously providing individualized formative feedback
- 2-year & 4-year institutions
- Academic Affairs/Instruction
- Faculty (full and part-time)
- Dean of Instruction & Department Chairs
- Online Learning Administrators
- Faculty who teach or plan to teach online/hybrid courses
- Online Learning Professionals
- Anyone interested in learning more about designing multiple choice tests & multiple choice assessments
“The value of multiple-choice assessments lies in their versatility; multiple-choice assessments can be used to promote students’ engagement with material prior to class, gauge understanding during classroom activities and measure understanding at the conclusion of a learning unit.”
B. Jean Mandernach, Ph.D. is Research Professor and Executive Director of the Center for Innovation in Research and Teaching at Grand Canyon University. Her research focuses on enhancing student learning through assessment and innovative online instructional strategies. In addition, she has interests in examining the perception of online degrees, the quality of online course offerings and the development of effective faculty evaluation models. Jean received her B.S. in comprehensive psychology from the University of Nebraska at Kearney, an M.S. in experimental psychology from Western Illinois University and Ph.D. in social psychology from the University of Nebraska at Lincoln.
Bio current as of March 2021.
What is a live webinar?
A live webinar is an interactive online training. Participants can communicate with the presenter(s) during the event via a live chat feature.
What is an on-demand webinar and how do I get access?
An on-demand training is a previously recorded webinar available online which can be accessed anytime, anywhere. You can register for on-demand trainings at any time. You will receive the recording the week following the live event. The recording is a campus access license and is available for one year from the purchase date.
How long are the webinars?
Webinars range from 60-90 minutes. Please check each training for the exact timeframe.
How do I register?
You can register online by adding the product to your shopping cart. You can also register by fax (1-866-508-0860), email (firstname.lastname@example.org), or mail by completing the paper-based registration form. The paper-based registration form is unique to each webinar and is available on each product page. If you are a Go2Knowledge member, you register on your personal dashboard.
Are webinars captioned?
We automatically caption all of our live webinars.
When do I register?
You can register at any time, even the morning of the live event. If the live event has occurred, you can purchase the on-demand training (recording) of the session. You can register for on-demand trainings at any time.
What is the process for attending a live event?
The process is as follows:
- You will receive the login information within 5 minutes of registering for the webinar.
- The presentation materials will be available before the live event. Please check the link in the login information email.
- On the day of the live event, participants can log in 30 minutes prior to the start time. Once logged in, participants can see the PowerPoint slides, ask questions, and make comments via the chat feature.
- We email participants a link to the recording the Monday following the live event.
Is there a recording available? And how long is the recording good for?
Approximately one week after the conclusion of the live webinar, participants will receive a link to the recording which can be forwarded to all faculty and staff for viewing anytime, anywhere. The recording is a campus access license and is available for one year from the date of the live event.
What are the technical requirements?
Innovative Educators uses Zoom as its web conferencing provider. If you have not previously attended a Zoom event, please Join a Test Meeting to make sure your computer is compatible with Zoom. Be sure to complete this test prior to the live event.
What are the benefits of online training?
Cost-Effective: No travel required. Online Training is an innovative way to provide your entire faculty and staff with a variety of professional development opportunities for one low price! The more you train the more you save, as the registration fee is per institution, not per person.
Easy: You will receive a detailed list of instructions via email. If you run into any problems, we're always here to help.
Practical: Our training sessions focus on the most critical and relevant issues facing educators today. Our primary goal is to provide participants with the information, training, and skills necessary to immediately implement positive change at their institutions.
Top-Notch Presenters: Our presenters are subject matter experts and recognized in their field.
Value Added: When you purchase an online training, you also receive a campus-wide access license to the recording for one year. Faculty and staff can access it as often as they like from any location.
Satisfaction Guaranteed: Our online training is 100% guaranteed. If you are not satisfied, we will give you a credit for a future webinar or on-demand training of your choice (of equal or lesser value).
How can we use these trainings?
- Live: Promote and attend a live webinar and debrief immediately following.
- Hybrid: Distribute the recording to all faculty and staff at the beginning of each month and plan a discussion session at the end of the month to determine how you will implement the strategies presented.
- On-Demand: Distribute the recording to faculty and staff so they can watch anytime, anywhere.
In-Service Training: Plan an in-service around a live webinar or schedule a day and time to show the recording in a lecture hall or large conference room and invite faculty and staff to brainstorm and discuss implications for your institution.
Online Faculty and Staff Learning Communities: Distribute the recording to faculty and staff so they can watch anytime, anywhere.
Staff Recognition: Develop a program around the webinar with monthly themes and recognize the staff members that implement the best idea related to the theme.
Team-Building: Utilize these trainings to develop cross-functional and cross-discipline teams to foster collegiality.
New Employee Training: Include the online training as part of your new employee training program to ensure consistency.
Implementation and Follow-Up: Use the guide and evaluation materials provided by Innovative Educators to plan, implement, and track your progress.
How much does a live webinar or on-demand training cost?
1 Training (Live Webinar or On-Demand Training) - $425 Unlimited connections, campus-wide access license to the recording for one year.
- 3 Trainings - $900 Unlimited connections, recording for one year (campus-wide access)
- 6 Trainings - $1500 Unlimited connections, recording for one year (campus-wide access)
- 12 Trainings - $2200 Unlimited connections, recording for one year (campus-wide access)
What type of payment do you accept?
You may pay with a credit card (American Express, MasterCard or Visa), PO, check, or electronic transfer of funds. You can email us (email@example.com), call 303-955-0415 or fax 1.866.508.0860.
Payment terms are net 90 days.
Where do I send payment?
Please mail checks and POs to our mailing address:
3277 Carbon Place
Boulder, CO 80301
What is your cancellation policy?
Below is a breakdown of our cancellation policy.
- 30 days prior: Full refund
- 14 days prior: $100 processing fee
- Fewer than 14 days: Credit toward another IE event. Please note that registration for the live webinar includes access to the recording which you can share, as it is a campus access license. We email the recording the Monday following the live event. For questions, please email us or call 303.955.0415.
What is included in the Go2K Membership?
Go2Knowledge is your turnkey solution for ongoing professional development in higher education. Your yearly membership includes unlimited access to approximately 150 live webinars and over 350 of our best on-demand training videos. Go2Knowledge can be customized in numerous ways (add your videos, curate our content, custom tracks, etc.) to meet your professional development needs. If you are interested in a customized option, please schedule a call.
Is the price for Go2K a one-time fee or an annual fee?
The purchase price for Go2K is an annual fee, which includes access to approximately 150 live events/year and 350+ on-demand training workshops.
Are certificates of completion available?
If you have a Go2K membership, your personal dashboard includes automated certificates of completion and can be printed or downloaded after completing the training. You also have a certificate summary that acts as a transcript for all of your completed training. If you do not have a Go2K membership, you can request a certificate of completion from firstname.lastname@example.org.
What kind of reporting is included?
We provide a monthly usage and evaluation report to the designated admin. Upon request, we can provide annual or bi-annual reports. Additionally, designated college administrators have full access to pull reports in real time.
How do staff members access our Go2K site?
After you purchase Go2K, we create a college site and send your specific Go2K url which you can distribute to everyone at the institution. College personnel then simply create a user login to access their personal dashboard. From the dashboard, they can search for specific training topics, register for upcoming live webinars, and view/print certificates of completion.
How do staff members access the on-demand training videos?
All on-demand training is available in a user’s personal dashboard. You can search for specific titles or use the catalog feature to find topics.
Can the Go2Knowledge platform be customized to include college created content or to create learning paths?
Yes. The Go2K platform is fully customizable. You can add your content, curate our content, and/or creating learning paths. This is a custom upgrade and comes at an additional cost. To learn more about custom options, please schedule a demo.
How are colleges using Go2O?
Colleges are using Go2O in a variety of ways:
- As a stand-alone online orientation
- As an online complement to their face-to-face orientation
- As an online orientation for their distance learners
- As an online enrollment checklist to ease the registration process
- As an online orientation for specific groups (TRIO, Transfer, International, etc.)
Does purchase of Go2O include completion reports?
You can receive completion reports two ways:
- Automated delivery (ftp dump, email, webhook)
- Instant access via administrative login
What is the registration checklist and is it included in my Go2O purchase?
The online enrollment checklist is designed to walk students through your enrollment process ensuring they make it to the first day of class! The registration checklist is one of the modules available with Go2O. Based on your purchasing level, the module can be a selection or will be included.
Are there various pricing options available for Go2O?
Yes, we have 4 pricing levels available.
Is Go2O ADA compliant and responsive?
Go2O is ADA accessible and fully responsive on any mobile device, tablet, or computer.
How are colleges using SL?
- Career Readiness
- Learning Centers
- TRIO Programs
- IX Training
- IV Grants
- With Advisors & Counselors
- Extended Learning Institute
- FYE & Student Success Courses
- Online FYE Seminar
- Extra Credit
- Financial Aid Appeal
- College Level English Requirement
- Supplemental Instruction
- Target Online Learners
- Student Disability Office
- Writing Center
- Can be required by faculty
- Student Success Course
Which workshops are your top rated?
We take quality seriously, and so all of our workshops are top notch in terms of content and look. However, if we know how you plan to use StudentLingo (FYE, Retention, Probation, TRIO, etc.), we can make workshop recommendations.
Funding is a little tight, how many workshops can I get for $XX?
You can buy the entire StudentLingo package (50 workshops) or we offer several a la carte options. Unlike a lot of other software companies, we don’t charge per user, so we can work with any budget. We recommend talking to other departments on campus to increase your purchasing power and to share the cost. Also, we do have a pay model whereby students pay directly. If you are interested in learning more about this option, give us a call.
Is there an admin view?
This is a feature of our platform, but comes at an additional cost. With the admin view, you get access to more advanced reporting, and you have it in real time. With the standard membership, you get monthly reports.
How have other schools leveraged reporting?
- One of our partners, College of Southern Nevada, used StudentLingo to measure learning outcomes. in a study, one class had access to StudentLingo and the other cohort did not. 92% of the cohort with access to StudentLingo earned a whole letter grade higher than the cohort without StudentLingo.
- A lot of our partners use the data to justify funding for grants, operating dollars, or for new program development.
- The evaluation data is particularly useful in identifying what type of learning students prefer and the qualitative takeaways from viewing the workshop.
- Some of our partners use the data to connect with their student information system, like Banner or Peoplesoft.
- I think a great way to get started on how you want to use the data is to ask yourself some questions:
- How will I know if StudentLingo was successful?
- What do I hope students will gain from access to StudentLingo?
- What data is important for me to know in order to show that StudentLingo is a worthwhile purchase?
What is TutorLingo?
TutorLingo is a flexible tutor training plaform and can be purchased at the Bronze, Silver or Gold level to meet your training needs. Every level consists of a series of 9 Core online training modules that prepare tutors by providing the basic skills necessary to support their peers in becoming more independent and strategic learners. The Silver & Gold purchasing options include TLNow training and our Additional Level 1 or Level 2 training topics. See our pricing guide for details.
Should the TutorLingo trainings be viewed in any particular order?
No, each training is a stand-alone workshop, and they can be viewed in any order.
What is the CRLA?
The CRLA is a group of student-oriented professionals active in the fields of reading, learning assistance, developmental education, tutoring, and mentoring. In 1989, the CRLA created a training program that is considered ‘best-practice’ for tutor training. A fews years ago, Innovative Educators partnered with the CRLA to create supplementary training modules. Our training supports learning centers in achieving level 1 certification through CRLA.
Do CRLA members receive a discount on TutorLingo?
Yes. CRLA members receive a 10% discount on TutorLingo.
Is there an admin view?
Yes. The admin view is an upgrade and costs $500, but gives you the ability to access completion data more efficiently and in real time. If you don’t want the admin view, we provide monthly usage reports.
What is ParentLingo?
ParentLingo is a go-to resource that institutions can easily share with parents to encourage student independence, growth & success. ParentLingo is a shared resource that recognizes and includes parents as partners in the educational process.
What is the cost?
Please contact us for pricing information. Email email@example.com or 504-206-6585.
What features are included?
- 8 Targeted Modules For Parents Branding (Logo On Every Page)
- ADA Compliant & Responsive Design
- Promotional Materials
- Registration (First Name, Last Name, Email)
- 8 Targeted Modules For Parents Branding (Logo On Every Page)
- ADA Compliant & Responsive Design
- Promotional Materials
- Completion Reports & Standard Evaluations
- Custom Branding (Logo & Institutional Photos)
- 3 Additional Registration Points
- Add & Edit Custom Content
- Customizable Evaluations & Surveys Advanced Reporting
- Designated Instructional Designer
- Flexible Pricing (College Payment Option Or Parents Pay Directly)