About Course Beta Testing#
To gather information about the overall experience of taking a course, you can recruit beta testers to test your course before it is available to general learners.
During a beta test, you gather information about the experience of taking your course in an effort to improve its quality. The phases of the beta testing process, and some questions that you may find useful to consider as you prepare for each phase, follow.
Planning: What are your objectives for the test? What is the schedule for testing?
Recruitment: How many testers do you need? How much time do you expect testers to spend? When do you need them to complete their feedback?
Provide access to your course: Can testers access your entire course immediately, or are its sections and subsections available at different times? How will that affect the schedule?
Collect feedback: How do you want testers to provide feedback? Who checks for feedback, and how often?
Evaluate feedback: How is feedback evaluated? Who decides what changes to make as a result, and on what schedule?
Conclusion: How do you acknowledge the efforts of your testers? What do you share with them about the results of their efforts?
There is no one answer to any of these questions. They are included here as background on the role that beta testing can play in the preparation of your course.
The Beta Testing Role#
Beta testers have early access to the course. Beta testers are not members of the course team: they do not have information about “how it is supposed to work”. They use their own computers and Internet connections to view videos, follow links, and complete problems. They interact with the course as learners will to find, and make, mistakes.
However, beta testers are not the same as other learners in the course, either. They have privileged access to the course and have more time to review and complete the course materials than the enrolled learners do. Course discussions are not open before the course start date, so beta testers cannot participate in community conversations. As a result of these differences, beta testers do not receive certificates when they complete a course.
Note
If one of your beta testers wishes to earn a certificate for the course, she must create a separate user account with a different username and email address. She can then use that separate, non-privileged user account to enroll in the course and repeat the work, completing assignments and exams when they are available to all learners.
Qualities of Good Beta Testers#
A beta test is valuable in part because it is unscripted. Your beta testers are not following a predetermined series of steps, or using a particular computer environment, when they access your course. When you recruit beta testers, however, you may find these skills and characteristics to be helpful.
Curiosity.
Attention to detail for identifying problems and inconsistencies.
Solid communication skills for reporting problems to the course team.
Your beta testers should also have varying levels of knowledge about the course content:
To recognize when material is wrong or missing, at least one tester should know the content well.
To identify material that is unclear, at least one tester should be less familiar with the content.
Depending on the objectives you set for your beta test, you might want to consider recruiting testers who use assistive technologies, who have different native languages, or who have varying levels of familiarity with computer software.
Using Course Team Members as Beta Testers#
The course team can provide valuable feedback about your course. However, they are typically stakeholders in the success of your course and have a significant amount of knowledge about it. As a result, they can be too close to the course to interact with it in the same way as learners will. They can also be either reluctant to provide feedback, or overly zealous.
If you do want a team member to be a beta tester, note that the privileges of the Staff or Admin role override those of a beta tester. The team member must use a different, second email address to create an additional account and enroll in the course, and the course team must assign only the beta tester role to that account. The course team member experiences the course as a learner only when she signs in using the beta tester account.
What to Test#
Beta testers should interact with everything in the course.
Select all links.
Watch all videos.
Download video transcripts and other files.
Complete all problems.
Beta testers can use a desktop computer to access your course website, the mobile app on a smartphone, or both to complete their tests. As they work, beta testers log issues and questions for the course team.
How Beta Testers See Course Content#
To beta test a course, you:
Define a number of days before content releases for testing to begin.
For instructor-paced courses, this number applies to the release date for each section and subsection. Beta testers can only access course content the number of days you specify before each section or subsection is released.
For self-paced courses, this number applies to the course start date. Because sections and subsections for self-paced courses do not have individual release dates, beta testers can access all course content when the beta test begins.
Identify beta testers.
Designated beta testers see course content before learners can. The matrix that follows shows the course content that beta testers can access earlier than other learners in an example instructor-paced course.
Yes |
No |
|
|---|---|---|
Before the course enrollment date |
X |
|
Before the course start date |
X |
|
Before the section release day |
X |
|
Before the subsection release day |
X |
|
Before the unit is published |
X |
|
Before a draft replaces a live unit |
X |
The course team can continue to add content in Studio after the beta test begins. When new content is ready for testing, be sure to publish the unit.
Important
When you set up the beta test and define when it starts, verify that the first section, subsection, and unit in your course are available to be tested. If that first unit is not available, an error occurs when testers attempt to access any other content in the course.
Maintenance chart
Review Date |
Working Group Reviewer |
Release |
Test situation |