September 2020 Member Meeting
Written by Sarah Pauling, Examity
Even prior to the COVID pandemic, the IT credentialing industry was an early adopter of online proctoring. During 2020, the broader assessment industry, including many more IT programs had to make the shift at a rapid pace to continue serving candidates. With some of the initial rush behind us, and an increased focus on new approaches and technologies, it’s time to step back, take a deep breath, and evaluate what we have learned and where we are headed. Online proctoring is not a short-term solution, rather, it is here to stay. For programs looking to scale as well as establish a redundancy measure per today’s new normal, online proctoring is a necessity. That’s why it’s critical to understand best practices for introducing this as a solution within your assessment programs.
The ITCC September 2020 Member Meeting featured a panel of three seasoned assessment professionals from Adobe, Caveon, and Examity. During the discussion, many timely and important themes were discussed, including improving the candidate experience, prioritizing security, identifying threats to online proctoring, and projections on what the future holds. Below is a recap of what was discussed. For more information about the panelists, please see “About the Panelists” below.
Test-Taker Experience
Online proctoring is a new experience for many test-takers, so it’s important to set expectations early and often. During the discussion, panelists recommended best practices such as walking the candidate through the testing process, frequent communication prior to the test date, and presenting a practice test so candidates can get a feel for what the experience will be like.
Systems checks prior to testing are a great way to confirm your candidates are set up for success. For instance, test-takers may not be aware that the proctoring solution could potentially have limitations due to existing firewalls. In addition, we have to allow more flexibility to the test-taker and consider factors such as geographic location and internet speeds.
Security and Threats
The importance of keeping the testing process secure for those involved was evident in all of the panelists’ remarks. It is crucial to balance security with the test-taker experience. When designing an exam, you must take charge of the security, as one panelist pointed out. Don’t just rely on the proctoring solution to ensure the safety of your exam.
It’s advantageous to identify potential security threats, so that you can eliminate them before cheating occurs. These threats include the use of hidden cameras, memorizing and sharing test content, proxy testing with fake IDs, and rogue proctors who can capture test content. By thoroughly vetting and selecting the right online proctoring solution, you can mitigate these threats, but you also need to take additional steps. For example, instituting oaths, threats of legal actions, and a formal test-taker agreement can be useful deterrents. In addition, introducing data forensics to detect content breaches as well finding and destroying breached test content on the web should be practices you consider building into your program.
Benefits
The benefits of online proctoring were discussed by all panelists with a heavy focus on scale and resulting program growth. For tests with high volume, scalability is something that needs to be strongly considered. In order to provide scalable solutions, test sponsors need to make sure they have the right solution in place. Online proctoring offers the quickest, fastest way to scale. It was also noted that moving to online proctoring helps reduce program costs.
Future
There was a strong sense of optimism about the future of online proctoring. It is definitely here to stay, and, in the long-term, this form of testing will continue to be even more effective and secure. To make this happen, online proctoring companies must adapt to support current IT certification efforts, not undermine them. And IT professionals need to continue to design tests that effectively implement online proctoring into their ecosystems.
Overall, the panelists did an outstanding job explaining their learnings over the past eight months and laying out a roadmap for the future. It is clear to see that IT professionals and online proctoring companies are working together to make sure that test-taking remains safe, secure, and effective.
Members can log in to access the session recording.
About the Panelists
Vice President, Professional Business Development at Examity
Over my 26-year career in assessment I have held market‐facing leadership positions with Pearson VUE, IJCAHPO, BrightLink, and now Examity, where I get to be at the forefront of the migration to online proctoring.
Through my various roles I have had the honor of serving or being part of credentialing programs across markets, geographies, and missions. I have implemented, managed, and grown programs in healthcare, IT, finance, trades, admissions, federal agencies, and many other markets, for administration domestically and globally. I have gotten to know the missions, attributes, and challenges of a wide variety of assessment organizations, including many of those represented in ITCC. I have volunteered with a variety of industry associations, including ICE, ATP, PTC, and CNG. For ICE I have spoken at Exchange, served on the Program Committee, and more recently participated on the branding committee. For ATP I have helped lead the Certification/Licensure division, been the Vice Chair and Chair of the Innovations conference, and served on the board. I am finishing my second term on the board of CNG.
IT credentialing programs are unique in the assessment industry in terms of reach, commitment to innovation, and volume of credentials offered. I believe ITCC plays a critical role in helping IT credentialing programs collaborate to implement best practices, leverage the latest approaches to test design and delivery, and push each other to drive needed innovation. I’m delighted to be a part of ITCC and contribute however I can to the community.
An experimental psychologist and psychometrician, David has spent more than 35 years in the measurementindustry. From 1990-1997, David directed the certification test development efforts at Novell. As part of the the company’s pioneering IT certification program, Dave is credited with introducing computerized adaptive testing, simulation-based performance testing, and tests provided in dual languages. Following his tenure with Novell, he co-founded Galton Technologies to provide test development technology and services to certification programs.
In 2003 David co-founded the industry’s first test security company, Caveon. Under David’s guidance, Caveon has been unwavering in its dedication to promoting validity and fairness in testing, and has created new security tools, analyses, and services to protect its clients’ exams.
David has also served on numerous boards and committees in the measurement arena. He is a past President of the Association of Test Publishers (ATP), has served on the Personnel Certification Accreditation Committee and Board of Directors for the American National Standards Institute, and, for several years, participated as a member of guiding council for the International Test Commission. He has also sat on technical advisory boards and committees for various testing programs and companies. In 2002 he founded the Performance Testing Council with the mission of promoting test designs that more closely measure the skills used on the job or in the classroom. David has authored numerous articles for industry publications and scientific journals, as well as a textbook on experimental design. He has contributed several chapters and articles on test security and has presented extensively at industry conferences, both nationally and internationally.
David holds a Ph.D. in Experimental Psychology from Brigham Young University. He completed a post-doctoral fellowship in Psychobiology at Florida State University and has served on the faculties of Brigham Young University, Weber State University, and Western Carolina University. His background in research and innovation inspires him to apply a scientific approach to the problems of testing while mingling a healthy skepticism of current best practices with a scientific approach to new solutions.
This outlook has led David to create and implement a number of innovations during his career, including the first large-scale global use of computerized adaptive testing and the effective use of simulations in tests. In addition, he has pioneered several new technology-based security solutions such as the introduction of online proctoring and improvements to selected-response formats, including the Discrete Option Multiple Choice (DOMC) and SmartItems. And with his colleagues at Caveon, has conducted research supporting their value and use. After 36 years in the testing industry, Dr. Foster has not slowed down, and no doubt has a few more contributions to make.
Tyler Tu has 25 years of experience in Education and Certification, and is currently heads the Credential and Accreditation programs at Adobe.
Tyler’s career started with Hewlett-Packard back in 1993. As an instructor, he was involved as a subject matter expert and helped build the first HP Certified NetServer certification. Tyler Joined the ExpertOne Certification program in 2002 and held various leadership positions while he was there. He was a part of the HP and Compaq certification integration merger and led the consolidation of multiple delivery and credential management systems to support a portfolio of over 150 certifications. Tyler worked at Western Governors University from 2013-2018 as a Senior Manager, where he ran the departmental exam development operations.
In 2018, Tyler joined the Adobe Certified Expert certification program. Today, Adobe has 36 certifications across three levels with job roles in Business Practitioner, Developer, and Architect. By 2020, the program grew 85% YoY in new certifications awarded, with a 125% increase in exam delivery.
Tyler is an active member of ITCC, ATP, and CEdMA.
