Presentation by Marcus Robertson and Frederick Mendler, founders of TrueAbility
Written by Gerry Mimick, Pearson VUE
The ITCC June Member Meeting featured guest speakers, Marcus Robertson and Frederick “Suizo” Mendler from TrueAbility, who presented the topic of Solving Latency Challenges with Performance Based Assessment Delivery.
Marcus Robertson and Frederick Mendler founded TrueAbility to solve the problem of determining an IT professional’s “true ability.” Working previously at a large global hosting provider where they pioneered early cloud technologies, it was there they found success adding hands-on labs to in-person interviews. In 2012 they set out to build a scalable cloud-based platform by bringing together both access and automation, creating a standard for the development and delivery of performance-based assessment.
Latency is the delay that a test user experiences due to the time it takes information to travel from source to destination and back. Examples include a delay in seeing characters appear after typing, a curser movement that goes to the wrong location or constantly waiting for the test to ‘catch up’ with user inputs. During most web browsing, latency is short enough that it is not an inconvenience. But for performance-based assessments, it can be a source of major frustration.
TrueAbility studied this problem with 100 users in 14 countries on all 7 continents using 14 datacenters. It found that the acceptable limit of latency was 150ms. Also, top issues impacting latency included:
(1) Physical distance of the transfer of information
(2) The number of hops, or networking devices the data crosses
(3) To a lesser extent, computer hardware
(4) The “last mile” of the network – i.e. company firewalls, other home bandwidth use, etc.
The user experience for performance-based testing can be optimized by ensuring the data center hosting the exam is as close as possible to the test candidate and minimizing ‘last mile’ latency issues. Other recommendations included using wired connections for testing instead of wireless routers and running tests of the exact same hardware + network prior to the exam. Performance-based testing is an outstanding way to assess an individual’s capability post learning and minimizing latency issues will help this become a more standard part of testing now and in the future.
Members can log in to access the session recording.
