Written by John Kleeman, Founder of Questionmark, EVP at Learnosity/Questionmark and 2021 Chairperson of ATP
IT certification is a crucial gateway into the industry for young professionals round the world. It opens life chances for people to obtain a career start in IT and to get advancement.
For everyone to have an opportunity for these life chances, IT certification exams must be equitable and inclusive. For individuals, inclusive exams open a door into a satisfying and productive career in IT. And for IT certification providers, making exams meet the needs of all candidates helps encourage a diverse set of people working in their sector. Diversity is not just fashionable branding, there is lots of evidence that it helps long term success. As the Islamic proverb suggests, “a lot of different flowers make a bouquet”. More prosaically as prominent HR analyst Josh Bersin puts it, “Companies that embrace diversity and inclusion in all aspects of their business statistically outperform their peers”.
Our community of testing and certification professionals are on a collective mission to build better and more equitable testing experiences. Many ITCC members are leaders in diversity.
An IT certification exam measures a construct (e.g., the ability to understand and apply a particular aspect of IT). To be a fair exam, it is critical that the exam does not advantage or disadvantage individuals because of characteristics irrelevant to the construct being measured. It is not always easy to ensure this is the case, but here are some questions to help think through the issues.
Do your item writing guidelines encourage cultural neutrality?
July is not summer in the Southern hemisphere. Not all birthdays are celebrated with cake. Item writing guidelines or a style guide are vital to ensure question quality and uniformity. Make sure you provide your authors clear examples to write questions which are applicable for all cultures. Have a good review process. It’s helpful to ensure diversity within your authoring team and especially your review team.
Do you review question statistics to check on diversity?
If you have the candidate demographic data, it is valuable to compare questions to see if some are less well answered by some diversity groups. Any such questions need review or removal.
What can candidates do if they don’t have a quiet place to take an exam?
Remote proctoring is a boon for candidate convenience as it means that people can take exams at home, and no longer have a long journey to a test center. But what should a candidate who lives with others in a one room apartment do? Not all of us can go as far as the U.S. Law Skills Admissions Council who offer such candidates a $125 credit to hire a quiet space, but it’s worth thinking how you approach this issue. At the very least instruct your proctoring provider to be tolerant.
How powerful IT equipment do candidates need?
Existing IT workers will likely have access to a quality device with good connectivity, but new entrants may not. IT certification programs need to think about how they provide access to capable people who don’t have a high spec device at home and/or have limited connectivity.
Do all candidates speak English fluently?
Although being able to read English is useful in IT, English knowledge is not usually an explicit part of the construct being measured. Many candidates internationally will speak a different language, and even within the U.S., there are many who speak a language other than English at home.
Many programs translate exams professionally into some languages. For languages that it’s not practical to translate professionally, you can consider on-the-fly computer translation as a candidate aid. See the recent SAP/Questionmark capability that recently won the 2022 ITCC Innovation award for one way of doing this (screenshot below). Other approaches used are to ensure that only simple language is used to make it easier for less proficient non-native speakers and to offer extra time for non-native speakers.
Are you supportive of religious needs?
Ensure exam scheduling is flexible enough to avoid people having to take exams on their holy days. And respect the proctoring needs of some religions – for example if a female candidate requests a female remote proctor, make it happen. And do not require the removal of headscarves from candidates who need to wear them for religious reasons.
How well do you support accessibility?
There is increasing understanding that diversity includes neurodiversity and that there are significant percentages of candidates who benefit from good accessibility support. One good source on accessibility is this white paper from my Learnosity colleagues.
How expensive is it to prepare for exams?
As well as considering the inclusivity of the exam itself, it’s important to think about how less well-off people can learn the material they need to practice and pass the exam.
How inclusive are time limits?
This one is more controversial. Strict time limits have been a function of most tests for decades. But leading psychometrician Steve Sireci in his recent paper “Standardization and UNDERSTANDization in Educational Assessment” questions whether time limits provide value. Clearly there are practical benefits of a time limit in IT exams, for example when scheduling proctors. But is there is an argument that strict time limits fuel test anxiety and may put off people? I suggest reviewing how much time candidates use in your exams, and whether strict time limits deter less advantaged candidates.
Are some candidates impacted by test anxiety?
Some competent people struggle to do well at tests, does this include some of your candidates? Some struggle with test anxiety, others do not do as well in a test as their ability shows they should. This isn’t a well understood area, but one interesting approach is to encourage people from disadvantaged backgrounds with support, coaching and mentoring to adopt a more confident mindset.
As a community, including within ITCC, we are working on how to be more inclusive and diverse. There are more questions than answers, and it’s often uncomfortable to change. But diversity benefits everyone, and I hope this article helps encourage debate on how we can be more inclusive.
