Regulator plans to recruit diverse pool of question writers and assessors
All questions on the new super-exam will be reviewed for “cultural bias” so that Black, Asian and Minority Ethnic (BAME) candidates are not disadvantaged, the Solicitors Regulation Authority’s (SRA) education and training chief has said.
In a webinar shared on YouTube yesterday, Julie Brannan explained the risks and benefits to equality, diversity and inclusion that the new Solicitors Qualifying Exam (SQE) may bring, and planned mitigations to manage those risks, before it comes into force from autumn next year.
The regulator previously found as part of a pilot that in the first stage of the assessment, known as SQE1, white candidates generally performed better than BAME candidates. This was true both of the functioning legal knowledge test and skills assessment within SQE1.
The SRA, in realising the design of the SQE1 skills assessment was not sufficiently robust, has since reviewed the process and “will be putting in place protections, processes and safeguards” to ensure the assessment is fair to all candidates.
These measures include reviewing all exam questions for cultural bias “to make sure that there is no implicit cultural knowledge or understanding which is assumed in the questions”. Further, the regulator will recruit and train “a diverse pool” of question writers and assessors. “I think this is one of the most critical ways to make sure the assessment is fair to everybody,” says Brannan, adding that as the SQE rollout date nears, the SRA will be recruiting a wide range of question writers and assessors, who must be qualified solicitors, to reflect the diverse make-up of the profession.
Brannan goes on to discuss “emerging findings” from the pilot for the second stage of the assessment, the results of which will be published this summer.
Kaplan, the organisation appointed to assess the SQE, found that “univariate” analysis (performance by individual candidate variables) shows a significant performance difference between white and BAME candidates compared with “multivariate” analysis (which seeks to identify the best true predictors of candidate scores). However, both analyses should be viewed with caution due to overlapping candidate variables and the small numbers in the different groups, Brannan stressed.
The design of the SQE2 model, which will examine practical skills, is yet to be revealed but the SRA is considering whether to apply a “uniform” or “optional” model. It is exploring whether candidates should be examined on the same skills in the same practice areas, or whether it should allow an element of choice so that they can choose from a list of five “contexts” that may perhaps be more relevant to their chosen area of specialism. “It’s a tricky issue,” contends Brannan, who goes on to evaluate the risks and benefits of each route.
