74% score
The creators of an AI-powered paralegal say it has successfully passed part one of the Solicitors Qualifying Examination (SQE).
The bot, dubbed ‘Lawrence’, achieved a score of 74% compared to the typical pass rate of between 55% and 65%.
SQE1 is broken down into two Functioning Legal Knowledge (FLK) assessments and covers a broad range of legal topics including contract, tort, property, crime and trusts.
Lawrence was able to successfully answer 67 of the 90 multiple choice sample questions which appear on the Solicitors Regulation Authority’s website. This, according to Lawhive, the lawtech firm that created the bot, demonstrates its “ability to learn, digest and offer considered responses to various legal situations”.
As for the Lawrence’s weak points the creators say that while there were no clear themes, the bot did struggle with questions featuring “complex chains of logic and wider context”. It also struggled when two concepts shared similarities, confusing public nuisance versus private nuisance for example. Law students will sympathise.
But Lawrence didn’t just stop at the SQE. He (it?) and a human lawyer were presented with the same client’s will and probate case in order to compare tone, empathy and legal knowledge.
On the AI paralegal’s performance, Lawhive said:
“Whilst Lawrence managed to steer the conversation with the client to gain the necessary information about the client’s late relative’s will and assets, the conversation remained largely transactional and of half the length of the human solicitor. Feedback from the client was positive to both responses, but critiqued Lawrence for not showing as much empathy as the human counterpart. Lawrence also failed to question the client on their late relative’s spending habits and asked around a wider context that ultimately uncovered financial liabilities the solicitor would need to be aware of.”
Don’t worry, though; Lawrence isn’t out of a job just yet. The SQE-passing paralegal is currently being used to support the companies team of solicitors and legal experts.