A Day in the Life of an Emergency Medicine Expert Witness A Day in the Life of an Emergency Medicine Expert Witness

A Day in the Life of an Emergency Medicine Expert Witness

Colin Holburn is an EWI fellow, governor and founding member. A consultant in accident and emergency medicine, he has been practising as an Expert...
NMC Health PLC v Ernst & Young LLP [2024] EWHC 3021 (Comm) NMC Health PLC v Ernst & Young LLP [2024] EWHC 3021 (Comm)

NMC Health PLC v Ernst & Young LLP [2024] EWHC 3021 (Comm)

The defendant made an application for adjournment on the proposition that it could not be ready for trial because its experts required additional time...
Government Response on Revisions to the Medical  Reporting Process for Road  Traffic... Government Response on Revisions to the Medical Reporting Process for Road Traffic...

Government Response on Revisions to the Medical Reporting Process for Road Traffic...

The Government has published its response to the consultation it ran from 18 July to 10 October 2023 on 'Revisions to the Medical Reporting...
Podcast Episode 8: Re-evaluating your opinion Podcast Episode 8: Re-evaluating your opinion

Podcast Episode 8: Re-evaluating your opinion

In the 8th episode of the Expert Matters Podcast, we discuss re-evaluating your opinion. We look at possible reasons why you might wish to re-evaluate...
Justice for people with a hearing impairment Justice for people with a hearing impairment

Justice for people with a hearing impairment

A psychiatrist whose evidence had often been admitted in capacity cases was assisted in this case of a hearing-impaired person by an interpreter who...
Family Court reporting pilot to be extended nationally Family Court reporting pilot to be extended nationally

Family Court reporting pilot to be extended nationally

The Family Procedure Rule Committee has approved a proposal to roll-out the family court reporting pilot nationally, through changes to the Family...
Judicial analysis of written expert healthcare evidence Judicial analysis of written expert healthcare evidence

Judicial analysis of written expert healthcare evidence

This is an important judgment for experts who prepare personal injury reports in the Republic of Ireland but also for all experts, in all of the...
New Immigration and Asylum Practice Direction New Immigration and Asylum Practice Direction

New Immigration and Asylum Practice Direction

The new Practice Direction of the Immigration and Asylum Chamber of the First Tier Tribunal came into force on 1 November 2024. The Practice Direction...
Podcast Episode 7: Review of 2024 Podcast Episode 7: Review of 2024

Podcast Episode 7: Review of 2024

In the last podcast for 2024, we look back at the ten key issues for expert witnesses that we've seen over the course of 2024, and highlight the...
Day in the Life of a Financial Expert Day in the Life of a Financial Expert

Day in the Life of a Financial Expert

Uwe Wystup is a practitioner in the field of foreign exchange options, as well as a senior academic, trainer, and judge. He is the founder of...
Podcast Episode 6: In Conversation with Giles Eyre Podcast Episode 6: In Conversation with Giles Eyre

Podcast Episode 6: In Conversation with Giles Eyre

In the 6th Episode of the Expert Matters Podcast, Simon talks with retired Barrister and expert witness trainer, Giles Eyre, who is retiring as an EWI...
A Day in the Life of a Medicolegal Expert Witness A Day in the Life of a Medicolegal Expert Witness

A Day in the Life of a Medicolegal Expert Witness

Sue Lightman is a Professor of Ophthalmology and Consultant Ophthalmologist who has been undertaking medicolegal Expert Witness work for over 20...

Check out our Case Updates and Member Magazine

Looking for more news relevant to the Expert Witness community? Why not check out our database of cases relevant to Expert Evidence or the latest and previous editions of our member magazine, Expert Matters.

News

Clicking on one of the topics below will display news items relevant to that topic. You can also use the search bar below to identify news items.

How not to use AI in expert evidence
Sean Mosby 614

How not to use AI in expert evidence

bySean Mosby

 

Summary

In this US case, an expert in fiduciary services used Microsoft’s Copilot to cross-check calculations he used in expert evidence. He was unable to recall the prompts he used, state the sources Copilot relied on, or explain how the tool worked and arrived at its outputs. The judge provided some useful insight into the challenges with using AI in expert evidence.  

Learning points

Learning points for experts:

  • Do not use an AI tool unless you fully understand it and can explain how the tool works, how you have used it, how it generated the results, and what the results mean.

  • If an AI tool is not generally utilised in your field, consider whether it is appropriate to use it for expert evidence.

  • Do not rely on an AI tool if the results are not fully replicable.

  • Check the AI tool to see if its developers consider it sufficiently reliable to be used in court proceedings.

  • Be aware that AI tools are often trained primarily on data from non-UK jurisdictions.

  • If you wish to use an AI tool, tell your instructing party as soon as possible which tool you intend to use, explaining why you require it and how you intend to use it.

  • Disclose your use of the AI tool to all parties and the court, clearly setting out the prompts and configurations you have used.

  • Make sure you use the AI tool thoughtfully and be extremely careful before entering any confidential or personal information. Remember that AI tools will generally retain any information you enter and the organisation managing the tool may use that information to provide services to third-parties.

  • If you use, or are considering using, AI tools in your expert witness work, make sure you understand any guidance on the use of AI in court proceedings, and especially any guidance or case law on the use of AI in expert evidence.

  • Be appropriately cautious with documents generated by others using AI tools.

  • Lastly, it is worth considering how AI tools might impact your practice. Developments in AI are advancing rapidly and may have an impact in the future on the work of many expert witnesses. Even if you don’t choose to use AI yourself, the opposing expert might.

  • You can watch Sir Keith Lindblom’s keynote address to the 2024 EWI Conference to hear the judiciary’s view on the role of AI in expert evidence. Also of interest is Sir Geoffery Vos’s speech on AI – Transforming the work of lawyers and judges and the Judicial Guidance on the use of AI.

Learning points for instructing parties:

  • Ask the experts you instruct whether they intend to use any AI tools.

  • If they intend to use an AI tool, make them aware of any guidance or case law on the use of AI tools in expert evidence.

  • Ensure that any use of AI tools is disclosed to the court and the other party.

  • Ideally, you should include the request to use AI in your application for permission to rely on expert evidence, explaining why the tool is needed, how it will be used, why it is reliable, and how any confidential or personal information will be protected.  

The case

The petitioner was seeking an order for settlement of the accounts of the estate she had been administering. The objectant, who was a beneficiary of the Trust, claimed the petitioner had breached her fiduciary duty by deciding to retain a property in the Bahamas when selling it would have been a better financial option. He also objected to her travel to the island which he claimed amounted to a conflict of interest or self-dealing.

The expert evidence

The objectant relied on the expert evidence of Charles Ransom to prove the damages caused by the retention of the property in the Bahamas. Mr Ransom provided two reports. The court found numerous deficiencies in Mr Ransom’s evidence, finding that his “calculations and specifically those with regards to damages are inherently unreliable, are based on speculation, hypothetical market performance, and are unsupported or outright contradict by facts in the record”.

The expert’s use of AI

The court went on to examine Mr Ransom’s use of Microsoft Copilot, a generative AI tool, in cross-checking his calculations. Mr Ransom could not recall the input or prompt he used, could not state which sources Copilot relied upon, and was unable to explain how Copilot works and how it arrives at a given output.

The judge noted that the court did not have an objective understanding of how Copilot works and, when it attempted to replicate Mr Ranom’s results, Copilot provided different outputs. The judge noted that “[w]hile these resulting variations are not large, the fact there are variations at all calls into question the reliability and accuracy of Copilot to generate evidence to be relied upon in a court proceeding.”

The judge then queried Copilot about its reliability and accuracy. The tool responded that “my accuracy is only as good as my sources so for critical matters, it's always wise to verify” and “I do my best to be as reliable as possible. However, I'm also programmed to advise checking with experts for critical issues.” When asked whether its calculations were reliable enough to be used in court, Copilot replied “[w]hen it comes to legal matters, any calculations or data need to meet strict standards. I can provide accurate info, but it should always be verified by experts and accompanied by professional evaluations before being used in court”.

The judge concluded that “it would seem that even Copilot itself self-checks and relies on human oversight and analysis. It is clear from these responses that the developers of the Copilot program recognize the need for its supervision by a trained human operator to verify the accuracy of the submitted information as well as the output.”

Although arguing that the use of AI tools for drafting expert reports was generally accepted in the field of fiduciary services, he was unable to name any publications regarding its use or sources to confirm its use was generally accepted.

The court found that due to the rapid evolution of artificial intelligence and its inherent reliability issues that prior to evidence being introduced which has been generated by an artificial intelligence product or system, counsel has an affirmative duty to disclose the use of artificial intelligence. Permission to use such evidence should properly be the subject to a pre-trial hearing with the scope of the evidence determined by the court.

Share

Print
Comments are only visible to subscribers.