John Burton
Bias in AI – what you can do to be on alert
Lies, damn lies and statistics. For centuries, data has been used and manipulated to confirm or deny viewpoints and arguments. Presenting data in a selective or misleading manner can result in someone taking a completely different message away than what the raw unmanipulated data is actually revealing. This isn’t always due to bad faith actors either, limitations in the methodology, experimental design or collection mechanisms can result in biases and errors creeping into the data. Data is, without doubt, the most effective and important way to reveal the truth as to what’s really happening, but that doesn’t take away the fundamental limitations of the human collecting the data. We are after all the sum of our experiences, and to think this wouldn’t impact on the scientific method is a limitation.
One of the most famous examples of data biasing our conclusions is the survivorship bias. Planes in world war 2 which showed damage on particular areas of the fuselage when they returned from missions were analysed and reinforced in these areas in order to prevent further losses. After these modifications were made, there was no change in the rate of planes being shot down. Of course, the planes that were shot down didn’t appear at all in the data set since they were…shot down!
Biases can and have impacted not only research but production of day-to-day products that significantly affect our lives. Seatbelts and other car safety features were initially tested with crash test dummies with dimensions most comparable to the average man, meaning safety features weren’t as effective for women and had an increased risk of injury in a car accident.
Given the amount of biases inherent in data, LLMs and other more traditional machine learning algorithms are trained and fine-tuned on data which may lead to misleading, inaccurate or outright incorrect results. The pace in which GenAI can allow people to work at, coupled with the pressure they’re under to be productive, can significantly increase the risk of dubious work affecting results and impacting people depending on what work is being done.
How you can be an accuracy advocate
The main aspect of ensuring robust and accurate results is to adopt a curious and rigorous mindset when it comes to using and looking at the outputs of LLMs and GenAI. A healthy dose of scepticism in a world of increasing deepfakes and AI slop is essential, but it’s also important to keep questioning the outputs of all sources of information being used. Comparing independent data sources, peer review of LLM outputs, sanity checking and building in time for additional reviews are all important to ensure that your work is as free from bias and incorrect information as possible. It’s increasingly hard to ensure there is time for this when the pace of work is picking up and enabled with AI & automation, but it’s also increasingly important.