Good Data Guide - Panel Discussion Continued…
On the 19th of May, our highly anticipated Good Data Guide launch event brought together industry experts, thought leaders, and enthusiasts from the legal and technology sector to delve into the complexities of data ethics and the challenges of navigating the digital age responsibly.
The panel discussion, a key highlight of the event, touched upon various intriguing topics, from generative AI to GDPR. As we reflect on the insightful dialogue, it is worth exploring some unanswered questions that emerged during the discussion, provoking further thought and inquiry.
Q1 - Policy and strategy are important, what are the difficulties with controls? Capability, understanding? Doug – you mentioned the AI parole adviser being impractical because of the base data set. Was the data insufficient or did it show a history of poor decisions?
‘Controls are by their very nature designed to manage the use or outcome of a set of events. In terms of AI, issues can arise where for example a machine learning model or an ensemble is used which learns from their application. This means that the controls need to be reviewed on a frequent basis as the model itself could be subject to drift, e.g. changing over time producing different outcomes. There are further challenges in terms of the types of review required, the governance model that is applicable to the process/outcomes, training of people to understand and provide appropriate business rules/ interventions and the creation of capabilities to be able to detect anomalies and alter the appropriate person(s) to take remedial action to name but a few.
In relation to the data sources, the specific use case related to the identification, capture and validation of data sets to be used as the seed training data for the model.
It is often the case that the data sets can be biased and have been created based on a particular set of circumstances or context which will materially affect the purpose and outcome of a particular model that will be trained and developed using the data. Therefore, it is not often poor decisions that affect the quality of the model, it is the nature of the data itself that can lead to bias influences and assumptions e.g. the predominance of a certain segmentation or the absence of diversity to be able to ensure the outcomes are valid for the purpose the algorithm has been designed.’
Doug Brown, Vice President of Data & AI at NCR Corporation
Q2 - PR compliance is high on agendas because of the potentially huge fines, not morality. Without risk of high fines will good data ethics get enough traction?
‘Data ethics covers what we should do with data, which includes but goes beyond legal compliance. Therefore there will always be ethics questions that will never be resolved under the threat of fines. Whilst reputational risks for getting data ethics wrong may still lead to financial consequences, organisations need to find other incentives for putting data ethics on their agenda. For LOTI in local government, the reason for working on data ethics beyond compliance is to improve the quality of your data projects as well as building trust with residents, and there may well be similar benefits for industry too.’
Sam Nutt, Data Ethicist at London Office of Technology and Innovation
Q3 - Given the possibility for ‘drift’ in the application of LLM outputs, can they ever be consistent with principles such as “purpose limitation” under GDPR?
‘The answer to this could be both yes and no. At the point an algorithm is adopted for use, it may be explainable, and so its purpose is knowable.
BUT when you are using LLMs you are likely to start with a specific purpose for which the inputs will be processed by the LLM. In line with purpose limitation under data protection laws, you must only allow the LLM (and must not train it otherwise than) to process that input data as strictly necessary for that specific purpose. AND there may be aspects of the algorithm that are opaque at the point of adoption or the algorithm itself may adapt through use and develop opacity. We’d recommend 1) the use of a DPIA or an AIA as soon before the point of adoption as possible, so that risks are known and 2) a regular review of the algorithm in use.’
Sue Chadwick, Strategic & Digital Planning Advisor at Pinsent Masons and Research Fellow at Open Data Institute
Thank you to our expert panel:
Natalie Cramp, CEO at Profusion, Founder of the Data Ethics Advisory Board and Chair for Women in Data
Sue Chadwick, Strategic & Digital Planning Advisor at Pinsent Masons and Research Fellow at Open Data Institute
Sam Nutt, Data Ethicist at London Office of Technology and Innovation (LOTI)
Doug Brown, Vice President of Data and AI at NCR
Jeremy Khan, Senior Writer at Fortune Magazine (Moderator)
Want to discuss what this means for your organisation with our team of experts?
Contact us: hello@profusion.com