Turning off the algorithm
The Digital Duty of Care is a great and much needed initiative that is long overdue. For years social media companies have controlled what users see on their feeds. It’s about time we had some control back on our social media feeds and online experience in general.
Different platforms have different recommender systems and they can cause harm, especially for vulnerable populations, by reinforcing negative beliefs, ‘hooking’ people into doom scrolling and creating echo chambers.
By bringing intentionality back into how we engage with social media, we can regain control on what we consume and also how long we engage, and thereby, how much more time we can have in our lives for other activities. However, the bill needs to be accompanied by awareness campaigns, training and education for users of all ages.
The PM’s announcement refers to both opting in and opting out of algorithms. These are vastly different control settings. With the opt-out option users will have the default recommender system running as it is and will have to make a conscious choice to opt-out of it.
The ‘opt-in’ is a higher control setting where no recommender system decides the feeds by default. Inertia means people tend to continue with default settings, so some education will be in order.
More control on how and what we view is particularly important in the age of AI where artificially generated content and hyper personalised and often manipulative advertisements are starting to flood users’ feeds. It is becoming increasingly difficult to distinguish between real and fake content generated by AI.
I am happy to see the PM’s announcement makes mention of online games, apps and AI chatbots. While under 18s may be at particular risk and should be protected, I think this should extend to all users.
As an example, people with mental health conditions are known to reach out to AI chatbots for quasi-therapy purposes with some very drastic outcomes.
And our latest research into how the general public and clinicians are engaging with AI chatbots such as ChatGPT and Claude shows some level of dependence on these systems for healthcare information and advice that may be problematic in light of known AI errors, hallucinations and AI sycophancy.
The new legislation seems to be a good first step in the right direction. I hope through this and more safeguards we are able to address more current and emerging AI harms.
Rashina is Associate Dean and Associate Professor in Software Engineering at the Faculty of IT at Monash University. She’s researched agile methods and practices in the software industry for more than 15 years.
