Dragan describes some problems with self-driving cars, like this example of a car giving up on merging when there was no gap. After adding some more aggressive driving tactics, researchers then also had to add some courtesy to moderate those. One odd outcome of this was that when the car got to an uncontrolled intersection with another, the car would back up slightly to signal to the other driver that it could go first. Which actually worked fine! It mostly led to the other driver crossing the intersection more quickly (probably because they felt confident that the self-driving car wasn’t going to go). …….except if there’s another car waiting behind the self-driving car, or a very unnerved passenger in the car. It’s a challenge to work out what robots should be optimising for, when it comes to human-robot interactions. Generating good behaviour requires specifying a good cost function, which is remarkably difficult for most agents.
Designers need to think about how robots can work in partnership with humans to work out what their goals actually are (because humans are often bad at this). Robots that can go back to humans and actively query whether they’re making the right choices will be more effective. This framework also lets us think about humans as wanting the robots to do well.
This talk focused specifically on individual (rather than structural) issues in AI ethics. It drew on behavioural economics, philosophy of technology, and normative ethics to connect a set of abstract ethical principles to a (somewhat) concrete set of design choices.
Draws on an understanding of online manipulation as the use of information technology to impose hidden influences on another person’s decision-making: this undermines their autonomy, which can produce the further harm of diminishing their welfare. Thaler and Sunstein’s Nudge discusses choice architecture: the framing of our decision-making. We act reflexively and habitually on the basis of subtle cues, so choice architecture can have an enormous impact on our decisions. Adaptive choice environments are highly-personalised choice environments that draw on user data.
What kind of world are we building with these tools? Technological transparency: once we become adept at using technologies they recede from conscious awareness (this is kind of the opposite of how we talk about transparency in a governance context). Our environment is full of tools that are functionally invisible to us, but shape our choices in significant ways. Adaptive choice architectures create vulnerabilities in our decision-making, and there are few reasons to assume that the technology industry shaping those architectures are trustworthy. However, manipulation is harmful even when it doesn’t change people’s behaviour because of the threats to our autonomy.
We might think of ourselves as a dual system model: system one is fast, effortless, emotional and heuristic, system two is slower and more laborious. We often need to balance short-term desires (EAT THE DONUT) against longer-term goals (HOARD DONUTS INTO A GIANT PILE TO ATTRACT A DONUT-LOVING DRAGON). [Note: these are my own examples.]How do we deal with this? We need to have good models for understanding how irrational we are. We also need to balance these two systems against each other.
Problem: there’s a misalignment between individual and social welfare in many cases. AI research can draw on economic research around the contract design problem. Economists have discovered that contracts are always incomplete, failing to consider important factors like the expenditure of effort. Misspecification in contract design is unavoidable and pervasive, and it’s useful for AI research to learn from this: it’s not just an engineering error or a mistake. Economic theory offers insights for weakly strategic AI. Human contacts are incomplete, and relational – they’re always shaped by and interpreted by the wider context. Can we build AIs that can similarly draw on their broader context?
Algorithmic auditing is meant to hold AI systems accountable. There are several major challenges, including hostile corporate responses, the lack of public pressure, access to targets for evaluation, and benchmark bias. This research offers several solutions to these problems. For example, if we think about bias as a vulnerability or bug, we might use the model of coordinated vulnerability disclosure to overcome corporate hostility. When it comes to benchmark bias, the Gender Shapes project provided an intersectional data set to test systems.
Evaluating companies’ systems once they were targeted for these audits showed continued gaps in accuracy (white men were most accurately identified), but the gap did close. Audit design matters: we can make design decisions that encourage certain public and corporate reactions. We don’t just need data fairness, we need data justice. The Safe Face Pledge is a new project working on this.
A framework for benchmarking discrimination-aware models in machine learning, by Rodrigo L. Cardoso, Wagner Meira Jr., Virgilio Almeida and Mohammed J. Zaki was unfortunately too technical for my sleep-deprived brain to manage.
Towards a Just Theory of Measurement: A Principled Social Measurement Assurance Program, Thomas Gilbert and McKane Andrus
Often, work on ML fairness starts with a given institutional threshold without interrogating the reality they refer to. Some recent work is starting to look more closely at this, like Goodhart’s Law. Can we resituate ML and AI within the institutional pipeline to grapple with what ‘fair’ or ‘just’ decision-making as a whole means. AI ethics isn’t just about how we make the tool fair, it’s about how we use the tool to make institutions more just. Instead of using Fair ML and Ethical AI frameworks to apply existing policies, what if we used them to interrogate those frameworks?
For example, we might look at the ways in which people who are poor are much more susceptible to surveillance from the state. The authors offer different ‘justice models’ as a way of intervening: Rawls, Nozick, and Gramsci. (This was an interesting paper and notable for its emphasis on using ML and AI to change the status quo, so here’s a reminder to myself to read the full paper later when I have eventually had some sleep.)
Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements, Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Allison Woodruff, Christine Luu, Pierre Kreitmann, Jonathan Bischof and Ed H. Chi
leftover words 