How might we empower people to discuss, critique, and reflect on how personalization algorithms might find their way into unexpected areas of their lives? FutureShift is a workshop for making alternative future scenarios and values more tangible during the design process.
For my Master’s capstone project in Human Centered Design & Engineering, I created a speculative design workshop to explore the implications of machine learning algorithms in the near future. Participants explored Black Mirror-esque scenarios and created boundary-pushing systems and prototypes that might exist in those scenarios.
Speculative design gives us an opportunity to debate and critically reflect on what we want and don’t want to see in our future as a society. To understand how we want to shape and be shaped by algorithms, we need to create space to challenge presumed virtues and dominant values inscribed in them. (Since this workshop, other companies have used a similar process.)
Skip to our Process Book where we documented our workshop and learnings!
- Paul Roberts: Design Strategist & Project Manager
- Ariel Duncan: UX Researcher
My role: Visual Design Lead. Responsible for creative direction, and writing & editing for the workshop and process book. Led literature review and partnered with Research Lead on expert interviews. Collaborated on all other aspects of this project.
Timeline: 10 weeks
Is frictionless design the best design we can create?
At the outset, we wanted to question the ideal that frictionless, seamless design is good. From fake news on Facebook to predictive policing software, we realized we were blind to how algorithms dictate many aspects of our lives. As algorithms increasingly make decisions for us, they gain agency without us noticing it.
We wanted to shift the agency back towards people, and let them decide how much power algorithms should have over their day-to-day decisions.
To do this, we shifted our thinking from creating incremental solutions to using design to provoke, critique, and reflect on our own unquestioned biases.
- Enable participants to explore the social and ethical implications of technological predictions through making a “speculative object” that expressed their (one of many) desired relationships with algorithms.
- Display these objects at a “yard sale of the future” where the fictitious products would be sold alongside items one might find at a real yard sale. The yard sale provided a real context to start conversations with visitors around the values embedded in fictitious products.
We created a workshop called FutureShift.
We conducted our workshop with 4 participants who came from the University of Washington. Diversity of participants was critical, so the university environment allowed us to recruit people from different majors, from Junior to Ph.D level. At the end of the workshop, we surveyed participants on their experience.
- Meet My Algorithms Icons
- Algorithmic Self Drawing Sheets
- Future Signals
- FutureShift Card Deck
- Yard Sale Price Cards
- Workshop Evaluation Form
- Workshop Protocol
While we didn’t expect to cultivate agency in 3 hours, we saw hints of agency emerge through both form and narrative, and during our discussions and post-workshop evaluation.
We had a total of 19 visitors and engaged in conversations about workshop participants’ speculative objects with 10 visitors. We filmed some of the visitors’ responses.
Understanding the issue at scale
To conserve time, we conducted 6 interviews with experts in the field and read over 50 pieces of academic literature and industry reports.
High-level research goals:
- Unpack the interplay between algorithms and their impact on culture & society, and uncover methods for revealing how algorithms work (i.e. reverse-engineering algorithm’s outputs).
- Explore strategies from the field to inform how we could inspire awareness, discussion, and reflection around algorithms and people’s desired relationship with them.
Findings from discovery research
From talking to experts, we learned that we can repurpose our design process into conceptual tools to challenge our assumptions, preconceptions, and givens about the roles algorithms play in people’s everyday lives.
1) People have conflicting relationships with algorithms
People were conflicted about how algorithms “see” them because they didn’t trust the intentions behind the companies (i.e. Facebook, Google). But at the same time, they were frustrated when these systems didn’t know enough about them to personalize the experience.
2) People make up stories about how algorithmic systems work
Then, they behave according to what they believe–sometimes trying to train the system to better align with their preferences. People see the data and content that are served up but not how they’re being pulled together or weighted.
3) Opaque decision-making processes contribute to people’s uneasy relationship
- It’s hard to challenge a decision made by algorithms when the process is “blackboxed.” Usually we only see the outputs, and the lack of transparency makes it difficult to trace a particular decision and much less hold the system accountable.
- Algorithms are perceived to be objective when they’re not. They encode human biases in the form of rules, categories, and criteria. It’s easy to accept a result as a given, and this sense of certainty discourages people to challenge an algorithmic output.
- We risk creating algorithms that exclude and discriminate if we don’t actively challenge our narrow view of the world. Right now, the capabilities to change how algorithms work rest with developers, designers, and researchers. By including more diverse segments of our population, we can avoid blindspots and anticipate emerging desires around the kind of relationship people want with algorithms.
If machine learning algorithms already walk a fine line between creepy and annoying, there was an opportunity to work backwards from people’s preferred relationship with them.
Reframing the Challenge
Without challenging personal assumptions that go into designing algorithms, we recreate current inequalities in algorithmic systems. But that’s difficult to change if we don’t shift the agency back to “non-experts” – in a process to highlight people, perspectives, and values that might be forgotten during product development.
Empowering non-experts to voice their concerns and tensions about how algorithms are currently designed.
How might we empower people to discuss, critique, and reflect on how algorithms might find their way into unexpected areas of their lives?
Two immediate audiences:
- Non-experts: These are people who don’t work at McKinsey (for example), contribute to industry trend reports, or have regular access to design/development teams on algorithmic products. We want to hear from people who aren’t usually included in developing algorithms or thinking about long-term future.
- Design, development, and research teams: These people have direct impact on creating algorithmic systems. We want to empower them to challenge their own perspectives about “what’s desirable.”
Why it matters:
By working backwards from preferred visions, we can enable tangible actions today while including alternative values and considering the ethical implications of rules, categories, and criteria we embed into algorithms.
Arriving at Our Concept
Broadening our view of “the future”
To challenge our conceptions of a “desirable future,” we needed to push the extremes into the “ridiculous.” We took inspiration from Dunne & Raby’s ideas on speculative and critical design and Montgomery & Woebken’s hands-on, participatory approach to challenging conventional values. To see how we might apply these ideas in practice, we borrowed from Dr. Dan Lockton’s “designing for agency” and A.B. Kocaballi’s agency-sensitive design qualities.
Questions to inform our design strategy
- How might we empower people to challenge the status quo of how the future is currently being imagined and to add their own personal critiques and desires into the consideration?
- How might we make algorithms and their impact more visible and tangible?
- How might we include a more diverse audience when we discuss future implications of algorithms?
Deciding on a participatory workshop framework
After exploring digital “interventions” and social experiments, we decided to modify a workshop framework from the Extrapolation Factory because it was an approachable way to engage everyday people in thinking about the future. The workshop walked people through making speculative, future objects and then placed these objects in everyday contexts to engage the public in spontaneous discussions about possible futures.
Iterating the Workshop Concept
We iterated on the Extrapolation Factory’s workshop concept to support our design strategy.
1) From present solutions to future implications
But we eventually took out the STEEP lenses (Social, Technological, Environmental, Economic, Political) because participants were getting stuck at categorizing the future signals instead of articulating their desired scenarios. We also simplified the Futures Wheel to keep the momentum of coming up with varied implications from participants’ chosen signals.
We curated a list of future signals from research firms like Gartner and Intel to provoke discussions and probe for people’s desired relationships with algorithms.
To embed their own personal critiques and desires, participants created narratives to articulate their point of view about the future implications of their chosen signals. Even through we removed the STEEP lenses, we brainstormed provocative questions about each lens and created a deck of “STEEP” cards to help participants articulate their thoughts.
2) From barely noticeable to visceral
Participants made tangible objects to playfully materialize their hopes and concerns about algorithms, and interact with the future on a visceral level. By using tangible materials, we were able to talk about other possible versions of future in a more concrete way.
We iterated on the kinds of materials that would be conducive to expressing abstract ideas. The challenge was to overcome messages already encoded by professional industrial designers about what the finished products “should” be. In the end, we opted towards modular materials, breaking apart finished products if we needed to. We selected items that more loosely communicated about how they should be used.
We also tested around a context to display participants’s objects. An appropriate context encourages people to consider near-future products side by side with the everyday and the “outdated” products. This sort of comparison brought out visceral and emotion connections to potential scenarios embodied in future products, and we used them to generate discussions with the public.
We tested our assumptions around how “familiar” a context would be to ensure that it grounded both the public and workshop participants. Using guerilla testing, we asked people to list as many objects as they could think of in a potential context and then interviewed them about the personal stories attached to those objects.
We landed on yard sale of the future concept based on accessibility, familiarity, natural evaluative-ness, and feasibility.
3) From deductive thinking to inductive thinking
Future predictions are often dictated by research firms and academic think tanks. We opened up space for non-experts to voice their own versions of “what’s a desirable future?” To make it more relatable to everyday life, we tested two precursor activities: Meet My Algorithms and Algorithmic Self Drawing Exercise.
We leaned heavily on facilitation, working 1-on-1 with participants on articulating their hopes and concerns about a future prediction. Facilitation was a key element in building awareness and reflection through spontaneous conversations.
We also worked 1-on-1 with participants during the making activity to help them “think with” the materials. In the post-workshop survey, participants said that our facilitation made the workshop more enjoyable and helped them express the narrative they wrote.
Walkthrough of FutureShift Workshop
Our workshop, which we named FutureShift, consisted of several phases.
Let’s Use an Actual Example
Experiencing the Workshop
Here are the steps through the FutureShift workshop.
1) Meet My Algorithms
As they entered, participants were invited to chart their relationship with common algorithms (e.g. Facebook, Amazon, Google, Netflix) on a wall. They mapped these algorithms on a 2×2 diagram using two scales: “trustworthy” to “creepy” and “gets me” to “doesn’t know me at all.”
2) Algorithmic Self Drawing Exercise
To reveal the invisible work of algorithmic systems, participants sketched and shared versions of themselves from the perspective of three popular algorithms: Facebook, Netflix, and Amazon.
3) Choosing a signal
Participants were invited to review 35 predictions about the impact of algorithms on culture and select one or two that they’d like to explore further. These “signals” challenge stereotypical ideas about what counts as “the future” by exposing participants to a variety of scenarios about how algorithms might influence unexpected areas of their life.
4) Sorting futures
Participants chose whether their signal about the future belongs in “probable”, “plausible”, or “possible” future. Using a pink thread, they mapped a route through the appropriate section of the Future Cone and into the first arrow of the next phase of the diagram. The Future Cone helped participants understand “futures” in the plural.
Participants fleshed out the “world” of their signal, creating multiple narratives that explored the implications on other aspects of life in that future scenario. A deck of provocative questions helped encourage participants to consider different perspectives. We created a two-level chevron as a visual cue to push them to explore secondary and tertiary consequences of their signal’s scenario.
6) Making speculative products
Participants were invited to choose materials that were of interest to them and physically render any aspects of the narratives they created. We placed the materials close to the Future Cone area to help them make connections between the materials’ qualities and the scenarios they were considering.
7) Putting the products up for sale
Participants were invited to imagine their object existing in the context of a yard sale of the future. They created narratives about how their object ended up for sale and the price they would ask for it. Doing this helped them think about how their object might be used, become obsolete, or be discarded.
8) Yard Sale of the Future
We hosted an actual yard sale where participants’ objects were displayed amongst “normal” items one would find at a typical yard sale. Participants’ objects were used to inspire spontaneous discussions about the possible futures they represented as visitors browsed items for sale. We were able to engage passersby in brief but emotionally-charged conversations about the role algorithms might play in the near future.
The future is something that people expect to imagine with others.
Imagining the future is hard, precisely because we focus on the now, the tangible experience. But that highlights a larger issue in user-centered design. As designers, we focus around the narrow goals of the user and the business as it exists today and underestimate the long-term impact of things we design. Our work processes are set up this way: agile, lean methodologies, design sprints, etc. Without changing our work processes, we won’t be able to change our outcome. We need to shift away from our myopic tendencies and ask different questions of ourselves and our design parameters. When we haven’t been asking “what if?” and “is this a good idea?”, it’s harder to parse out which possibilities are preferable and which are terrifying, as Black Mirror invites us to consider.
We can accomplish this through critical reflection. But as facilitators, we need to think of ourselves as both individuals and researchers, rather than imagine that we can be outside of the processes we’ve designed. To effectively generate constructive debates, participants need to trust us–and trust each other–to bounce perspectives off of one another. We realized that creating the story (storymaking step) was critical in helping one another reflect and consider the implications of several possible futures. Writing a story about a desired future builds shared understanding of the values and preferences built into it and “backcast” to the present to address upcoming shifts.
Is a “disruptive innovation” a good one? We won’t know if we don’t talk about it.