Between the Words.
An interactive experience that highlights the subtle yet pervasive ways our voices and personal data are collected, manipulated, and repurposed without our full awareness—challenging participants to reconsider their assumptions about digital privacy, consent, and surveillance in everyday life.
about the project.
With the popularity of digital games, social media, and other data-sharing platforms, more and more individuals’ information is tracked and used without users being aware. Often, people give companies information about themselves without knowing how that data may be shared or sold to other parties. Our exhibit aims to draw attention to this privacy issue - that too often, individuals are convinced to provide data about themselves, in seemingly legitimate ways, while companies may take advantage of and misuse the data.
We captured this idea in our exhibit by creating a seemingly fun and lighthearted game, in a style inspired by the New York Times Dialect Quiz, to trick people into saying specific words. However, at the end of the experience, instead of using users’ data to guess their accent as we originally proposed, we reorganized and manipulated the recordings to make users appear to say silly things - in this case, the lyrics of Mary Had a Little Lamb. To conclude the experience, we presented users with this recording, as well as their likeness in a grid format with others’, to create a group announcement about data privacy. Our exhibit is particularly effective in that the breach of trust between users and data holders is revealed when it happens - we confront people at our exhibit right after their experience with the manipulated video. In the real world, people and their data are taken advantage of and may never know.
We essentially built our own camera app from scratch to record people’s video and audio, to accompany a graphical user interface we designed to lead users through our “game”. After individuals’ recordings are taken, our code removes the silence from both ends of the recordings to create smooth transitions between words, for when the recordings are concatenated to make sentences.
team members.
Charlotte Gilmore
Hanes Hsu
Arlie Jackson
Timmy Lin
Lade Tinubu
contribution.
Created a visually engaging and intuitive UI layout in Figma, ensuring that the design aligned with the project's concept and provided a seamless experience for participants.
Structured the interactive experience by determining the sequence of tasks, refining the game mechanics, and mapping out the user journey to effectively engage participants while subtly collecting voice data.
Wrote clear, concise user instructions to guide participants through the experience and created supporting illustrations to enhance understanding, ensuring accessibility and clarity.
Oversaw the entire production of the final video, from directing participants and capturing footage to editing and assembling a compelling narrative that highlights the project's purpose, execution, and participant reactions.