Seo Hye Lee: Shaping Sound Through Text

by Juli Cordray

We had the chance to catch up with artist Seo Hye Lee during her time at Vital Capacities — a UK-based accessible online residency program — to learn about what she’s been up to over the last month(s) and her practice more broadly.

For Seo Hye, the residency presented an opportunity to build on previous works that consider modes of communication, particularly in the visual form of gestures and subtitles:

“I was interested in further exploring dynamic forms of communication through subtitles and the residency provided the perfect opportunity to expand on this. My research during Vital Capacities has incorporated more about the history of subtitles, the difference between closed caption and subtitles, and also the history of intertitles during the silent film eras and the role of the Subtitler in translation in film/media.”

Words and symbols describing sound — interpreting, transcribing or supplementing spoken dialogue, atmospheric undertones, music — subtitles and closed captions are also processes of displaying text on a moving image, against a backdrop of changing scenes, moving bodies, and emotional shifts. In her research and work, Seo Hye is exploring and experimenting with the language used in such descriptions, what they convey or leave out, and the role played by this tool for accessibility.

A block of text with black bold fonts on white background.
Seo Hye Lee, subtitles, via Vital Capacities. *Images appear in color when hovered over.

Seo Hye, who has a cochlear implant, often draws on her experience of hearing loss in her work, while emphasizing the distinction between “hearing” and “listening” — the latter, she explains, is possible regardless of a person’s hearing ability. Her current research expands on a project from 2020, titled Partial Gestures. Created to accompany a zine that she also produced — titled BSL (British Sign Language) — the video work includes auto-generated subtitles from Youtube, with all their inaccuracies, transposed over images of close-cropped hand gestures, including sign language, and other partially visible body parts in motion, without revealing the full context. 

An image of a rectangular screen. The screen is much longer than it is high. On the screen, there are white subtitles with a black box around them that say “future you would be giving them a cochlear and the opportunity”. Behind the subtitles, a few blurred shapes can be seen which appear to be a table and shoe. The screen is within a black environment which surrounds it.
Seo Hye Lee, Partial Gestures, 2020 (video installation)
A close-up image of four open books positioned in two rows next to each other. The close-up means that only part of the books can be seen. In the top left corner of the image, part of a book shows part of a black and white photograph of two hands creating two O shapes touching at the fingers. Next to the photograph in the bottom right corner is a red letter B. In the bottom left corner of the image, part of a book shows part of a black and white photograph of two hands, the left hand showing five straight spread out fingers, palm facing forward and the right hand showing the index finger touching the middle finger of the left hand. Next to the bottom right of the photograph is a red letter I. In the top right corner of the image, part of a book shows part of a black and white photograph of two hands creating a D shape. Next to the bottom right corner of the photograph is a red letter D. In the bottom right corner of the image, part of two pages of a book show black and white photographs. On the left page, there are two photographs one on top of the other. The top photograph shows a left hand pointing its index finger and a right hand folding its index finger and touching the middle of the left index finger. To the right of the photograph, there is a red letter K. The bottom photograph shows a left palm and a right hand with index finger touching the left palm. To the right of the photograph, there is a red letter N. On the right page, there is a photograph showing a left palm and a right hand with its three middle fingers touching the left palm. To the left of the photograph there is a red letter M.
Seo Hye Lee, BSL, 2019 (print publication)

During her residency, Seo Hye has focused her research on the language of subtitles specifically within archival films depicting pottery-making processes and other crafts — likewise incorporating movements of the body. As Seo Hye describes, her new video work “explores the relationship between the language of subtitles and the visual language of hand-crafted ceramic pieces, forming language and communication from movement, gesture, and feeling.” 

Overlaying silent footage of hands as they touch and shape wet clay, molding it into round objects on a potter’s wheel, Seo Hye’s subtitles offer simultaneous sound cues to the viewer.  The same gestures and images are repeated in a row, like a filmstrip, accompanied by a different type of subtitle in each frame: one action-based, one abstract, and one music-based.

“I was interested as to how the identical videos took on varied meanings and the context of subtitles altered the experience for the viewers. By juxtaposing abstract, action-based, and music-based subtitles, I aim to highlight how powerful the use of imagery and words can be and how much this can alter our perception of events.”

Seo Hye’s action-based subtitles consist of a single verb in present continuous form: turning, shaping, etc. The abstract subtitles, on the other hand, feature short poetic descriptions without a concrete point of reference: “sound of listening inward”; “sound of remembering fondly”. While the music subtitles, too, are descriptive but not definitive: e.g., “mysterious string music”. Taken together, they prompt us to imagine what certain feelings and impalpable moments might sound like; and by extension, what subtitles themselves might sound like. Seo Hye elaborates:

“I found it fascinating how the ‘action-based’ subtitle can tell us the specific sound, enabling an immediate connection with visuals, while the more abstract and music subtitles are left open to our own interpretations.“

This specific connection between pottery, sound and language had already been materialized in her previous works — first in Artefacts of Sound and then in Many Shapes of Volumes (both from 2019) — in which she created ceramic sonic vessels. It’s a direction and medium she started to explore while living in Berlin and getting to know a new community. According to Seo Hye:

“I took ceramic classes in a local studio. Coming from an illustration background, naturally I started to explore ways of drawing sound into shapes and making those shapes into ceramics. I started to question — how could I create a shape that would contain sounds I want to hear? I gradually started to experiment with the idea of the shape of sound and began creating objects to reflect this.”

A photograph of a large room with five plinths and objects on them. The room has high ceilings, a wooden floor and large windows letting a lot of light in. The white plinths appear to have a speckled pattern on them in a darker colour. On top of the plinths there are different white vase-shaped objects.
Seo Hye Lee, Many Shapes of Volumes, 2019 (audio-visual installation)

The objects produced for Many Shapes of Volumes — fabricated by ceramicist James Duck — were based on Seo Hye’s drawings and acted as carriers of sound: “this allowed people to listen to and touch the pieces as they moved about the space.” Here, a process of translation manifests through tactile relationships, hand gestures, and physically shaping — giving form. 

“The work Many Shapes of Volume utilised both primitive materials and new technology; the choice of ceramic significant in its timeless nature, along with the integration of modern audio technology within. I find myself drawn to the connection between these two, and I have continued to explore this within the film archives of pottery and the digital art media format.”

In reference to working with these themes within moving-image media, in one of her posts on Vital Capacities, Seo Hye offers this insight: “Craft videos are fascinating as they frequently show pairs of hands making objects from a shapeless form into something beautiful. For me, this formation presents a parallel between the idea of digitally shaping words into the language of subtitles, exploring its poetic nature.”

An image of a rectangular screen. The screen is much longer than it is high. On the screen, there are white subtitles with a black box around them that say “wait total that but I feel so much pressure about this I’ve had enough”. Behind the subtitles, a few blurred shapes can be seen but it is uncertain what they are. The left side is brown and could be brick, the right shows a white shape. The screen is within a black environment which surrounds it.
Seo Hye Lee, Partial Gestures, 2020 (video installation)
A photograph of a television facing forwards and slightly to the left. The television is a black box shape and mounted to a white wall. Coming out from the left and right side of the television are two black tubes. On the screen, the fading words ‘WHAT?’ are repeated three times overlapping on top of each other at different angles.
Seo Hye Lee, What Did You Say?, 2017 (audio-visual installation)

In her latest work, the subtitles are also digitally shaped by square brackets, which typically signify an insertion of additional information for the purpose of description, of sound effects for instance, or clarification — in this case not transcribing dialogue, but the language of sound itself.

As Seo Hye points out, auto-generated subtitles, like those on Youtube, are notoriously inaccurate. But what potential do inaccuracies — or, perhaps, the absence of a definitive notion of correctness — hold in this context? Can miscommunication instead be used constructively, to offer varied experiences to viewers and allow space for new correspondences to emerge? 

In the various subtitles that appear over the same recurring image in Seo Hye’s current work, we might also recall the prevalence of captioned images on the internet, in the form of memes. Familiar images appear again and again in new contexts created through imagined dialogues or descriptions of ambient sounds. It’s within this digital sphere of communication (and miscommunication) that Seo Hye’s work likewise unfolds and opens up new narratives. Here, Seo Hye seeks out modes of communication based on feeling and perception, rather than clarity and accuracy, highlighting the nuances that already exist and embracing them for their constructive potential. At the same time, such gradations render sound plastic, as a form that is not only tactile, but also malleable — intended to be held, touched, felt, as well as shaped and reshaped, like clay.

Check out Seo Hye Lee’s new work in Vital Capacities’ online exhibition from July 22, 2021.