textur

feature

Seo Hye Lee: Shaping Sound Through Text

by Juli Cordray

We had the chance to catch up with artist Seo Hye Lee during her time at Vital Capacities — a UK-based accessible online residency program — to learn about what she’s been up to over the last month(s) and her practice more broadly.

For Seo Hye, the residency presented an opportunity to build on previous works that consider modes of communication, particularly in the visual form of gestures and subtitles:

“I was interested in further exploring dynamic forms of communication through subtitles and the residency provided the perfect opportunity to expand on this. My research during Vital Capacities has incorporated more about the history of subtitles, the difference between closed caption and subtitles, and also the history of intertitles during the silent film eras and the role of the Subtitler in translation in film/media.”

Words and symbols describing sound — interpreting, transcribing or supplementing spoken dialogue, atmospheric undertones, music — subtitles and closed captions are also processes of displaying text on a moving image, against a backdrop of changing scenes, moving bodies, and emotional shifts. In her research and work, Seo Hye is exploring and experimenting with the language used in such descriptions, what they convey or leave out, and the role played by this tool for accessibility.

A block of text with black bold fonts on white background.
Seo Hye Lee, subtitles, via Vital Capacities. *Images appear in color when hovered over.

Seo Hye, who has a cochlear implant, often draws on her experience of hearing loss in her work, while emphasizing the distinction between “hearing” and “listening” — the latter, she explains, is possible regardless of a person’s hearing ability. Her current research expands on a project from 2020, titled Partial Gestures. Created to accompany a zine that she also produced — titled BSL (British Sign Language) — the video work includes auto-generated subtitles from Youtube, with all their inaccuracies, transposed over images of close-cropped hand gestures, including sign language, and other partially visible body parts in motion, without revealing the full context. 

An image of a rectangular screen. The screen is much longer than it is high. On the screen, there are white subtitles with a black box around them that say “future you would be giving them a cochlear and the opportunity”. Behind the subtitles, a few blurred shapes can be seen which appear to be a table and shoe. The screen is within a black environment which surrounds it.
Seo Hye Lee, Partial Gestures, 2020 (video installation)
A close-up image of four open books positioned in two rows next to each other. The close-up means that only part of the books can be seen. In the top left corner of the image, part of a book shows part of a black and white photograph of two hands creating two O shapes touching at the fingers. Next to the photograph in the bottom right corner is a red letter B. In the bottom left corner of the image, part of a book shows part of a black and white photograph of two hands, the left hand showing five straight spread out fingers, palm facing forward and the right hand showing the index finger touching the middle finger of the left hand. Next to the bottom right of the photograph is a red letter I. In the top right corner of the image, part of a book shows part of a black and white photograph of two hands creating a D shape. Next to the bottom right corner of the photograph is a red letter D. In the bottom right corner of the image, part of two pages of a book show black and white photographs. On the left page, there are two photographs one on top of the other. The top photograph shows a left hand pointing its index finger and a right hand folding its index finger and touching the middle of the left index finger. To the right of the photograph, there is a red letter K. The bottom photograph shows a left palm and a right hand with index finger touching the left palm. To the right of the photograph, there is a red letter N. On the right page, there is a photograph showing a left palm and a right hand with its three middle fingers touching the left palm. To the left of the photograph there is a red letter M.
Seo Hye Lee, BSL, 2019 (print publication)

During her residency, Seo Hye has focused her research on the language of subtitles specifically within archival films depicting pottery-making processes and other crafts — likewise incorporating movements of the body. As Seo Hye describes, her new video work “explores the relationship between the language of subtitles and the visual language of hand-crafted ceramic pieces, forming language and communication from movement, gesture, and feeling.” 

Overlaying silent footage of hands as they touch and shape wet clay, molding it into round objects on a potter’s wheel, Seo Hye’s subtitles offer simultaneous sound cues to the viewer.  The same gestures and images are repeated in a row, like a filmstrip, accompanied by a different type of subtitle in each frame: one action-based, one abstract, and one music-based.

“I was interested as to how the identical videos took on varied meanings and the context of subtitles altered the experience for the viewers. By juxtaposing abstract, action-based, and music-based subtitles, I aim to highlight how powerful the use of imagery and words can be and how much this can alter our perception of events.”

  • A photograph of a sculpture positioned in the centre of the image. The sculpture is an off-white colour and features a small cuboid shaped box with five hollow cylinder shapes of different heights positioned vertically on top of the box. The sculpture is on a white floor which has various black capital letters spread out on it. Behind this is a grey background.
    Seo Hye Lee, Artefacts of Sound, 2020 (audio-visual installation)
  • A photograph of a person leaning their ear towards an object on a plinth. The person has dark brown hair and is wearing a green puffer jacket and has a brown leather bag. The object is white, round and looks like a vase. The top part of the plinth is visible and it has various squiggly yellow, blue and green lines on it. The background is out of focus and shows a person sitting at a table on the right side of the photograph.
    Seo Hye Lee, Many Shapes of Volumes, 2019 (audio-visual installation)

Seo Hye’s action-based subtitles consist of a single verb in present continuous form: turning, shaping, etc. The abstract subtitles, on the other hand, feature short poetic descriptions without a concrete point of reference: “sound of listening inward”; “sound of remembering fondly”. While the music subtitles, too, are descriptive but not definitive: e.g., “mysterious string music”. Taken together, they prompt us to imagine what certain feelings and impalpable moments might sound like; and by extension, what subtitles themselves might sound like. Seo Hye elaborates:

“I found it fascinating how the ‘action-based’ subtitle can tell us the specific sound, enabling an immediate connection with visuals, while the more abstract and music subtitles are left open to our own interpretations.“

This specific connection between pottery, sound and language had already been materialized in her previous works — first in Artefacts of Sound and then in Many Shapes of Volumes (both from 2019) — in which she created ceramic sonic vessels. It’s a direction and medium she started to explore while living in Berlin and getting to know a new community. According to Seo Hye:

“I took ceramic classes in a local studio. Coming from an illustration background, naturally I started to explore ways of drawing sound into shapes and making those shapes into ceramics. I started to question — how could I create a shape that would contain sounds I want to hear? I gradually started to experiment with the idea of the shape of sound and began creating objects to reflect this.”

A photograph of a large room with five plinths and objects on them. The room has high ceilings, a wooden floor and large windows letting a lot of light in. The white plinths appear to have a speckled pattern on them in a darker colour. On top of the plinths there are different white vase-shaped objects.
Seo Hye Lee, Many Shapes of Volumes, 2019 (audio-visual installation)

The objects produced for Many Shapes of Volumes — fabricated by ceramicist James Duck — were based on Seo Hye’s drawings and acted as carriers of sound: “this allowed people to listen to and touch the pieces as they moved about the space.” Here, a process of translation manifests through tactile relationships, hand gestures, and physically shaping — giving form. 

“The work Many Shapes of Volume utilised both primitive materials and new technology; the choice of ceramic significant in its timeless nature, along with the integration of modern audio technology within. I find myself drawn to the connection between these two, and I have continued to explore this within the film archives of pottery and the digital art media format.”

In reference to working with these themes within moving-image media, in one of her posts on Vital Capacities, Seo Hye offers this insight: “Craft videos are fascinating as they frequently show pairs of hands making objects from a shapeless form into something beautiful. For me, this formation presents a parallel between the idea of digitally shaping words into the language of subtitles, exploring its poetic nature.”

An image of a rectangular screen. The screen is much longer than it is high. On the screen, there are white subtitles with a black box around them that say “wait total that but I feel so much pressure about this I’ve had enough”. Behind the subtitles, a few blurred shapes can be seen but it is uncertain what they are. The left side is brown and could be brick, the right shows a white shape. The screen is within a black environment which surrounds it.
Seo Hye Lee, Partial Gestures, 2020 (video installation)
A photograph of a television facing forwards and slightly to the left. The television is a black box shape and mounted to a white wall. Coming out from the left and right side of the television are two black tubes. On the screen, the fading words ‘WHAT?’ are repeated three times overlapping on top of each other at different angles.
Seo Hye Lee, What Did You Say?, 2017 (audio-visual installation)

In her latest work, the subtitles are also digitally shaped by square brackets, which typically signify an insertion of additional information for the purpose of description, of sound effects for instance, or clarification — in this case not transcribing dialogue, but the language of sound itself.

As Seo Hye points out, auto-generated subtitles, like those on Youtube, are notoriously inaccurate. But what potential do inaccuracies — or, perhaps, the absence of a definitive notion of correctness — hold in this context? Can miscommunication instead be used constructively, to offer varied experiences to viewers and allow space for new correspondences to emerge? 

In the various subtitles that appear over the same recurring image in Seo Hye’s current work, we might also recall the prevalence of captioned images on the internet, in the form of memes. Familiar images appear again and again in new contexts created through imagined dialogues or descriptions of ambient sounds. It’s within this digital sphere of communication (and miscommunication) that Seo Hye’s work likewise unfolds and opens up new narratives. Here, Seo Hye seeks out modes of communication based on feeling and perception, rather than clarity and accuracy, highlighting the nuances that already exist and embracing them for their constructive potential. At the same time, such gradations render sound plastic, as a form that is not only tactile, but also malleable — intended to be held, touched, felt, as well as shaped and reshaped, like clay.

  • An image of two rows of three small, square drawings each. The top left drawing has a blue background and purple squiggly lines on it. The top middle drawing shows a black curved line with brown shading on it and grey shadow-like shapes. The top right drawing shows various purple dot-like shapes and brown and black squiggly lines. The bottom left drawing is a square coloured in pink with some light orange and blue in it. The bottom middle drawing is a small rounded square-like shape in green made up of circular strokes. The bottom right image shows three blue swirly shapes that emulate a tornado shape. There are small grey dot-like shapes in between the left and middle swirly shapes.
  • An image of two rows of three square post-it notes with various writing on them. The top middle post-it note is split horizontally into three pieces. The top left, top right and bottom middle post-it notes are blue. The top middle post-it notes are pink. The bottom left post-it note is orange. The bottom right post-it note is yellow.
Seo Hye Lee, Sphere of Sound, 2016-17 (illustration)

Check out Seo Hye Lee’s new work in Vital Capacities’ online exhibition from July 22, 2021.