Portrait by Stefan Schwartzman & Jake Levy
I spoke with Jacky the day after her first theater screening, held at the Whitney Museum to complement its “Dreamlands” exhibition.
Congratulations on the screening! How did it go?
I'd say it was very well received. It was a big thrill for me, because that was my first time viewing the film in a theater. It looked so different at that scale, and sounded better than ever before.
Did watching it that way lend you any fresh insights into the piece itself?
Actually, yes. I found myself noticing details that I’d forgotten about, or perhaps hadn’t fully recognized to begin with. I'm used to the sense of immersion that comes with working on a small screen for extended periods of time. Viewing at the Whitney was equally immersive, given the scale and the sound, but in a completely different way. It was as if my desktop had been blown up 1000%; I found myself paying attention to things like the NPC [non-playable character] Sims in the background, or to various details in the landscape. I think video art tends to work better in a gallery, but to me, Hudson Valley Ruins makes a lot of sense in a cinema. I was pleasantly surprised by how well it translated.
You’ve been screening the film steadily over the past few months, and in a range of settings—online, galleries, off-site installations, museums, and now cinemas. Have you noticed that people watch the film differently in these various environments? Do you feel like there's an ideal viewing mode for your work?
I wouldn’t say there’s an ideal mode, but I do think we tend to watch things differently in a theater than in, say, a gallery—although even in galleries, people often stay and watch Hudson Valley Ruins in its entirety. Prior to this recent theater screening, I’d preferred viewing it in its native environment: the computer. I think the work’s content, the effect that I’m trying to create through editing, is highly informed by the experience of playing the game for long hours in this hermetic environment. I’m trying to encapsulate the kind of dissonance you feel after staring at a screen for too long, and I think that comes across very poignantly on a desktop.
Absolutely. I feel like online viewing lends itself to a particular sort of absorption, maybe a higher sensitivity. For instance, in Hudson Valley Ruins, there’s this strange sound effect—I can only describe it as a digital glistening—that accompanies the Sims’ “outdoor ambience” setting. It’s peripheral, and probably unintentional, but watching at home with headphones, it was pronounced enough that it became this active, even central presence in the film for me.
Yes! That too was about wanting the work to reflect the filming process itself. The idea of recasting this “ambient” sound as an uneasy, monotonous drone is something I stumbled upon by playing the game for long hours and having the soundscape stuck in my head. The sound itself reflects a very hermetic, even dreadful experience. But what’s even more interesting is that because the game’s "nature" and “weather” sounds are sampled from reality—you’re hearing birds, insects and dogs, but in this glitchy, algorithmically compressed iteration—it sort of blends with the noise of external reality and makes your physical environment seem similarly algorithmic. Neither feels entirely “real.”
Well, to that point, the treatment actually reminded me of moments from the soundtrack to The Shining—the tangled electronics and treated noises that occupy the space between musical phrases. Again, the effect’s almost subliminal, but vital in creating a sense of anxiety.
That’s one of my all-time favorite films. I’m really drawn to these sort of psychological experiences of architecture or interior spaces that "haunt" through their past lives, where the emotional, the historical and physical all bleed into one another. The Shining is especially interesting because of the way the production unfolded: filming went on for months and months, and it seems like the cast and crew began to experience the same sense of isolation and madness that drives the film’s plot.
There’s that great on-set documentary by Vivian Kubrick, where it’s clear that Stanley’s going out of his way to provoke people in order to achieve a particular emotional state. He really pushes it to the point of cruelty; your heart sort of breaks for Shelley Duvalle.
Definitely—she seemed almost hysterical by the end of the shoot. As a filmmaker, I can relate to this idea of an intense, hermetic production merging into one’s true-life experience. Although I work on a desktop, the immersion and isolation involved in making something like Hudson Valley Ruins was such that it flirted with madness at times.
Is making the work a taxing proposition for you? Do you enjoy it?
I would say both. I really enjoy the process of filming a virtual environment, the total control you have in inventing your characters and constructing their surroundings, but it does take a toll. I wouldn't describe spending that many hours playing a simulation game as a healthy or harmless experience. (laughs)
This idea of places reenacting past traumas is a motif most common to horror films; along with The Shining, movies like Poltergeist and Hausu come to mind. You’re clearly interested in working with cinematic conventions in your films—I wonder if you could speak to the ideas you’ve adopted specifically from the horror genre?
I am a huge horror fanatic—especially haunted house films of the Korean or Japanese tradition, but also literary gothic horror tales, Edgar Allan Poe, and so on. My films are heavily influenced by those stories, as well as by bildungsroman/coming-of-age type stories, where it’s sort of about "the horror of growing up," with a focus on childhood. This is why most of my films have taken place in this virtual "Hudson Valley" that I’ve built. Growing up there, I was always interested in the region’s history, which is full of myths and supernatural tales—it’s a ghost hotspot, with the Legend of Sleepy Hollow, and also an alleged UFO haunt. I’ve gone ghost hunting in the area and explored a lot of its abandoned locations, and I definitely wanted to include that influence by using it as an evocative backdrop in my work. So what transpires in these recent films hints at this notion of “hauntedness” on that level also.
But what’s notable about your work, especially compared to conventional horror films, is how detached your leads seem to be from the proceedings. Even in undergoing something traumatic, the characters are barely responsive; where the Psycho poster had Janet Leigh mid-scream, Hudson River Valley’s would have the young girl staring quizzically or something.
Part of that might be inherent to the medium—the Sims' expressive range is pretty limited—but unlike a lot of other machinima directors, you've taken this additional step by denying them voiceovers or even subtitles, which essentially reduces their articulations to simple gesture. So there’s this sense of restriction that’s palpable and genuinely unnerving, but also frustrating in a really interesting way: it’s an intensity with no allowance for release—none for the characters and thus none for the viewers.
Right. The Sims’ range is definitely limited, especially compared to the subtleties that human actors might achieve; when they "emote,” it’s quite cartoonish and absurd. I actually retained a few of those absurd animations, mostly in designing the more sinister characters, but for the most part, the Sims do seem removed, almost alienated from what’s happening. Still, I’ve always felt most drawn to the Sims in their idle, “default” modes: the vacant moments of standing, blinking, breathing. They actually seem most human to me in these moments, even as they’re being their most robotic.
It calls to mind Bresson’s Notes on Cinematography: his insistence that actors should “behave” rather than “act,” the suggestion that a film might evoke emotion by resisting it.
It’s definitely a similar line of thinking. Actually, as I was making Hudson Valley Ruins, I thought about Au Hasard Balthazar, which to me is such a beautiful film. Somehow, the donkey’s steadfast, mute temperament while undergoing all types of cruelty becomes a perfect screen. It seems like a very emotional performance, even though it’s just this animal standing there, blinking, looking almost animatronic.
In foregoing dialogue, you’ve allowed Foley sounds to assume a heightened role in the story, functioning almost as a replacement. At one point, I actually dimmed my screen and simply listened to the film, and I was really impressed: as lush as the work is visually, much of the narrative, and even the suggestion of emotion, is carried by the sound design. It really speaks to your editing, that you achieved such resonance using these preset, unassuming noises.
Well, because there’s no dialogue, and because I’ve captured such a vast amount of footage, I actually do edit largely by sound, almost out of necessity. But I think you’re right in suggesting that the edit might form a composition all its own. It’s fascinating to think that when the spoken elements of a film are stripped away, the sound effects and ambient noise become almost like characters unto themselves.
Along those lines, I wanted to talk about Tales From The Borscht Belt, as it seems like you’ve altered your approach a bit—not only in terms of sound, but also structure.
For Borscht Belt, I fashioned three chapters as mini-vignettes, each its own sort of capsule fairytale. I wanted to combine literary folklore with video game and Internet folklore. So, for example, the chapter with the ice babies was inspired by Outside Over There, which is a Maurice Sendak children’s book (and which also served as inspiration for The Labyrinth). But Borscht Belt was also my first foray into including an outside soundtrack; everything up to that point was taken entirely from within the game itself. So I’ve included music from Parasite Eve, a horror role-playing game from my youth, as well as a Boards of Canada song that had been the soundtrack to Salad Fingers, an early creepypasta series I watched on YouTube when I was in middle school.
The film also features a voiceover, which is a new strategy.
Right. The text was written by a girl I knew in high school, ripped directly from her abandoned LiveJournal from the early 2000s—these sort of goth girl, fairy tale poems that became a source of narration.
In exploring outsourced sounds, would you ever consider working with someone on an original score?
Definitely! Looking ahead, the main elements I'm interested in developing with my works are sound-related. I would love to collaborate with the right person on an original score, and perhaps on original texts sometime as well. My favorite music is often more cinematic, ambient composition anyway, so for someone to develop a soundtrack within those constraints would be pretty exciting. I guess I’m just waiting to meet my own Badalamenti.
It’s so funny you mention him—I had this random urge today to revisit that footage where he lays out the composition of "Laura’s Theme.” I actually watched it right before we talked.
That video makes me ecstatic. It’s like the most perfect meeting of the minds.
Yes! It always puts me in a good mood.
Actually, Lynch is an apt reference here on a couple of levels. One obvious parallel is the mise-en-scene: you’re both very precise in constructing a self-contained universe, supplementing narrative with sets and props. I’ve read you used Sims 3 in part because it allowed for greater specificity in designing objects, clothing, decor. But I wonder if there remained any elements beyond your control—limitations you had to work around (or with)?
The biggest one was the element of time. With Sims, each minute of real world time equals an hour in-game; you can fast-forward slightly, but even then, you can't just pick a time of day and start filming. So if your scene takes place at sunset, you have to wait for golden hour and shoot before the light’s gone. I think that type of temporal imprisonment was influential to the mood of the film: it meant that I was locked into this world, living through it for much longer periods of time than if I’d generated the scenes from scratch. In that sense, the process became more like a filmmaker’s than a traditional animator’s, as I was essentially working on set in real time.
There were other in-game particularities that I used to my advantage, none more than the randomness of the Sims’ natural behaviors. Though I might build a set and assign a certain narrative—tell them to sit down, fall asleep, etc.—the Sims never enact a scene the same way twice. Even with their limited expression, there can be a variety of interpretations for a given narrative, so I’d often repeat certain moments, trying to capture the perfect take. I liked that combination of having extreme creative control, while still also harnessing these elements of randomness.
In terms of unexpected results, do any particular scenes stand out as lucky takes?
It was mainly the more choreographed scenes—the dancing sequences, certainly, but also the heavily populated settings, like the cafeteria or the art class. The more characters that are involved, the more you’re subject to random actions and different iterations. Sometimes you have to go through it many times before you get something that works. With the art class, for instance, I wasn’t controlling the students’ paintings and drawings, so I had to make them start new pieces over and over until I saw what I liked. There’s ultimately a limited number of possibilities for what they’ll create, but as I discovered, it’s still quite a few artworks.
It’s so interesting to think that these avatars might actively contribute to the filming process, even the story itself.
It’s something I really enjoy. The other funny thing is that they improve in their tasks over time. There were a lot of scenes that could never be redone the same way, simply because the characters become too skilled. Like the girl falling on the treadmill, or the ballerina dancing less masterfully than her classmates—those could never be reshot. They’ve grown too adept!
I’d imagine it goes the other way as well—if you’d wanted the Rock Chick to shred, you would have had to leave her to her own devices for a while before filming.
Right, I’d have to wait for her to become a rock and roll master. But then, I also wouldn’t be able to record her plucking away poorly ever again. It was funny how her playing progressed over the course of filming: initially, she’d fumble her way through this simple little tune, but after a few weeks, she was suddenly jamming the blues. (laughs)
Still from Fawn's Leap, 2015
I’m definitely interested in exploring digital filmmaking techniques, but I do imagine that I’ll produce a film using live-action elements at some point. I think the atmosphere that I’ve been working in thus far has been greatly influenced by the virtual and the algorithmic, and does document the logic and experience of computer games to an extent. But like we were saying earlier about the Sims’ “ambient” sounds, I also feel like I notice the same phenomena, the same sensibilities and behavior I’ve tried to capture, in waking reality as well. You mentioned David Lynch: I think his more digital/virtual-influenced works are a good example of how that phenomena isn’t discreetly lived on the desktop, but is actually part of the fabric of our reality, how we experience the real world.
You’re currently in the process of designing new Sims environments and characters. Is this something you’re working on for an upcoming show?
Yes, I’m working towards a project for Daata Editions called Anhedonia—a series of six short vignettes that also form a single longer work, similar to structure used in Borscht Belt. I’m in the process now of conceiving new sets and characters. For example, one environment is a Southern-style landscape, with languid weather and kudzu trees; one of the sets I've built there is sort of this group home/halfway housing unit with a lot of strange passageways. So without giving too much away, I’d say there may be a Southern Gothic tale in the works.
As we were saying earlier, I definitely want to continue showing in different environments. Hudson Valley Ruins has a few upcoming screenings, so I’m excited to continue showing in more theater-type environments. But I’m also still very interested in web releases. I really enjoyed making Borscht Belt accessible online, and I think the Vdrome method of screening is a great model. Moving forward, I’d like to pursue them both.
I guess I’m generally drawn to a darker, more lush type of palette, with lots of shadows and glowing colors. But I also think it’s connected to memory and early childhood—that time of year is very evocative for me, definitely metonymically linked to the supernatural realm, to things like ghosts and Halloween. It’s also quite mournful or reflective as the precursor to winter; autumn’s like a last ecstatic burst of experience before a period marked by greyness and deadness. It’s ominous, but still really sublime.
I also find myself drawn in general to totems or ornaments that mark different holidays or seasons—the super crystallized, mass-produced objects we use to organize our experience of time and regulate our life’s passage. Especially now that most of us don’t experience traditional rituals like "the harvest," we remark on time passing through what we see in a Rite Aid window display. You see a familiar pumpkin decoration and remember things you’ve experienced in previous years.
Is it a stretch to compare those ideas to what you've done with Sims—using this consumer platform from your adolescence to recast specific elements from your past? Do you comfortably think of your work as autobiographical in that way?
It’s quite similar. I’m into the idea of using the “stuff” that accumulates around us as a portal to a higher-level articulation of past experiences, even building new worlds by recombining these mass-produced elements. As far as my work being personal, there are definitely autobiographical elements in the work, as well as elements from the personal histories of other people I’ve been close with. The references are direct and literal, even when the plot itself might not be.
To that end, what would you say is your most treasured belonging?
I think it would be this lamp given to me by a close friend. We found it upstate at a junk shop; it has this quilted pattern that looks almost like stained glass when illuminated. I find myself looking over at it a lot while I’m working; it has a very evocative and calming effect.
Have you included it in any of your films?
I haven’t, but I really should. I’d love to make a 3D model of it. It’s funny, though, because we assume that one’s favorite belongings would be physical, but in truth, many of my favorite decorative objects are virtual—like my pretzel bench, which I love. I’ve actually considered making a “real” version of it. I just like the idea of blurring that line and having to consider whether it’s the digital or physical version that’s actually “virtual.”
Interview originally published in Sex Magazine, March 2017