.png){kind=small}
PRODUCTS & IMPLEMENTATIONS
The Jestr platform has a wide range of uses, from serious work to creative play.
For professional applications, it’s a precision controller. Jestr’s unique interface is easy to learn and use, yet precise and powerful enough to handle complex tasks. As an advanced controller, it opens up new ways to release the power of sophisticated smart devices, such as LED lights.
Because the Jestr interface tracks so closely with expressive hand motions, it’s an instrument for new forms of art and entertainment. It’s ideal for live performances and for creating multimedia environments. It’s adds a new dimension to immersive gaming. And, it’s the ideal platform for shared experiences when, in the future, everybody in the audience shows up wearing their personal spatial computing headsets.
Importantly, whether used for work or play, the Jestr interface—the hand gestures and holograms—are fundamentally the same. Once you learn one application, it’s easy to learn the next.
FILM PRODUCTION
LIGHT WAVE FOR FILM
Jestr began as a tool designed by professional cinematographers for film production. It has technical features that lighting designers and gaffers need, such as easy switching between Kelvin and HSL modes. And it can run up to eight universes talking directly to Artnet-enabled devices such as the Lumen Radio Stardust.
But the biggest advantage is that Light Wave makes it easy to light the set and actors expressively without thinking about dimmer values or channel numbers. Instead of having to communicate to a technician about a change to a specific light, the creative can turn toward that light and make the change with a simple hand gesture.
Record and play-back of lighting sequences is also executed with simple gestures, rather than the complex programming required with current control systems. With Light Wave, the creatives can record a sequence, see how it looks in the room, and if they don’t like the result, make quick changes and try another sequence. They stay in the creative flow and can focus on the artistic effect, rather than get bogged down with programming details. This process is also highly efficient and cuts significantly the time it takes to set up a shot.
Jestr’s single-hand mode expands the possibilities. Each hand can be assigned to a different color, for example, or to separate pre-recorded sequences ready to be triggered. Or all lighting effects can be controlled with one hand, leaving the other hand free to operate the camera.
Light Wave being used on set for Paramount Pictures' Hip-Hop Christmas, August, 2022
VIRTUAL PRODUCTION
Virtual production is a new frontier in the film industry enabled by new technologies for blending live action, physical props and set pieces with virtual objects and full-screen projected backgrounds.
In the same way Lightwave can control set lighting without requiring a technical specialist, Jestr allows the creatives on set (the director, DP, and production designer) to pre-visualize all virtual and physical objects together in the same blended space and manipulate them without requiring extra programming. Because all controls are in the headsets, they don’t have to walk over to the computer desks on the periphery of the set to look over the shoulder of a technician. Instead, they can remain in the center of the set while making adjustments to virtual objects.
JESTR PILOT
Jestr is ideal for piloting any machine that moves in 3D space, such as a camera dolly or crane. Or drones.
Jestr drone control uses our proprietary MAGI (Multi-Axis Gesture Interface) control system, a virtual 3-axis joystick. Direction, speed, and complex turns and maneuvers are controlled concurrently with fluid hand gestures. In purely virtual space, Jestr is a drone flight simulator. When commands are sent to an actual drone, it’s an advanced controller that can do as much, if not more, than a hardware joystick controller.
DOLLY PATHS
Robotic camera control is a technology used by many film productions. It’s especially important for virtual production.
Motorized dollies, such as the Agito, are wheeled robots mounted with booms. They can position the camera head anywhere in physical space, at any elevation, and pan and tilt the camera to shoot at any angle. All movements are controlled digitally, transmitted wirelessly, and they combine to create a dynamic and repeatable camera path.
A classic filmmaking technique is a “finger rehearsal” in which the director and DP walk through the scene while the camera operator holds up a pinky and index finger to represent the camera frame and to mime the movement, rotation, pan, tilt, speed, and height of the camera—all factors that comprise the camera path. This simple exercise helps the creatives make decisions about the shot, plus it communicates a rich set of directions to the crew.
With the introduction of digitally controlled camera dollies, the camera path can be fully automated. But automation requires substantial programming by a trained technician before the creatives can see the results.
Now, using Jestr and the same two-finger pose, virtual camera paths can be drawn in real-time during the finger rehearsal and instantly played back to steer the camera. The creatives can design the shot directly—without stopping to communicate camera moves to a technician—and have the flexibility to quickly try out new ideas. This also results in a natural and cinematic camera path rather than the mechanical, stiff movements typical of camera paths programmed on a computer.
And because the path is drawn in virtual space and saved, it can be edited, refined, expanded or shortened during pre-planning or during live shooting with the actors. It’s also possible to superimpose multiple camera paths in virtual space and show the crew every camera move planned for that day. This saves time, aids collaboration with the crew, and reduces costly delays because fewer instructions need to be given during the actual shooting.
Jestr gives filmmakers hands-on creative control of their technical tools, which frees them for more artistic experimentation and expression. Putting advanced technology directly into the hands of artists is a recurring theme of all Jestr products.
Jestr holds a patent on robotic camera control and will be working with film industry pioneers to implement this groundbreaking tool for all types of commercial film production.
PERFORMING ARTS
LIGHT WAVE FOR EVENTS
In the process of developing and testing Lightwave for film sets, we discovered that it is so intuitive that people with no lighting experience can learn it in a few minutes and create a spontaneous light show. They say it makes them feel like a wizard casting spells. We experimented with Lightwave at music events, dance clubs, and festivals around the world—in Prague, Malta, Barcelona, Los Angeles and San Francisco—and were surprised at how quickly these party people found ways to shape Lightwave to their own expressive styles.
In some cases, our newly recruited wizards went on to use Light Wave to perform as lighting designers at shows and music venues. None of these wizards had prior experience with professional lighting or lighting controllers.
Now, for the first time, because technicians aren’t needed to create intricate light shows on the fly, Jestr is putting sophisticated lighting arrays into the hands of the artists.
Highlight reel of Light Wave at festivals and events in Europe and the US.
BALL
Ball is a game, a training tool, a controller, a dance partner. It features a single ball floating in virtual space and governed by virtual physics. You generate the ball from your hand and bounce, drop, or throw it around the room. Using our proprietary continuous controller MAGI (Multi Axis Gesture Interface), you pilot the ball to steer its direction, tilt, spin, and speed. When struck with specific hand gestures (“pose taps”), it responds differently, shooting bursts of color and triggering sounds.
The ball can be loaded with sequences and control information. It changes appearance to reflect its current state and whatever sequence it holds.
Ball is fun, but it’s also a radically new controller. It’s movements send out OSC signals that can be directed to control a music platform (Ableton, Logic Pro X) or multimedia software (Resolume, Touch Designer). Coupling virtual physics and expressive gestures with the ability to record sequences and play loops generates endless possibilities for DJs, VJs, lighting professionals, multimedia entertainment, and flow arts.
Ball is a demonstration of a key Jestr design philosophy: use the simplest elements to achieve rich results. By focusing narrowly and delving deeply into what can be done with virtual physics in 3D space, a single ball becomes a highly expressive and versatile tool and toy. This design philosophy is the secret for how the Jestr interface overall can do so much while keeping the interface so simple anybody can use it.
Our Hololens 2 version of Ball, 2022. New version coming to the Apple Vision Pro in 2024.
CIRCUS BOUNCE
Circus Bounce embraces all the new forms of creation and group entertainment that are only possible in the blended reality of spatial computing. It merges lights, sound, musicians, dancers, and audience members in the same live performance in the same physical space.
As a practical application, Circus Bounce is a show controller. It gives an operator one-point control over a complex array of multimedia for stage productions, theater, or film production.
Circus Bounce choreographs all the elements of immersive, location-based entertainment. The possibilities are limited only by the imagination of the designers and producers.
As spatial computing headsets become more affordable and ubiquitous, Circus Bounce will be the platform for explorations and new art forms based on truly interactive, group creativity.
GAMES
Jestr is a platform for immersive gaming that includes other people in the room, whether they are wearing headsets or not. Compared to VR games where a solo player wearing a headset is isolated in a virtual world only they can see, Jestr games blend the virtual world with the real world. Lights, video walls, sound effects, music, and even smoke machines are integrated into the game. Players do magical things and the audience shares in the fun.
Jestr games is a way for people to engage with spatial computing as groups, opening new opportunities for lively in-person events and location-based entertainment. Participants can be taught how to work Jestr in a few minutes before entering the arena. People not wearing a headset join in with cheering and encouragement, and share in the beautiful patterns and sequences of colored light generated by the players’ maneuvers.
Games are an excellent training tool for learning Jestr because the Jestr command language is the same across all applications. Also, because gamers typically push technology to its limits in the pursuit of competitive advantage, we use games to “battle test” new Jestr features and builds. Gamers discover new ways to use Jestr, and we listen and learn from them.
COLOR HUNTER
Color Hunter is an interactive game that makes a game board out of LED lights set up around the room. It’s modeled after the classic Wack-a-Mole. It challenges the player wearing a headset to quickly spot the target light—recognized by a color that stands out from the other lights—and zap it with their holographic magic to win points. The target lights flash only for a few seconds before the system randomly picks another target light. A game counter increments the score with each “kill.”
Other people in the room see the lights change and yell and point to help the shooter and cheer when she zaps a light and wins points.
Each shooter plays three increasingly challenging rounds. With each round the pace accelerates, the colors change, the music intensifies. When the three rounds are over, all the lights instantly brighten to 100% white, and then after 10 seconds, dim down to 50%. It’s hard to convey what a powerful immersive effect these progressive changes in lights and music have on everyone in the room, not just the player in the headset.
The headset runs the game by sending DMX and OSC signals. The OSC is sent to an external laptop running Ableton such that the music and SFX are interactive. Each zap of a light sends a new sound effect to the room’s speakers. The better you are at the game—the faster you blast the targets—the more upbeat and dynamic the music becomes. The player is a de facto DJ controlling the music in the room. We see Color Hunter as the precursor to a full application for DJs to control lights and music together in new ways.
Color Hunter Trailer, "How To Play", and Tournament Event in Prague
WIZARD BATTLE
Wizard Battle is a series of shooting games we’re planning that are modeled after dodgeball and capture the flag. As we add capabilities to Jestr, we’ll incorporate them into Wizard Battle to amp up the challenge and intricacy of the games.
In the first Wizard Battle game, two wizards (or two teams) bombard each other in a blended arena, throwing holographic magic to capture the opponent’s lights (or tower or battlements), gaining power with each capture. Following the wizard theme, we add spells and potions and shifting allegiances to challenge the players’ strategy and skill. Each level-up is more complex and dynamic.
WIZARD JOURNEY
Wizard Journey is our vision of a totally immersive location-based role-player experience. It’s a story-driven game that incorporates all the latest features of Jestr to challenge the skill, luck, and strategy of gamers in new ways.
Wizard Journey merges Hollywood level set design, music, and production values with state-of-the-art mixed reality to create a dynamic fantasy world. Gamers navigate this world, solo or as a group, in search of adventure or to fulfill an assigned mission, encountering along the way a cast of digital characters—friends, foes—and other “realies” pursuing their own journeys.
Wizard Journey borrows from open world video games genre, but goes beyond to create a new type of group experiences in a shared physical space in real time. It’s one example of how Jestr opens new possibilities when mixed reality is enhanced by a universal gesture language.