This article is missing information about chroma key virtual production. Please expand the article to include this information. Further details may exist on the
talk page.(September 2023)
On-set virtual production (OSVP), also known as virtual production (VP), or In-Camera Visual Effects (ICVFX), and often called The Volume, is an
entertainment technology for television and
film production in which
LED panels are used as a backdrop for a set, on which video or
computer-generated imagery can be displayed in real-time. The use of OSVP became widespread after its use in the first season of The Mandalorian (2019), which used
Unreal Engine, developed by
Epic Games.
History
Australian film director
Clayton Jacobson first had the idea of improving the
green screen that was then in use, when filming a TV ad for detergent in 2003. Watching his son playing
videogames and seeing the 3D technology used in them gave him the idea. Eventually, in 2016, Jacobson and his son made one of the first prototypes for a virtual production stage in their shed, using a set of LED screens. However, he could not get anyone to take an interest in developing the technology further, so gave up on it. Other filmmakers had also caught on to the idea though, and in 2018 an Australian
cinematographer,
Greig Fraser, used the technology to film the Star Wars franchise
spin-off series, The Mandalorian (released 2019). Instead of using the green screen during the filming stage, the team combined
post-production with the production stage of the series. They installed huge LED walls linked to powerful computers which ran
Unreal Engine gaming software (used for
Fortnite, among others). They called this soundstage "the volume", a term already used to refer to a stage where
visual effects techniques take place.[1]
Since its inventive use in The Mandalorian, which used ILM's StageCraft, the technology has become increasingly popular. Miles Perkins, industry manager of film and TV for
Epic Games and maker of the
Unreal Engine, estimated that there were around 300 stages by October 2022, increased from only three in 2019.[2] Most of these were built during or after the
COVID-19 pandemic, when
lockdowns meant that
production studios had to find ways to produce films without travelling to other locations.[1]
On-set virtual production (OSVP) is also known as virtual production (VP), In-Camera Visual Effects (ICVFX),[3] or The Volume.[citation needed]
Technology
With careful adjustment and calibration, an OSVP set can be made to closely approximate the appearance of a real set or outdoor location.[4] OSVP can be viewed as an application of
extended reality. OSVP contrasts with virtual studio technology, in which a
green screen backdrop surrounds the set, and the virtual surroundings are
composited into the green screen plate downstream from the camera, in that in OSVP the virtual world surrounding the set is visible to the camera, actors, and crew, and objects on set are illuminated by light from the LED screen, creating realistic interactive lighting effects, and that the virtual background and foreground are captured directly in camera, complete with natural subtle cues like
lens distortion,
depth of field effects,
bokeh and
lens flare. This makes it a far more natural experience that more closely approximates
location shooting, making the film-making process faster and more intuitive than can be achieved on a virtual set.[citation needed]
To render parallax depth cues correctly from the viewpoint of a moving camera, the system requires the use of
match moving of the background imagery based on data from low-latency real-time
motion capture technology to track the camera.[citation needed]
Stages that use OSVP include the various StageCraft stages,
Pixomondo's Toronto-based LED stage, which has a longterm lease from
CBS, ZeroSpace's Volume in New York City,[8] or Lux Machina various stages.[2] In Japan, the LED wall and virtual production were used by
Toei Company for its
Super Sentai shows Avataro Sentai Donbrothers[9] and Ohsama Sentai King-Ohger, with the latter also being produced in collaboration with
Sony PCL Inc.[10]
As of March 2023[update] the largest OSVP is at Docklands Studios Melbourne in Australia.[1]