It’s almost as if you were there: immersive media let us dive deep into images and videos – sometimes to the point where we completely forget the world around us. In order to transform this new kind of media from a niche phenomenon to a widespread practice, the Ars Electronica Futurelab and four partners from all over Europe are working on a common research project, Immersify.
The time frame is set for two and a half years, five partners from all over Europe are working on it and the objectives span a wide range from technical aspects to content creation – this is Immersify, a European research project focused on immersive media. “First and foremost, we want to present visual or audiovisual media contents in the most impressive, realistic and relatable way possible,” Roland Haring, Technical Director at the Ars Electronica Futurelab, sums up the objectives of the EU-project.
To turn these goals into reality is not as easy as it might seem: To create media in the most immersive way possible, the right video technology is needed, as are better codecs, displays, media display rooms, specially created content, artistic visions and more. There is a lot more to it than simple details: “Immersive content is not just normal video content in very high resolution,” reports Mauricio Alvarez-Mesa of Spin Digital Technologies GmbH, one of the partnering institutions. “Content creators let the audience explore the micro- and macro-cosmos in unprecedented ways, and present elements of reality that are hidden to the naked eye.”
Aim high
A lot needs to happen before this is possible. Since creating beautiful content increasingly means working with huge amounts of data, a new data compression technology is needed. In order to be able to play the newly created content, media players and other formats should support as many different technical environments and devices as possible. Technologies like ultrahigh definition or CGI in 2D or 3D have to be freely combinable – and all of these developments have to stay accessible for creators to use and audiences to enjoy. Demonstrations at the Ars Electronica Festival and the Marché du Film in Cannes show: Immersify should not be developed behind closed doors.
Visualization Center C, 3D dome theater. Credit: NVAB
“One goal of immersive media must be to create content and experiences that feel as close to reality as possible”, explains Erik Sundén of the Swedish Visualization Center C. “That’s impossible without state-of-the-art technology.” He and the Immersify team even go one step further: Immersive media should not only be as close to reality as possible, but also enable participation and interaction. “The goal is to let the audience participate, explore, and interact with the content”, says Mauricio Alvarez-Mesa of Spin Digital.
Immersify at the Marché du Film – Festival de Cannes. Credit: Eryk Skotarczak
That turns into a real challenge as soon as the audience consists of more than one person. How can you offer an immersive experience to many users at once? “Individual displays are a serious limitation to start reaching significant audience figures”, shares Jérôme Paillard of Marché du Film in Cannes. However: “In a few years, we could see some cinemas with immersive and even interactive rooms, where the spectators will be able to watch, all together, a film from inside, just seated or walking in the center of the room.” Media rooms like the Deep Space 8K in Austria or the Visualization Center C in Sweden already offer a little taste of what might be possible in a few years.
One big challenge, many partner institutions
All things considered, how is it possible to develop immersive media further? One strategy is to focus on the most important core elements. “I think that technological progress, approach to narrative and content are now crucial for immersive media”, shares Maciej Glowiak of the Poznan Supercomputing and Networking Center (PSNC). For this reason, Glowiak and his team are working on technical and scientific questions at their in-house media lab, all the while experimenting with new content.
PSNC – Cave 2.0 Visualisation Installation. Credit: PSNC
Additionally, Immersify concentrates on bringing experts from varying fields together to be able to cover a lot of ground together. “We want to involve several partner institutions who all work with different media infrastructure,” Roland Haring of Ars Electronica Futurelab explains. “This way, we can try to improve the quality of our media with regards to content as well as technologically, in order to develop reference projects.”
Prima Materia / NOHlab. Credit: Robert Bauernhansl
Besides the Ars Electronica Futurelab, Spin Digital Video Technologies GmbH from Berlin and the Polish PSNC, partners include the Marché du Film – Festival de Cannes from France and the Visualization Center C from Sweden. With media rooms like Deep Space 8K in Linz or the 3D dome theater in Norrköping, media labs with lots of energy to experiment, know-how with regards to codecs, infrastructure and content, the five partner organizations are perfectly equipped to tackle the challenge.
Impressive first results
First results already show that the effort to bring partners from all over Europe together pays off. Visualization Center C is, to name just one example, using their in-house dome theater with 6P-RGB-laser projection system to create interactive films. “We are currently developing content production with an interactive storyline for high-resolution data of the surface of Mars,” reports Erik Sundén. “The people in the audience in various forms decide on what they want to explore.” PSNC in Poland on the other hand is working on audio installations, among other things: “We invited the jazz band Anomalia to cooperate with us. Several musicians were arranged in a circle and we recorded each instrument independently”, Maciej Glowiak describes one of the current projects. The band was also recorded with a 360° camera. “Thanks to this, we were able to synchronize the sound with the image and now turning our heads using HMDs (head-mounted displays), we can hear clearly where the sound of a given instrument comes from.”
Immersive project by Visualization Center C. Credit: Visualization Center C.
At Deep Space 8K, artistic installations like “Singing Sand 2.0” by Tadej Droljc are explored just as much as projects dealing with cultural heritage. At the basis of point cloud scans at the pyramids of Gizeh, a 360° video in ultrahigh definition was created – something that would have been nearly impossible to show without the newly developed codecs and technologies. “Until the end of Immersify, we want to reach 12K times 12K definition,” Roland Haring of Ars Electronica Futurelab reveals. “That’s very much on the limit, of course, but it’s at least a possibility to push what the medium video has to give right now really right to the border of the possible.”
Sounds promising. For those who are curious to see immersive media for themselves now: At the 2019 Ars Electronica Festival (September 5 – 9), it will be possible to find out just how immersive media can get. See you there!
About Immersify: The European research project works on creating more immersive media. All in all, there are five partner institutions involved: the Poznan Supercomputing and Networking Center (PSNC) from Poland, Spin Digital Video Technologies GmbH from Germany, the Ars Electronica Futurelab from Austria, Marché du Film – Festival de Cannes from France and the Visualization Center C from Sweden. Immersify was started in October 2017 and runs until March 2020. The research program is financed by the EU program “Horizon 2020”.
Dr. Mauricio Alvarez-Mesa is the CEO and co-founder of Spin Digital, a company specialized in video compression for ultra-high-quality media. Mauricio holds a degree in Electronics Engineering from the University of Antioquia in Colombia, and a PhD degree in Computing Engineering from Polytechnic University of Catalonia in Spain. Mauricio has more than 15 years of experience in the field of video compression and high performance computing. He has worked as a researcher for several international institutions such as the Polytechnic University of Catalonia in Barcelona, IBM Labs in Haifa, Fraunhofer HHI in Berlin, and the Technical University of Berlin. He has been part of several international research projects combining the media industry and the ICT sector. Currently he is the technical coordinator of the Immersify project.
Maciej Glowiak, the Head of the New Media Department at Poznan Supercomputing and Networking Center (PSNC) is the coordinator of the Immersify project. His work is focused on innovative solutions and research related to ultra-high definition and immersive media production, processing and network streaming. He is an author of many scientific articles and publications and leads a group of multimedia software developers, TV and experimental content producers, media engineers as well as electronics designers. Maciej Glowiak was involved in many Polish and European research projects and he is active member of many international consortia and research groups.
Roland Haring studied Media Technology and Design at Hagenberg University of Applied Sciences. Since 2003 he has been a member of the Ars Electronica Futurelab and one of the driving forces behind the lab’s R&D efforts. His activities include research and development in several large R&D projects with academic, artistic and commercial partners and collaborators. Currently Roland Haring is the Technical Director of Ars Electronica Futurelab and co-responsible for its general management, content conception and technical development. With his many years of experience in the (software) technical management of large-scale, research-intensive projects, he is an expert in the design, architecture and development of interactive applications.
Jérôme Paillard started his career as an oboist and as a producer and CFO of the French classical record company Erato. In 1991, he became General Manager of Erato Films alongside Daniel Toscan du Plantier where he executively produced over a dozen feature films. He joined the Festival de Cannes as Executive Director of the Marché du Film in November 1995 to oversee the development and management of what is now heralded as the world’s leading film market. He is also the Co-Director of Ventana Sur, the first market specialized in Latin-American films, and the founder of Cinando.com, a leading online resource, communication and screening platform for the industry.
Erik Sundén is Technical Manager at Visualization Center C, which is research and science center in Norrköping, Sweden, conducting a unique mix of leading visualization research and public outreach activities. He is responsible for the large scale technology infrastructure within this center, with virtual reality arenas and labs, and the main facility being a state-of-the-art dome theatre for approx. 100 people, which is used for interactive and playback shows for the public but also for research projects. He is also developing interactive productions, research softwares and general tools for the center, to showcase and widen the use of the immersive facilities.