Description

Did you know, communication with astronauts on Earth’s moon has a latency (delay) of 1.25 seconds? Due to the protocols used, live video in the Decentraland world is delayed at around 10-20 seconds. This makes live events infeasible. For example, live video calls or interactive events take 20 seconds to get a response. This project introduces new ultra low latency streaming components to the decentraland SDK and builder to support ultra low latency video. Latency will reduce to ~1 second. These components will allow scene builders to easily incorporate ultra low latency video in their scenes. ### Background In recent years, global investment in technology and infrastructure underlying ultra low latency video streaming has been significant. Live streaming is set to explode in the coming years with low latency video being a big driver behind new applications in entertainment, gaming, media, education, acedemia. Ultra low latency video streaming typically operates on a sub 1 second delay. This project will integrate this hot, deep technology into the Decentraland platform. ### Competitive Advantage Metaverses such as Facebook's Meta already have native low-latency video and are building a strong portfolio of apps based on this technology. Other Metaverses do not yet have low latency video streaming technology. ### Experimental Prototype A video demonstration of the components I have developed is here: https://youtu.be/ml7nlTEk3p8 This video shows a working prototype of an experimental Decentraland SDK component featuring a low latency video stream. The video is streaming from North America to a video streaming server in Hong Kong (on Microsoft Azure) and then back to North America and into a Decentraland scene running in the browser. The clock times on the right side are the current time whereas the time on the left, in the Decentraland scene, are latent due to the transit time around the planet and the subsequent decoding and rendering of the video. Total time is demonstrated to be around 1 second. A 5 year old laptop with no GPU was used for the test. The video resolution is set to 640x360. The experimental prototype includes modifications to the Decentraland Unity Renderer and Kernel and adds a new video component to the SDK. ### Example Use Cases It is difficult to imagine a virtual future that runs on a 20 second delay. Use cases that need lower latency might include: - Any live interaction such as video calls. - Live events where people communicate from the virtual world to the real world. - Any use cases that include augmentation of the virtual world with a real object (augmented virtuality). - Virtual workspaces - Virtual learning spaces that requires interaction - Remote control of real world devices such as drones. (yikes!) - Immersive commerce - Gaming applications ### Existing DAO pre-Proposal An existing pre-proposal that covers this feature is here, from Tobik#7989. Note however this is a request for a protocol change, the proposed feature would likely be an additional component so that backwards compatibility is maintained: https://governance.decentraland.org/proposal/?id=0627a110-2fd6-11ec-ac84-77607720a240