Software as a service, infrastructure as a service, platform as a service, communication platform as a service, video conferencing as a service, what about cloud gaming as a service? There have already been several attempts to create cloud gaming (Cloud Gaming), for example Stadia, recently launched by Google. Stadia is not new to WebRTC , but can others use WebRTC in the same way?
Thanh Nguyen decided to test this possibility on his CloudRetro open source project. CloudRetro is based on Pion, the popular Go-based WebRTC library (thanks to Sean of the Pion development team for helping with this article). In this article, Thanh gives an overview of the architecture of his project, and also tells what useful things he learned and what challenges he faced during his work.
Introduction
Last year, when Google announced Stadia, it blew my mind. The idea is so unique and innovative that I constantly wondered how this is even possible with existing technologies. The desire to better understand this topic prompted me to create my own version of the open source cloud game. The result was fantastic. Below I would like to share the process of working on my one-year project .
TLDR: short slide version with highlights
Why cloud gaming is the future
I believe that Cloud Gaming will soon become a new generation of not only games, but also other areas of computer science. Cloud gaming is the pinnacle of the client / server model. This model maximizes backend control and minimizes frontend work by hosting game logic on a remote server and streaming images / audio to the client. The server does heavy processing, so the client is no longer subject to hardware constraints.
Google Stadia essentially lets you play AAA games(i.e. high-end blockbuster games) on an interface like YouTube. The same methodology can be applied to other heavy offline applications like operating system or 2D / 3D graphic design, etc. so that we can run them stably on low-spec devices across different platforms.
The future of this technology: can you imagine if Microsoft Windows 10 ran in the Chrome browser?
Cloud gaming is technically challenging
Gaming is one of those rare areas where constant quick user response is required. If we occasionally encounter a 2 second delay when clicking on a page, that's fine. Live video streams tend to lag a few seconds, but still offer a fair amount of usability. However, if the game is often delayed by 500ms, it is simply impossible to play. Our goal is to achieve extremely low latency so that the gap between input and media is as small as possible. Therefore, the traditional approach to streaming video is not applicable here.
Common cloud game template
Open source project CloudRetro
I decided to create a test sample of the cloud game to test if all this is possible with such tight network restrictions. I chose Golang for proof-of-concept as it is my most familiar language and well suited for this implementation for many other reasons, as it turned out later. Go is simple and evolves very quickly; pipes in Go are great for managing multithreading.
The CloudRetro.io project is an open source cloud gaming service for retro gaming. The goal of the project is to bring the most comfortable gaming experience to traditional retro games and add multiplayer.
You can learn more about the project here: https://github.com/giongto35/cloud-game .
CloudRetro functionality
To showcase the full power of cloud gaming, CloudRetro uses retro gaming. This allows for a variety of unique gaming experiences.
- Game portability
- Instant playback when opening a page; no download and installation needed
- Works in a mobile browser, so no software is needed to run
- Game sessions can be shared across multiple devices and stored in the cloud for next login
- The game can be streamed, or you can play it by several users at once:
- Crowdplay like TwitchPlayPokemon, only more cross-platform and more realtime
- Offline games online. Many users can play without setting up a network. Samurai Shodown can now be played by 2 players on the CloudRetro network
Demo version of an online multiplayer game on different devices
Infrastructure
Requirements and technology stack
, .
1.
, , . , CDN, . , WebSocket, WebRTC.
2.
Stadia, WebRTC. , WebRTC – , . WebRTC – , - API. , , VP8 H264.
, . . Google Stadia , .
3.
, , , . - (RTT). 1 , : , , , , . . , . , , .
4.
, . , . , . - . , WebRTC .
5.
. . LibRetro , LibRetro -, SNES, GBA, PS.
6. , crowd play (deep-link)
CloudRetro , CrowdPlay Online MultiPlayer -. deep-link , .
, . .
7.
SAAS , , . «-» , .
8.
CloudRetro (Digital Ocean, Alibaba, ) . Docker bash-, . NAT Traversal WebRTC, CloudRetro .
: ( , ) , . , .
: . WebSocket.
: . , , , /.
CloudRetro
CloudRetro 1 2, , . 3 HTTP ping. , . 4 . WebRTC.
. Golang . , .. .
:
- WebRTC: , .
- : . Libretro .
- .
- / : , , /.
CloudRetro WebRTC , , Golang, WebRTC. , .
WebRTC
WebRTC API.
NAT Traversal
WebRTC NAT Traversal. WebRTC . – , NAT- ICE. API WebRTC IP- STUN (TURN), .
CloudRetro . , . , . IP- , NAT.
Cloud Gaming. , . . CloudRetro , , . WebRTC NAT Traversal , .
– , . , VP8/H264, , .
, , + + , . , . – , , .
, , . , , . c Pacman’, .
Pacman
, , . Opus . , RTP (Real Time Transport Protocol – ). , mp3 aac, . 5~66,5 .
Pion, WebRTC Golang
Pion – , WebRTC Golang. C++ WebRTC, Pion Golang- WebRTC , Go, WebRTC.
. STUN, DTLS, SCTP .. QUIC WebAssembly. , .
Pion, , , WebRTC. , http://pion.ly/slack – .
CloudRetro Golang
Go
Go
Go, . , GoRoutines . . Golang (game tick). , . , , . , , . game tick’ , .
func (e *gameEmulator) gameUpdate() { for { select { case <-e.saveOperation: e.saveGameState() case key := <-e.input: e.updateGameState(key) case <-e.done: e.close() return } } }
Fan-in / Fan-out
Golang CrowdPlay Multiple Player. , . . , .
Golang
Golang . . Go – , . . , .
, garbage collector Golang , - . .
CGO
VP8/H264 Golang Libretro . C Go CGO. Dave Cheney. , :
- CGO, Golang RecoveryCrash;
- , CGO.
– , - . Pion Pion. . API, WebRTC Pion, . , , (P2P).
, P2P- . , IP NAT . , P2P- WebRTC.
CloudRetro , , , -. , , , , , , , . . , , .