WebRTC for Video Calling: The Dev@Vewd Review

WebRTC made its way into Vewd Core in our latest 4.22 release, alongside the new Service-Based Architecture we covered previously. Here’s a bit more technical information on WebRTC, our motivations for including it, and some of the technical work we did to enable it.

What is WebRTC?

WebRTC (Web Real-Time Communication) is a peer-to-peer way of exchanging audio and video data between applications running in web browsers, thus enabling users to engage in videoconferencing and video calls in the browser itself. In addition to audio/video data, any kind of arbitrary data can also be exchanged, thus allowing for support of additional features like file sharing, screen sharing, telephony, etc.

WebRTC provides Javascript APIs¹ to handle the various aspects of videoconferencing such as getting media from microphones and cameras, setting up connections, transferring data, etc, thus making it easy for a web developer to enable video calling or video conferencing in their web page or application. Base support for WebRTC is included in the Chromium source code, which the Vewd Browser is based on, but it had to be tightly integrated into the TV hardware and a number of other features had to be developed in order to fully enable WebRTC for end users. In the following sections, we take a look at these features and the motivation behind enabling WebRTC in Vewd Core.

Need for WebRTC

Since Vewd Core is available on many platforms, enabling WebRTC means that video conferencing services can create a single app to target Vewd Core, instead of developing native applications for each platform. This, along with the recent surge in the use of video conferencing for remote working motivated us to enable WebRTC in Vewd Core.

WebRTC integration with Vewd Core

Video Encoding and Decoding – The base implementation of WebRTC in Chromium includes libraries for software-based encoding and decoding of audio and video. This works well on a Desktop computer, or even on a high-end handset, but not so well on a big-screen TV with very limited hardware. For improved performance, Vewd Core is using the Vewd-specific, HW-accelerated video decoder integration: HLD, which was already used to decode OTT videos. On Linux, HLD manages hardware decoding of all video, and for WebRTC, HLD decodes video frames into video textures which are passed back to Chromium for presentation. On Android, hardware decoding was enabled by integration towards Android’s MediaCodec class.

Camera and Microphone support – On Linux, support for microphone and camera was added by enabling Advanced Linux Sound Architecture (ALSA) and Video for Linux (V4L), which are frameworks providing APIs for sound and video processing respectively. On Android, camera and microphone support were implemented via Android platform APIs.

Resolution control – A resolution control mechanism has been implemented to limit the resolution of the encoded video frames in order to achieve a smooth framerate on low-end platforms.

Access Control – For Privacy reasons, it is important to let the user control access to the camera and microphone. The generic Permissions systems on platforms such as Android TV are not fine-grained enough, as it considers the Browser Engine as one app. With web apps inside the Browser Engine, we wish to control access per app (e.g. web origin). An access control feature was developed to allow the end-user to explicitly grant or reject permissions for which web apps to access the camera or microphone.

Codecs and Platforms

As per the IETF specifications for video³, browsers must have support for VP8 and H.264 video codecs to be WebRTC compliant. Most WebRTC applications tested were observed to be using VP8, and some also had support for the newer VP9 codec. But some apps such as Skype and Microsoft Teams were seen to require H.264.

VP8 and VP9 codecs are royalty-free and there are significantly fewer licensing issues to be dealt with, unlike H.264 which is not royalty-free, and therefore disabled by default in WebRTC’s base implementation. We expect that when Video conferencing on TV devices becomes more popular, typical TV hardware will start to support video encoding with all these, and newer codecs in hardware. Alternatively, Vewd Core is prepared to be integrated to H.264 and H.265 video encoding when customers request it.

For audio, Opus and G.711 codecs are used, in line with the IETF specification on audio⁴.

Application Compatibility

Many applications have been tested with Vewd Core, including Whereby, Gruveo, BlueJeans, Veedeeo, Google Meet, Jitsi, Talky, Zoho Meeting, BigBlueButton, Goto Meeting, Appr.tc, etc. The Zoom web client also works but Zoom does not use WebRTC for video conferencing; it is based on the same Media APIs and WebAssembly, which are also supported by Vewd Core.

What is WebRTC?

Need for WebRTC

WebRTC integration with Vewd Core

Codecs and Platforms

Application Compatibility

References and Further Reading