Open Visual Cloud Building Blocks
The four core building blocks used to build a visual cloud service include: Encode, Decode, Inference, and Render. Each of these building blocks represents the underlying technology of the processes that make up a visual cloud service pipeline.
- Scalable Video Technology (SVT) Video encoding technology optimized for x86 processors. Supported codecs include HEVC, VP9, and AV1.
- FFmpeg - FFmpeg is a open source project consisting of a vast software suite of libraries and programs for handling video, audio, and other multimedia files and streams.
- x265 - x265 is a H.265 / HEVC video encoder application library, designed to encode video or images into an H.265 / HEVC encoded bitstream.
- x264 - x264 is an open-source software library and a command-line utility developed by VideoLAN for encoding video streams into the H.264/MPEG-4 AVC format.
- Open WebRTC Toolkit – Open WebRTC Toolkit is an open source real-time media delivery framework, which includes comprehensive media processing functions on video and audio streams.
Decode is defined as uncompressing encoded video data. Decode goes hand-in-hand with Encode, as once your video has been encoded, it needs to be decoded to process or view on a screen. Decode technology can be either hardware or software-based. As new HD video file codecs hit the market, we will see a greater adoption of these new codecs into hardware such as TVs, video cameras, smartphones, and others. As of early 2019, some of the open source decoders include:
- VLC Media Player - VLC is a free and open source cross-platform multimedia player and framework that plays most multimedia files.
- dav1d - dav1d is a new AV1 cross-platform decoder, which is open-source, and focused on speed and correctness.
- Mozilla Firefox - Firefox is an open source web browser that released AV1 support in Firefox 65 in Jan of 2019.
- Android Q - Google* has announced that Android* Q will support AV1 decode.
- Chrome - Google's Chrome/Chromium 70 web-browser is now shipping with AV1 video decoding support
- Deep Learning Deployment Toolkit (DLDT): A key component of the OpenVINO toolkit, DLDT provides a model optimizer and inference engine that supports Intel CPU and GPU (Intel® Processor Graphics) and heterogeneous plugins.
- OpenVINO - OpenVINO™, short for Open Visual Inference and Neural network Optimization, is a toolkit that provides developers with improved neural network performance on a variety of hardware (CPU, GPU, FPGA, VPU) and helps them further unlock cost-effective, real-time vision applications.
- Open Model Zoo - Pre-trained deep learning models and samples for use in OpenVINO.
- Intel® Embree - High performance ray tracing kernels.
- Intel® OSPRay - Ray tracing based rendering engine for high-fidelity visualization.
- Intel® OpenSWR - Highly scalable software rasterizer for OpenGL*.
- Intel® Open Image Denoise - An open source library of denoising filters for images rendered with ray tracing.