Trends in multimedia domains

Based on the experiences for the concluded “Open multimedia tools 2014” project, a list of seen trends and interesting technological development directions are presented.

  • Real-time collaboration: The advent of the WebRTC/rtcweb standards has already enable new browser base collaboration tools, without the need for plug-ins or other additional installed software in clients. It seems though that the standards aim for medium to low audio-visual quality. They will support negotiation for high quality media streams, but it remains to be seen if any of the popular browser will implement support for such streams. The none-commercial musical collaborations tools applied by UNINETT during experimentation lacks standardized signalling support. A potential improvement of these openly available tools is to add WebRTC based signalling and media transport. However keeping end-to-end delay to a minimum may turn out to be challenging. Good news with respect to delay is the growing availability of fast cameras, displays and projectors. Popular consumer interface standards like HDMI, DVI and USB now support bit-rates which enable image and audio capture at such high frame-rates, frequencies and resolutions that the inevitable buffering in the devices goes close to unnoticed.  Consumer standards also imply consumer products which again imply lower investment cost.
  • Multimedia search: Search in multimedia content is very much about extracting and/or adding text based meta-data to the multimedia objects. Most search engine index text-only efficiently. Extracting meta-data manually is time consuming, but results in high quality data. Extracting meta-data automatically is complex and computationally demanding, and often results in low quality data. A lot of research is ongoing in the multimedia meta-data extraction and classification domain, where deep learning (by neural networks) is a promising area. The big could-service players (Google, Facebook, Amazon, …) have already running systems with reasonable extraction accuracy, however these systems are not straight forward to access and may not be utilized in all contexts due to for instance privacy concerns. Content ownership has turned out to be a challenge with respect to search functionality for UNINETT’s video collections. Managing  authorization and access lists for content is and will continue to be a challenge since UNINETT’s users by default own all content they submit to the UNINETT multimedia storage facilities (i.e. lecture capture systems). Encouraging user to share content openly is an option, but care must be taken with respect to third party content included in the users content. Search systems which handle meta-data extraction as well as copyright issues in automated, efficient and trustworthy ways are still to be made available.
  • Streaming infrastructure: Video and audio streaming has been around for a decade and infrastructure technology has converged towards chunk based unicast streaming over network transport protocols (HTTP/TCP).  The “Dynamic Adaptive Streaming over HTTP” (DASH) standard has been available since 2011, a standard developed based on earlier protocols from Adobe (RTMP), Apple (HLS) and Microsoft (Silverlight).  Unicast streaming has obvious scalability issues and has contributed to the rise of content delivery network (CDN) providers (e.g. Akamai, Level3) which provide the necessary server infrastructure to fan-out the unicast streams. Unicast DASH-like streaming of to-days encoded multimedia content works well in wireless access networks since content recovery mechanisms for unicast flows are operational. However, during live streaming events where many users within the same wireless domain request the same stream, known as “flash crowds” (e.g. sport events, conferences, news broadcasts), unicast applied for broadcasting struggle to utilize network resources, which often lead to long delays and poor quality of delivered content. As client devices continue to improve their ability to consume high quality video and audio (as well as improve in capturing at high quality) more efficient utilisation of wireless resources is very much desirable. Multicast technology is in principle promising, but has to-day still many issues related to current media encoding standards and wireless access point functionality. Peer-to-peer live streaming (P2PLS) is on a rise, supported by the fresh WebRTC standard. P2PLS can reduce the need for CDN support, but will not likely help much with “flash crowd” challenges. Hence live streaming of high quality multimedia content in crowded settings is still to be handled properly in the future.

One thought on “Trends in multimedia domains

Leave a Reply

Your email address will not be published. Required fields are marked *