Saturday, June 16, 2012

Better performance for rich and extensible media platform

The whole performance is still as crippled as ever when it comes to beta testing of the products and there may be a bunch of angry feedbacks from beta testers in order to help the developers improve. Will the new plans and goals from the developers help? Who knows but let’s a look at the goals they’re planning to have.

  1. Maximum performance: The media playback must be fast and responsive, enabling the full power of the hardware while maximizing the battery life on the PC. It may be true that the battery life is quite long when viewing media files but then, certain apps like the Videos app are still taking so much of the battery. Hopefully, the Photos and Music app will not take a lot of the battery while retaining the maximum performance.
  2. Enable a breadth of scenarios: A high performing, highly efficient and extensible platform can enable a wider range of music, video, communications and other multimedia apps.

Performance is one of the key aspects for important activities but here, it is especially critical in multimedia scenarios as well as the user experience. Videos need to play in real time, voice communication needs to fell instantaneous, and all  these tasks need to minimize the drain on your battery.

Another improvement to the performance is when playing audio data without draining so much of the battery, that is, maximizing the battery life during the audio playback. By batching up large chunks of audio data and doing all the processing for that chunk at one time, the CPU can stay asleep for over 100 times longer which can result in dramatically increased battery life.

Unfortunately, this approach isn’t perfect for all scenarios like this since the increased buffering introduces additional delay. Audio and video offloading are just a couple of examples of the ways the media stack is optimized in Windows 8 to provide lower CPU utilization, lower memory utilization and better battery life for Desktop and Metro apps. Hopefully, the Desktop apps that support Windows 8 will not be taking up lots of battery regardless of the interface and interactivity. By getting those multimedia Desktop apps like that, we must know which ones support Windows 8 in addition to supporting the previous operating system. So can this work for video editing and audio editing and especially when using the advanced Desktop apps like the video editors, audio editors, Digital Audio Workstation to name a few? For Metro apps, it’s already a force to support such kind of guidelines in not taking up so much of battery life.

Real-time communication on the computers and mobile devices has seen a huge growth over the last decade. To make improvements to communications on Windows 8, here are the two areas of efforts they are focusing:

  • Enable built-in low-latency media capture and rendering: Low latency is essential for communications apps so Windows supports low-latency media capture and playback into the OS.
  • Support HD cameras to enhance video communication experience: Perhaps this should be done earlier but in Windows 8, the HD videos make your communication experience more real, enjoyable and shaper with HD camera devices. Provided that you have HD webcams like I already have, you can communicate even better. Hopefully, the communication should be smooth when using HD webcams, not choppy performing communications regardless of resolutions.

When you communicate with another user, you expect near-instant responses. However, there is a shame that there’s some kind of delay when communicating with another user like that on certain communication software while using the webcam for video communication.

Another example of improvement for Windows 8 devices is through OS support for HD cameras. New class drivers will work transparently with applications to provide support for HD video features. Windows 8 will offer a consistent, high-quality, hardware accelerated, power efficient media communication experience on devices designed for it.

Argh, when it comes to recording videos on the camera, we including the developers may have experience the frustration of the video being upside down or sideways when viewing it on the computer. Since the video scan pattern if fixed, videos may not be oriented properly when viewed. So when viewing pictures or videos taken using your smartphone or tablet device on your computer, you may have this kind of realization too.

To overcome the problem, the cameras are beginning to author orientation metadata in mainstream file formats when saving recorded videos to storage devices. To ensure the better viewing experience of the videos:

  • The orientation metadata will be supported in MP4 and ASF videos.
  • Videos with orientation metadata are auto-rotated during playback. Well, for smartphone and tablet devices, the problem is already solved so when turning your device in any orientation you want, the video rotation will follow. The only problem is the outside bars of the video and that surely depends on the video editing.
  • The thumbnail for the video with orientation metadata is auto-rotated.
  • Apps with video capture capabilities can easily read and author orientation metadata.

Adaptive bitrate streaming provides a smoother, more responsive video playback experience by enabling the device to adapt to the most appropriate bitrate under varying networking and resource utilization conditions. As a result, startup and seek times can be significantly improved as the first few frames can be delivered at a lower bitrate to reduce buffering time and increase responsiveness. If the network or device conditions change, the device can negotiate a lower or higher bitrate to minimize the buffering or increase video quality. Man, I hope that this feature is not annoying or some sort although this may be comparable to watching YouTube videos in respective qualities. Through the extensibility of Media Foundation Platform in Windows 8, apps can have custom media sources and adaptive bitrate media sources to support new formats. Custom media sources and streaming protocols can also take advantage of hardware offload and content protection.

It seems that you can play media files on the external device from your computer in multiple ways such as using networking for media streaming and especially when using third-party programs that allow video streaming. There’s also video streaming support for HomeGroup users and this is where Windows 7 introduced Play To. Using Play To, you can use it to stream media files to supported external devices from Windows Explorer and Windows Media Player. Well, what about Windows Media Center? Can you stream media stuff on Windows Media Center? I assume that it’s the same or different. In Windows 8, you can share personal media collections and HTML5 media with Play-To-enabled devices at home. With it, Play To is easier to discover and will deliver a consistent, high quality experience. Here are the improvements they have done include:

  • Improved Setup – Play To devices are automatically discovered and installed on your computer for home networks.
  • Improved device experience – Metro style apps only work with Windows certified Play To receivers. These devices are validated to support modern media formats, DLNA standards-compliant and have greater performance. This may include Xbox 360 with the update coming in to support the Play To technology later this year in which I will try to view some videos for better experience myself. The desktop experience first introduced in Windows 7 is added to the Windows 8 Explorer and will continue to support all DLNA DMR devices.
  • Integrated into IE Metro edition – This allows you to stream HTML5 media data to your external device. Hmm, how can this work on Xbox 360?
  • Works with Music, Video and Photo apps

Similar to YouTube, there will be a feature to enable captions of the written transcript in web media players on the web platform in Windows 8. Also, there will be multiple audio tracks in the same video similar to watching movies or playing games with multiple voice tracks. It’s the same for music where the same songs can be sung in different languages.

In summary, Windows 8 media platform is designed to deliver a responsive media experience with great battery life. As media applications continue to evolve, it will enable these experiences to shine across all Windows 8 devices. It may remind the users of Apple AirPlay where you can use ITunes to stream media from your computer running it to AirPlay certified devices. For me, we’ll get to find all these out after this blog post but if the update with Play To support arrives on Xbox 360, I’ll give that a try as well.