Introduction
Modern MSE and EME based media players such as dash.js can run on several platforms including Desktop Browsers, Smart TVs and Smartphones. While maintaining a single code base for the majority of the relevant platforms offers great advantages, it also introduces challenges. Different platforms and browser versions often exhibit varying behaviors. In the worst-case device-specific fixes or workarounds need to be implemented.
As part of our work on dash.js we need to make sure that changes to the code like new features and important bugfixes do not introduce any regression. Fixing one bug can easily introduce numerous new bugs. Manual testing of the dash.js functionality on various platforms prone to errors, too time-consuming and does not scale well.
The idea behind Functional Testing
To avoid manual testing of basic functionality such as play, pause, seek as well as advanced functionality such as handling of media timeline gaps, we implemented a functional testsuite for dash.js. The testsuite is based on the Karma Testrunner that integrates well with Selenium and Appium (Unfortunately, Karma has been deprecated, so we are looking into Web Test Runner as a replacement). Combining the functionality of these three libraries, we can execute the functional tests on Desktop Browsers, Smartphones, and even Smart TVs. Basically, all devices that support WebDrivers to control user agents are a potential target platform. As an example, the Shaka Team has open-sourced a generic WebDriver server to support the test execution on Tizen SmartTVs.
Functional Testing in practice
Let’s look at a concrete example of executing dash.js functional tests on Desktop Browsers. In this case, we are using Selenium Grid with three connected nodes: a MacOS machine, a Windows machine and a Linux machine. Selenium ships with a neat web-interface that provides an overview of the connected nodes and the running sessions:
The video below shows the execution of a test run that targets all three platforms and multiple browsers on each of the platforms.
After the test execution, the test results for each platform and each browser are summarized in a test report. In the example below, we executed a test that adds an MPD anchor to the manifest URL to start playback at a specific time:
Based on the test results, we can identify potential problems or regressions on a specific platform and react accordingly. The functional testsuite is also a great asset for dash.js users as it can easily be extended to add more testcases or customer specific test streams.
Next steps
The work on dash.js version 5.0.0 has already started. Next to important improvements such as a refactoring and re-evaluation of the ABR algorithms and the addition of new features like Adaptation Set switching, the dash.js developers are looking into further improving the testsuite of the player. New tests are added to validate the various dash.js settings. Moreover, scenarios like limiting the video resolution according to the portal size and preloading media segments into a virtual buffer are to be covered by functional tests. Furthermore, vendor-specific tests are added to verify the integration of dash.js against other frameworks and APIs.
If you require any further assistance regarding our DASH activities or dash.js in particular, feel free to check out our website and contact us.
Meet us at IBC 2023
Let’s connect at the leading conference for broadcast, streaming and innovative video technology and experience our latest developments. The IBC Show 2023 will take place from September 15-18, 2023 at the RAI Convention Center Amsterdam. Visit us at Booth 8.B80 to get in touch and experience our solutions.
More information can be found here: https://www.fokus.fraunhofer.de/go/ibc.