For Firefox 57, as part of the Photon Performance project, one of the things we worked on was dramatically reducing flickering in the main browser window. We focused especially on what happens when opening the first browser window during startup, and when opening new browser windows.

To identify bugs, we looked frame by frame at video recordings. This was good enough to file bugs for an initial exploration, but was time consuming, and won't help us to keep these interactions flicker free going forward.

I'm happy to announce that these two interactions are now covered by automated flickering tests that will catch any regression there:
https://searchfox.org/mozilla-central/source/browser/base/content/test/performance/browser_startup_flicker.js
https://searchfox.org/mozilla-central/source/browser/base/content/test/performance/browser_windowopen_flicker.js

These tests currently contain whitelists of known bugs (blocking bug 1421456).

Here is how these tests work:
  • as soon as the window we care about is created, we add a MozAfterPaint event listener to it.
  • for each received MozAfterPaint event, we capture a screenshot of the window using CanvasRenderingContext2D.drawWindow().
  • remove the event listener after the window is done loading and after several Services.tm.idleDispatchToMainThread callbacks to ensure that the window has settled.
  • compare the pixel data of each of the captured frames to identify areas that have changed.
  • for changed areas, see if they are whitelisted, if not make the test fail and dump base64 encoded data urls of the before/after screenshots, so that the test failure can be visually debugged.
We currently cover only the startup and window opening cases, but I expect us to add similar tests to more areas in the future. Each time we spend effort reducing flickering in some area of our UI, we should add tests for it to prevent regression.