Use external message pump when calling MessageLoopWork (to fix bugs and improve performance)

[cztomczak]

Todo: Compare cefpython/src/subprocess/main_message_loop* files with cef/tests/shared/browser/main_message_loop* files. These files may have been changed after v66 update.

--

Using external message pump will improve performance, but it is also required to get rid of some bugs, like missing keyboard events on Mac/Windows reported on the CEF Forum in this topic, and this.

Examples that call cefpython.MessageLoopWork should implement external message pump.

In CEF there are two functions for running message loop:

1. CefRunMessageLoop (cefpython.MessageLoop)
2. CefDoMessageLoopWork (cefpython.MessageLoopWork) - for integrating into existing frameworks.

In CEF Python we use message loop work in most examples (only PyWin32 uses MessageLoop). According to CEF doc comments CefRunMessageLoop will always have the best performance.

Task 1: GTK example should call MessageLoop() on Linux

The pygtk_.py example could use cefpython.MessageLoop on Linux. CEF will call GTK's native loop and it should work good - tested in idriissha's fork. Currently cefpython.MessageLoopWork is called in a timer.

Update: gtk2.py example uses MessageLoop() by default and MessageLoopWork when --message-loop-timer flag is passed. I tried to use MessageLoop() in gtk3.py, but it didn't work.

Task 2: Use external message pump in other examples

There is issue 1805 in upstream CEF to improve support for a host owned message loop. It is now marked as Resolved. The CefSettings.external_message_pump option and CefBrowserProcessHandler::OnScheduleMessagePumpWork method already landed in master and in branch 2743. See this commit. You can pass the --external-message-pump flag to cefclient to test the new mode and see how it performs.

Example implementation in upstream cefclient:

1. All platforms - main_message_loop_external_pump.cc
2. Linux - main_message_loop_external_pump_linux.cc
3. Windows - main_message_loop_external_pump_win.cc
4. Mac - main_message_loop_external_pump_mac.mm

Doc comments for the new functionality:

  ///
  // Set to true (1) to control browser process main (UI) thread message pump
  // scheduling via the CefBrowserProcessHandler::OnScheduleMessagePumpWork()
  // callback. This option is recommended for use in combination with the
  // CefDoMessageLoopWork() function in cases where the CEF message loop must be
  // integrated into an existing application message loop (see additional
  // comments and warnings on CefDoMessageLoopWork). Enabling this option is not
  // recommended for most users; leave this option disabled and use either the
  // CefRunMessageLoop() function or multi_threaded_message_loop if possible.
  ///
  int external_message_pump;

  ///
  // Called from any thread when work has been scheduled for the browser process
  // main (UI) thread. This callback is used in combination with CefSettings.
  // external_message_pump and CefDoMessageLoopWork() in cases where the CEF
  // message loop must be integrated into an existing application message loop
  // (see additional comments and warnings on CefDoMessageLoopWork). This
  // callback should schedule a CefDoMessageLoopWork() call to happen on the
  // main (UI) thread. |delay_ms| is the requested delay in milliseconds. If
  // |delay_ms| is <= 0 then the call should happen reasonably soon. If
  // |delay_ms| is > 0 then the call should be scheduled to happen after the
  // specified delay and any currently pending scheduled call should be
  // cancelled.
  ///
  virtual void OnScheduleMessagePumpWork(int64 delay_ms) {}

  ///
  // Perform a single iteration of CEF message loop processing. This function is
  // provided for cases where the CEF message loop must be integrated into an
  // existing application message loop. Use of this function is not recommended
  // for most users; use either the CefRunMessageLoop() function or
  // CefSettings.multi_threaded_message_loop if possible. When using this function
  // care must be taken to balance performance against excessive CPU usage. It is
  // recommended to enable the CefSettings.external_message_pump option when using
  // this function so that CefBrowserProcessHandler::OnScheduleMessagePumpWork()
  // callbacks can facilitate the scheduling process. This function should only be
  // called on the main application thread and only if CefInitialize() is called
  // with a CefSettings.multi_threaded_message_loop value of false. This function
  // will not block.
  ///
  void CefDoMessageLoopWork();

[cztomczak]

External message pump implemented in commit f8286e0. However this is still experimental, actually on Linux it makes app x2 slower and mouse context menu doesn't work. Reported Issue #2002 in upstream CEF.

[cztomczak]

It seems that speed of loading web pages is now fixed in CEF v58 with external message pump on Linux. However there are issues with frames being lost (in windowed mode), animation seem laggy even though HTML 5 Peacekeeper test shows the same score for default loop and external message loop.
In off-screen rendering mode with external message pump I didn't see any difference, so maybe this issue is only for windowed mode (however more testing is required). Mouse context menu still doesn't work in windowed mode with message pump.

In v58 the ApplicationSettings.external_message_pump is causing segmentation fault on Linux. Looks like src/subprocess/main_message_loop/ C++ files need to be updated (sync with upstream cefclient).

[cztomczak]

The segmentation fault occurs only when running hello_world.py example. When running wxpython.py or kivy_.py examples it works fine, however there is a segmentation fault, but only when exiting app. I don't see any difference in performance in kivy_.py example with external message pump. The speed of loading pages is now fixed, however I don't see any difference in frame rate. This might still get better, because external message pump seem still to be broken in upstream. See this comment in upstream CEF which expains the issues in details: https://bitbucket.org/chromiumembedded/cef/issues/2002/external-message-pump-on-linux-is-slow-and#comment-36662023

[cztomczak]

Related issue: #442 ("Message loop work hangs in wxPython / Qt examples on Mac").