This video is show how it lost focus on scrolling on focused window. https://yadi.sk/d/ZfaUSJW13GZBTt
I've seen that occurring when using a virtual machine, but that's not an xfwm4 bug but rather an input driver issue, looks like the pointer sometimes "jumps" outside the window, which causes the event to occur onto the root window instead. Try disabling "Use the mouse wheel on the desktop to switch workspaces" in the "Workspaces" tab of xfwm4-tweak-settings dialog.