after upgrading from xfce 4.10 to 4.12 all of my settings stayed the same but when starting xfce the menu would take 30 to 60 seconds to load. changing the menu's icon corrected the problem.
xfwm4 has nothing to do with that.
Moving to xfce4-panel maybe...