Use any application that has a "minimize to tray" option, i.e. that when minimized or closed it disappears from the process list and to restore it you have to click on some icon in the systray.
examples are the gaim buddy list, azureus, or amule.
open the window (not maximized). minimize it to tray. restore it.
at random (very often, like once every 2-3 tries) the window will be restored as maximized instead of its normal size.
The problem disappears if I switch to GTK+ 2.8.19, OR I switch to Beryl 0.1.4 as window manager.
Recompiling xfwm, and/or the applications, against GTK+ 2.10 doesn't fix the problem.
OS: Gentoo Linux 2006.1
desktop: XFCE 4.4.0
WM: XFWM 4.4.0
GTK+ version: 2.10.6
This looks like a race condition in the app itself. I am not able to reproduce that, but here, it shows just iconified.
On remap, the window has wrong EWMH flags (sometimes it's still hidden ,sometime maximized). Xfwm4 just applies the EWMH state given by the applications (maybe beryl doesn't implement that, so it doesn't show with beryl).
Anyhow, I truly doubt the problem is with xfwm4, because it's simply using the EWMH state specified by the application.
Notet that you should try with another fully EWMH compliant window manager, such as kwin (KDE) or metacity (GNOME). I see no difference in the behaviour between these two and xfwm4, the problem is similar in all three.
Also, can you try updating to gtk-+2.10.9?
after the latest emerge world, even if it didn't update neither xfwm4 nor gtk+ nor any of the applications, I can't reproduce the bug anymore :\