I have tried this on both the xfce4.4.0 package for ubuntu 7.04 feisty, and a compiled version from the installer.
xfdesktop seems to use increasing amounts of memory over time, sometimes exceeding 300Mb after 48 hours, for no apparent reason at all.  This makes the system, somewhat unusable.