I have the "Quit" button placed on my panel and it has happened several times that I have pressed it by mistake. I think it would be a good idea to display a confirmation dialog before quiting.
Of course. But, it already does that. So there must be something else going wrong. The only thing I can think of now is that the session manager crashes. Could you try to run 'xfce4-session-logout' in a terminal and see what happens? Also, maybe there is something in your ~/.xsession-errors file to help us find out what is going on.
Without additional information there's nothing I can do about this report. I'm closing this as WORKSFORME. Please reopen if you can provide more information.