Unfortunately Xfce4-mixer does not support changing the volume for particular applications like pavucontrol does.

I think it would be nice to have pavucontrol as a link in the panel plugin's menu or in the mixer application. 

Another way would be implementing this feature into the mixer application. But I guess this is much more work ;)

My statements are based on my experience with Xfce 4.10