Currently the plugin seems to be the only (nnon hackish) way to enable pulse audio volume control through 'multimedia keys' in XFCE, as used to be possible with xfce4-volumed for alsa.
I think the two functionalities (panel plugin and keyboard shortcuts) should be decoupled, because a user might want to have only keyboard shortcuts and not the panel plugin (vice-versa is already possible from the plugin preferences). Also it would be nice to to enable users to actually configure the keyboard shortcuts.
Please use xfce4-volumed-pulse (xfce4-volumed with pulseaudio support): https://launchpad.net/xfce4-volumed-pulse which does exactly what you described.
The plugin was designed to be a panel plugin with gui controls. Optional keyboard shortcuts and notifications were added for convenience only, so that users do not have to run xfce4-volumed in addition to the plugin.
Since good alternatives exist and I do not plan rewriting the plugin I am closing this bug report as "wontfix".