When in panel preferences there is a red dotted line around the panel. This line pushes the panels background image each time the border blinks. So instead of putting the border inside the panel maybe put it on top some how so it dont pushes things like background image inside the panel.
This has been fixed in git master: https://git.xfce.org/xfce/xfce4-panel/commit/?id=d8b2e63dc8160db5463cef743d4bf2330558a5fa