When checking "Automatically show and hide the panel" in the panel preferences nothing changes (panel does not "hide").
A reboot seemed to fix this behavior. Not sure why.