In Xubuntu 14.04 (xfce4 4.10), the panel's response to the background varies wildly. At work I have a dual quad-Xeon machine with two screens and the nVidia driver, and a dark background. On that machine, the panel and all its elements are set to colors that work well with a dark background. See https://drive.google.com/file/d/0B-3x0QT9-7fIUlhiNVRrVWFVcDQ/edit?usp=sharing However, on my home desktop, which is a Phenom II x6 machine with the same OS and the latest nVidia driver, the panel refuses to adjust and insists on black lettering with glaring workspace switcher and window list and impossible to read icons in the notification tray. See https://drive.google.com/file/d/0B-3x0QT9-7fIVWdQX2hCRmxPT1E/edit?usp=sharing for example. Even if I change the background to the same color as at work (blue 20 instead of green 20), it makes no difference. See https://drive.google.com/file/d/0B-3x0QT9-7fIVGNCRGZhNGpucVE/edit?usp=sharing. The only way to make the panel show what I need so I can see it is to color it with a very light color, and that not only clashes with the background, it strains my eyes. Please either fix this or add a control so we can tell it to invert colors for a dark background. Thanks.
I forgot that this seems to be related to bug 10340. I will add my last notes there, too.
As I reported on Bug 10340, this bug is related to clock-plugin, not the Xfce panel itself. Regarding your problem, probably the difference is the chosen theme. The plugin just follows the colors defined by themes. I don't know exactly where you can verify this, but some googling might give you the answer. I have created a patch for datetime-plugin(Bug 9487) which allows the user to define the label colors, just like the orage-plugin does. If anyone is interested to do the same with clock-plugin, it'll be a trivial hack. However, I don't know how or if it's possible to accomplish what MR ZenWiz suggests for inverted colors.
My apologies - I thought I had updated this a month or two back. Yes, the theme was the surface problem. I changed from Clearlooks to Bluebird and that solved the problem. Thanks.