! Please note that this is a snapshot of our old Bugzilla server, which is read only since May 29, 2020. Please go to gitlab.xfce.org for our new server !
Mbox parsing detects rogue From headers in mail content.
Local mail spool


Description Kjetil Erga 2008-04-12 13:22:45 CEST
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv: Gecko/20071224 Firefox/
Build Identifier: 

While parsing a mbox file, the mailwatcher plugin does not handle From "headers" that are inside the mail content itself. What will happen is that this will be counted as an extra mail, and because there is no following "Status: " header, the plugin thinks this is a new and unread mail. This can be annoying since the "You have new mail" status will be active all the time.

I have managed to fix this problem with a patch that I will attach. The patch will make the plugin look for an additional "Return-Path: " header that has to appear right after the "From " header, or else it will not be counted as a mail. I am not sure however, if this is the best solution to the problem, or works for everyone. I do not know if a "Return-Path: " header should _always_ come after the initial "From " header.

Reproducible: Always

Steps to Reproduce:
1.Receive mail with a string containing "From " inside the content.
2.Open the mailbox and "read" the mail.
3.Observe that mailwatcher still thinks there are "new" mails

Actual Results:  
Icon shows that new mail has arrived.

Expected Results:  
Icon should show that no new mail has in fact arrived.
Comment 1 Kjetil Erga 2008-04-12 13:23:58 CEST
Created attachment 1585 
Patch to modify parsing behaviour.
Comment 2 Brian J. Tarricone (not reading bugmail) 2008-08-30 09:12:32 CEST
Comment on attachment 1585 
Patch to modify parsing behaviour.

No, you can't be sure that *any* particular header will be right after the 'From ' line.

What mail client are you using that's generating/editing these mbox files?  The mbox spec says that any line that starts with a 'From ' token needs to be escaped by anyone writing the file.  Sounds like your mail program is broken.
Comment 3 Kjetil Erga 2008-09-13 19:17:32 CEST
I'm using getmail and mutt. I guess that getmail just writes directly to the mbox file without escaping anything, as you say. We can shift the blame, and you may close this issue if you want.
Comment 4 Brian J. Tarricone (not reading bugmail) 2008-09-13 19:35:54 CEST
Actually, I just looked at the mbox RFC (http://tools.ietf.org/rfc/rfc4155.txt).  It does state that there's really no formal standard, and different implementations work in different ways, but it does point out that many implementations don't escape 'From' lines inside message bodies, and parsers should be able to handle that.

I didn't write the mbox parser, Pasi did, and he's not really around anymore, so I'm not sure if I'll ever get to this, but I guess it would be worthwhile to try to work around it.
Comment 5 Greg Veldman 2011-10-27 18:54:02 CEST
Created attachment 3920 
mailcheck regex patch

Change the new message logic to more precisely match the "From " line using regular expressions.
Comment 6 Greg Veldman 2011-10-27 18:55:56 CEST
So, I meant to add some additional info to my previous comment with the patch, but it seems to have been eaten by bugzilla.  Here it is:

I've run into this bug also, and it's quite annoying.  I wrote a patch to fix it.  Instead of relying on the presense of any particular header after the "From " line, I changed the logic to see if we're in a new message to use a regular expression instead of just a simple strncmp.

I've tested this patch on a few different machines running different client software (Linux, Solaris, mh, mutt, Postfix, Sendmail) and it seems to work with no problems.  The regex is designed to match (as closely as possible) the format of the "From " line as defined in RFC 4155.  It seems unlikely that you would find this exact string in any normal mail message.
Comment 7 Greg Veldman 2011-10-27 20:08:53 CEST
Created attachment 3922 
mbox regex patch (refined)

Refine the regex a bit to further avoid false positives.
Comment 8 Landry Breuil editbugs 2014-11-23 21:42:39 CET
Mass-reassign all bugs from florian@ to goodies-dev@, thanks for the maintenance work! (and sorry for the bugmail spam..)
Comment 9 Git Bot editbugs 2020-05-24 00:39:05 CEST
-- GitLab Migration Automatic Message --

This bug has been migrated to xfce.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.xfce.org/panel-plugins/xfce4-mailwatch-plugin/-/issues/3.

Please create an account or use an existing account on one of our supported OAuth providers. 

If you want to fork to submit patches and merge requests please continue reading here: https://docs.xfce.org/contribute/dev/git/start#gitlab_forks_and_merge_requests

Also feel free to reach out to us on the mailing list https://mail.xfce.org/mailman/listinfo/xfce4-dev

Bug #4009

Reported by:
Kjetil Erga
Reported on: 2008-04-12
Last modified on: 2020-05-24


Xfce-Goodies Maintainers
CC List:
1 user


1.1.0 or older


Patch to modify parsing behaviour. (1.01 KB, patch)
2008-04-12 13:23 CEST , Kjetil Erga
no flags
mailcheck regex patch (1.41 KB, patch)
2011-10-27 18:54 CEST , Greg Veldman
no flags
mbox regex patch (refined) (1.43 KB, patch)
2011-10-27 20:08 CEST , Greg Veldman
no flags

Additional information