Two Critical Bugs in KMail 1.6 Detected

Two critical bugs crept into the just released version of KMail. One is related to POP filters and the other to spam filtering - both cause mail loss! They are fixed in CVS and patches are linked on the KDE 3.2 Info Page. The distributors have been asked to update their binary packages.

If you are using POP filters with KMail then, after upgrading to KDE 3.2, you have to reconfigure your POP filters because otherwise messages which are supposed to be downloaded later will be deleted from the POP server. Also due to this bug you must not use KMail 1.6 (from KDE 3.2) and an older version of
KMail in parallel because both versions will behave differently (one version might download your messages from the server while the other version might delete your messages from the server).

If you filter email in an IMAP folder trough spamc the message may lose its UID and consequently a delete job is sent to the kio_imap slave without a UID. The slave unfortunately interprets that as "expunge folder".

Dot Categories: 

Comments

by Rob Kaper (not verified)

Distributors and users might also want to take a look at

http://lists.kde.org/?l=kde-cvs&m=107562661503609&w=2

which fixes possible data loss in Kopete, not sure whether that was packaged or not.

by Rob Kaper (not verified)

Oh, possible as in occasional/conditional, not as in unverified.

by Anonymous (not verified)

You overvalue instant messaging content, you may miss some lines but I wouldn't call it loss because it's not intended to be a permanent storage.

by Rob Kaper (not verified)

1) If it's sent but not received, it's lost.

2) You really shouldn't try to decide what kind of communication others might find important or intent to store.

3) You should familiarize yourself with the Sarbanes-Oxley Act of 2002, which requires institutions to archive electronic communications.

by Rob Kaper (not verified)

Intend even. Stupid Dutch with its "t"s at the end of verbs got in the way.

by Anonymous (not verified)

Concerning 3), if there is no communication on receiver side either due to network or software problems then you simply have nothing to archive. You cannot be obliged to archive emails/whatever you never received.

by Rob Kaper (not verified)

If you make a decision to use third party software, that explicitly comes with a "comes with no liability, use at your own risk" clause in its license, to connect to a proprietary protocol, I'm not convinced you can't be held liable for malfunctions caused by not using the officially endorsed software.

by Anonymous (not verified)

What software comes with full liability?

by frankiboy (not verified)

You might be right today.. IM is a very efficient and good way of communication, but it will never be used in a formal (work related) way (like mail is today) because of the problem tracking discussions.

When IM solves this problem and interoperability between the different protocols is present, IM use will gain popularity in business related matters..

by Evan "JabberWok... (not verified)

Heh. If you mean email when you say "mail", then you are obviously quite young. Email had a long and slow road to be accepted. Same for faxes and voicemail. It takes awhile for tech to be integrated into business, but when it is, you often think that it was always done that way.

I don't doubt that there are still, even in 2004, plenty of offices where email is not considered acceptable for business practices in some executive circles (I also would imagine that the support staff of those executives use email quite a bit). At the same time, I do quite a bit of work with a QA group some 3000 miles away. We work through ICQ and log everything for reference. The ICQ log is considered the "official word". We'll often have a session open during phone conversations and fire summaries and formal wording of what we are discussing back and forth.

--
Evan

by Aaron Traas (not verified)
    IM is a very efficient and good way of communication, but it will never be used in a formal (work related) way (like mail is today) because of the problem tracking discussions.

Really? At my job, we're required to use AIM. It has to be running when we're at our desks, and moreso when we work from home. We use it more than phone communication because it's easier.

by teatime (not verified)

Last time I used the official MSN client (granted, a few years ago) it happily dropped messages left and right. It was a common occurrance and I wasn't the only one affected.

I guess I haven't trusted the protocol since, even if in this case, the error was traced to the Kopete code.

by Ingo Klöcker (not verified)

Some more information about the second bug (http://bugs.kde.org/show_bug.cgi?id=74017):

Since filters are not applied automatically on IMAP messages only manual application of filters (via Ctrl-J or Message->Apply Filters) can trigger this bug. Currenly the bug has only been reported in connection with that usage of spamc, but it's possible that other "pipe through" filters trigger the bug as well.

In order to prevent message loss you should disable "Apply this filter" "on manual filtering" for all "pipe through" filters. Alternatively you can of course simply not apply filters manually on IMAP messages.

Hm, that's less fortunate. Is this bug only present in KMail 1.6 / KDE 3.2 or also in older KDE / KMail versions?

by Ingo Klöcker (not verified)

Both bugs are not present in KMail 1.5.4 or older (i. e. KDE 3.1.5 or older). They have been introduced during development for KDE 3.2.

by Evan "JabberWok... (not verified)

Hmmm. So if I use IMAP and never use POP, and I don't run anything through a pipe filter, I'm safe for the time being (until I next upgrade)?

--
Evan

by Ingo Klöcker (not verified)

Yes, in this case the POP filter bug won't affect you and the IMAP filter bug shouldn't affect you. To be 100% sure you shouldn't apply any filters on IMAP messages.

Instead of waiting for the release of updated binary packages you can of course build the latest stable version of KMail from cvs. See http://kmail.ingo-kloecker.de for a short description how to do this. You should simply install KMail into the same directory that your distribution uses.

Thanks for that writeup - nice to just copy-n-paste the solution.
One typo - there should be a newline after "ln -s ../kde-common/admin" so the line should be:

ln -s ../kde-common/admin

cvs up -d -r KDE_3_2_BRANCH mimelib libkdenetwork kmail libkdepim ktnef libksieve libical libkcal

by Ingo Klöcker (not verified)

Fixed.

I don't remember exactly how it happened, but yesterday Kmail almost deleted all my email from an imap account without using any spam software. I was trying to make it see the inbox of the imap account (the server makes something strange there). I changed the root of all the email at the server and then go back. I realized it was deleting my email and stop it.

by Ingo Klöcker (not verified)

If you know how to reproduce the problem then please tell us (via bugs.kde.org).

by Nathan (not verified)

I just upgraded to KDE 3.2 including an upgrade to Kmail 1.6 and everything works great except when I start Kmail all of the sudden Xfree86 uses 50% of the CPU constantly and the kdeinit that launched kmail uses 25% of the CPU constantly. Even closing Kmail doesn't fix the problem I have to manually kill the kdeinit process. Any ideas?

by Ingo Klöcker (not verified)

Please find out what kdeinit process this is, e.g. by running 'ps xuww' in Konsole.

by Victor Orlikowski (not verified)

This seems to have to do w/ fonts (perhaps anti-aliased).

I've noticed it when running the Control Center app; shortly after I open a dropbox in the fonts dialog, the X server CPU jumps to 50-60%. What's it doing? According to top, it looks like it's sitting in select().

Also, my anti-aliased fixed width fonts aren't showing up in the fixed width font choice dialogs anymore - this is a change from 3.1.5; the fonts *are* showing up in the "all fonts" dialogs, so it does seem that they are available - just not recognized as fixed width in 3.2.

by Matt Howard (not verified)

I have the same problem... I was about to roll back to 3.1.5 until I read your note. Turning off AA fixed the issue with CPU time... I'm running a 2.4 with ck patches, XF86 4.3.0 w/ radeon driver (Slackware 9.1). kded and kwin would run about 20% user time each, and the system would run choppy as hell. After turning AA off, everything is fine. I also see a huge cpu-eater effect when I lock the screen...

Wiktor,
My comment is:
Skontaktuj sie ze mna. Ta cisza, nie jest niczym dobrym. Dobro jest czyms wiekszym niz gniew, nienawisc. Te sa napewno niszczace w kazdym sensie.
Kocham cie i prosze o kontakt which you deny me.
Tata

do you have any big searches defined?

by Stephen Boulet (not verified)

For all you Gentoo users, here's the changelog for kdepim-3.2.0-r1 (04 Feb 2004):

" 04 Feb 2004; Paul de Vrieze kdepim-3.2.0-r1.ebuild,
files/KMail-inboxEater-BRANCH.diff:
A new version that applies the kmail patch from the packagers list. We don't
want users to loose messages."

[Breathes a sigh of relief.]

Stephen

by Rex Dieter (not verified)

FYI, the eariler released KMail-inboxeater patch only fixes the pop filter issue.

-- Rex

by daniel (not verified)

Correct me if I'm reading things wrong, but the bug report linked above says the bug was reported on 2004/01/05, using KDE Beta 2, and it took one month to confirm it. I'm very impressed with the work of all KDE developers and I'm using KDE and KMail, but a (reported) bug deleting mail directly on the server seems a bit rough for a major release...

by Anonymous (not verified)

Another obversation is that there was no second user comment or votes from more than one user in this time frame. If this shows anything then that there are too few people (not necessarily developers!) dealing with/separating the flood of useful/useless/duplicate/invalid bug reports. So when do you join the http://kde.ground.cz/tiki-index.php?page=Bug+Triage ?

by Felix (not verified)

If I take myself as an example I don't wonder why I haven't found it:

a) I don't use pop filter, I don't delete mails on pop servers as it wouldn't show up in download time

b) I don't use IMAP since kmail doesn't support all features and it is just unuseable with an exchange (5) server.

For me it looks like that two features aren't used by many beta testers

by Ingo Klöcker (not verified)

ad b) It's interesting that you think it's unuseable with an Exchange server (BTW, what does the (5) refer to? Is this the version number of the Exchange server?) because I know for sure that many people are using KMail since years with Exchange servers. Maybe you should report the problems that you ran into. Otherwise those problems will probably never be fixed. If you already reported some problems then you should probably bring them to our attention again.

by Felix (not verified)

Hi

Yes, it is an exchange 5.5 Server. Imap is working fine, the problem is that if you have a big company-public resource on the server, kmail is always loading/refreshing whatever if you browse throught the folders.
Outlook just works, you don't see it loading mails it just shows what is there.
Also many special "mails" are not shown correctly, like notes.

Maybe that changes with an exchange 2000+ server, but if I upgrade, than not to exchange XXX ;)

Of course it might also be possible that I should try it again ;)

by Ingo Klöcker (not verified)

At the time the bug was reported I didn't have the time to look at it. (I didn't even have the time to confirm it.) Unfortunately I forgot to raise the severity of this bug and so I forgot about it. It became just another of 300+ open bugs. Since no other user ever confirmed or commented on this bug it was never brought to my attention again until I read in a user comment on http://www.heise.de/newsticker/ about this bug. After that it took me just a few minutes to confirm, analyze and fix the bug.

So what's to learn from this? That we need people who help with managing the bug reports. We need people who try to confirm them, who ask the reporter for more information, who add useful information to the bug reports which help the developers to quickly track down and fix the bugs, and who make sure that serious bugs are not overlooked by the developers. Those people don't have to know anything about programming. So everybody can get involved.

If you are interested in helping out then please contact the KDE Quality Team (http://kde.ground.cz/tiki-index.php?page=KDE+Quality+Team).

by Alex (not verified)

I hope 3.2.1 is released soon...

by Anonymous (not verified)

Why should it? Likely a 3.2.0a kdepim package will be released.

by Matthias Kilian (not verified)

Well, may be it's fixed somewhere in CVS, but it is *not* fixed in CVS tagged KDE_3_2_0_RELEASE, and there's no tag KDE_3_2_0_RELEASE_HOT_FIX or so.

IMHO the CVS tags are on fire. RELEASE-Tags had been moved in the past, the tar balls on the ftp server and mirrors are generally *not* corresponding to the revisions tagged as KDE_x_y_z_RELEASE in CVS, and branch and ordinary revision tags are mixed happily within several CVS modules (which is typically a tag's death).

But we should think positive, so I'd like to discuss CVS tagging and release issues with those who are "responsible" for the releases (hey, there must be someone who creates the ..._RELEASE tag and who packages the tar balls and puts them on the ftp server).

But, *who* are this releasing people? I'd be glad if anyone could tell me.

I tried the policy mailingliste two times. The first time my subscription was silently ignored, the second time the list owner suggested to use the kde-devel list. But developers typically don't want to discuss tagging and packaging releases, they want to hack and to fix bugs.

The latter is incredibly difficult if a developer has to ask the bug reporter which version contains the bug -- e.g. 3.1.4 from ftp, 3.1.4 from CVS (october 2003), 3.1.4 from CVS (december 2003), or perhaps 3.1.4 from ftp with a CVS update on november 11, 2003 -- all those versions are called "KDE-3.1.4" but are different.

Same for 3.2.0: there's a hot fix for the FTP version, there seems to be a hot fix in CVS, but not properly tagged. Using KDE_3_2_BRANCH isn't an option, since it's a branch and thus permanently floating.

Regards,
Kili, sligtly worried ;-/

ps: it may look like KDE bashing, but it isn't; I've experienced the problem of non-reproducable releases at work. To help KDE to become even better, I'd like to help introducing a reliable, stable release policy.

by Aaron J. Seigo (not verified)

> But, *who* are this releasing people? I'd be glad if anyone could tell me.

it changes every few releases, but right now the Release Dude is Stephan Kulow. previous Release Dudes included David Faure, Waldo Bastian and Dirk Mueller.

> But developers typically don't want to discuss tagging and packaging
> releases, they want to hack and to fix bugs.

you may notice from the above list that developers are the people who do the releases =)

> 3.1.4 from ftp, 3.1.4 from CVS (october 2003), 3.1.4 from CVS (december
> 2003), or perhaps 3.1.4 from ftp with a CVS update on november 11, 2003

the version number is (almost?) always changed when such things happen post-release. once the release is made official, the tags stop moving. 3.1.4 is 3.1.4 is 3.1.4. what does change are the binary packages, as distros do patches their packages in various ways. so more important than "when" is "what distro, and what's the package version"

> Using KDE_3_2_BRANCH isn't an option, since it's a branch and thus
> permanently floating.

using the branch is the right option since it includes ONLY bug fixes (and updated translations). those patches are well reviewed before/while going into the branch.

by Matthias Kilian (not verified)

> it changes every few releases, but right now the Release Dude is
> Stephan Kulow.[...]

Thanks. I'll try to contact Stephan.

On the remaining posting: I think I'll have to check the CVS
meta directories in my checked out versions first.

Regarding to the other posting here (from Anonymous) it seems
that there's something wrong with *my* working directories :-(

However, I still think that release tags (those with `_RELEASE'
suffix) should strictly correspond to what's available at the
ftp servers.

Anyways, the current KDE really rocks :-)

by Anonymous (not verified)

> the tar balls on the ftp server and mirrors are generally *not* corresponding to the revisions tagged as KDE_x_y_z_RELEASE in CVS

This is not true. Even if you can tell an example, please do, it's an exception and not the general rule.

> branch and ordinary revision tags are mixed happily within several CVS module

Pardon? Please clarify.

by Matthias Kilian (not verified)

>> the tar balls on the ftp server and mirrors are generally *not*
>> corresponding to the revisions tagged as KDE_x_y_z_RELEASE in CVS

> This is not true. Even if you can tell an example, please do, it's an
> exception and not the general rule.

Check out kdebase from CVS (KDE_3_2_0_RELEASE) and compare it with the
contents of kdebase-3.2.0.tar.bz2.

Well, for 3.2.0, only generated files (configure, Makefile.in, subdirs,
etc.) are affected, with a small exception in quanta, but there were
significant differences in some of the 3.1 releases.

>> branch and ordinary revision tags are mixed happily within several
>> CVS module

> Pardon? Please clarify.

Affected: kdeaddons, kdeadmin, kdeartwork, kdebase, kdebindings, kdeedu,
kdelibs, kdemultimedia, kdenetwork, kdepim, kdesdk, kdeutils.

Those CVS modules use the tag KDE_3_2_0_RELEASE as a normal tag for most
of the files, but for some files, KDE_3_2_0_RELEASE is a branch tag.

To verify this, you've to look at the CVS/Tag files, for example:

$ cd kdebase
$ cvs up -dP -r KDE_3_2_0_RELEASE
$ find . -path \*/CVS/Tag | xargs cat | sort -u
NKDE_3_2_0_RELEASE
TKDE_3_2_0_RELEASE

The tag in each Tag file is prefixed with `N' for ordinary tags and
with `B' for branches. Using the same tag for ordinary as well as branch
tags makes the tag completely useseless -- at least that's my experience.

Ciao,
Kili

by Anonymous (not verified)

> To verify this, you've to look at the CVS/Tag files, for example:

I cannot confirm this, your example outputs for my kdebase checkout only "NKDE_3_2_0_RELEASE".

> Using the same tag for ordinary as well as branch tags makes the tag completely useseless

It's not used for branching. The branch tag is called KDE_3_2_BRANCH.

by Matthias Kilian (not verified)

> I cannot confirm this, your example outputs for my kdebase checkout
> only "NKDE_3_2_0_RELEASE".

O.k., I'll try to take the tar balls from the ftp server and do a
cvs up -dP -r KDE_3_2_0_RELEASE on it. May be my working directories
are corrupted.

by Nicolas Goutte (not verified)

You known about the -D parameter of cvs update, don't you?

You can specify something like:
cvs update -r KDE_3_2_BRANCH -D 2004-02-06
and you will always get the same thing.

Have a nice day!

by Andre (not verified)

Didn't the same happen with KDE 3.1? I remeber a Kmail securtiy fix.

by Anonymous (not verified)

This is about safety, not security. And there are thousands ways to make a fault.

by TDH (not verified)

While this incident is unfortunate, kudos to Ingo and others for making sure these bugs and fixes are widely known and available quickly. I guess love (open-source) beats money (commercial) when it comes to "customer" support...

by Steven Vollom (not verified)

I want to use Kmail, however, it refuses the configuration that works for my server. I use a pop server. 1st, my email address is as above, [email protected], 2nd, my incoming server address is pop.att.yahoo.com. After setting the configuration, incoming emails are rejected and in the warning it shows my server address as pop.sbcglobal.yahoo.com. Note that Kmail changed my configuration settings, and I might add, the only ones that will work for me from att to sbcglobal. Next, my outgoing mail address is smtp.att.yahoo.com, however when I have appropriately entered the server address, then attempt to send an email, the warning for failure to deliver has changed to smtp setting to smtp.att.yahoo.net. Note that the only address is .com and your program has changed it to .net. Is there a way to force your program to accept my correct settings? I now remember why I removed Kmail back when I first installed and set configuration. I replaced it with Thunderbird, which works, with one small problem, but at least it works. Kmail doesn't work and continues to attempt to send and receive to addresses that don't exist. I am constantly having to try to get it to stop, because sending and receiving to an incorrect address will never work, and the only solution to get away from the annoying notices is to remove Kmail, which is what I am going to do right now. I would still like to use Kmail, a Linux product to support Linux, but not until you fix my problem.
Thanks! I hope my email is helpful to you.