Trash folder mess on Imap account after 60.3 upgrade

[bsh]

After updating from 59.x to 60.3, trash folder does not work and is a complete mess, on two separate Imap server accounts.
A new folder is created outside the personal namespace instead, and I can not delete this. Sometimes it seemingly disappears, but then reappears after a restart, sometimes it gets "copied" into the Trash folder under the personal namespace, but can not be deleted from there. As you see, it's a complete mess now: https://i.imgur.com/l2RFtx3.jpg

[DanRaisch]

Moving to Thunderbird Support as no bug has yet been identified here.

[jorgk]

We made various changes to TB's IMAP trash handling in
https://bugzilla.mozilla.org/show_bug.cgi?id=1335982
https://bugzilla.mozilla.org/show_bug.cgi?id=1428666
https://bugzilla.mozilla.org/show_bug.cgi?id=1427507
https://bugzilla.mozilla.org/show_bug.cgi?id=1427940
all hopefully for the better

To resolve the issue, I suggest you do the following:

First make sure the Trash folder is properly selected. That needs to be done in the Account Settings under Server Settings.

As far as I can see, you only have one Trash folder called "Törölt elemek". Make sure it is selected. You should empty it, to get rid of the one inside. Then check that you're no longer subscribed to the deleted subfolder. Then restart.

The next issue seems to be a hierarchy of folders inside a DELETED folder. This doesn't appear to be recognised by TB as trash folder. So the easiest way to get rid of them is to unsubscribe from that hierarchy and then close TB. Then delete all these folders via webmail.

Also important to not return to TB 52 since that will destroy the Trash selection and bring some phantom folders back.

[bsh]

We made various changes to TB's IMAP trash handling in
https://bugzilla.mozilla.org/show_bug.cgi?id=1335982
https://bugzilla.mozilla.org/show_bug.cgi?id=1428666
https://bugzilla.mozilla.org/show_bug.cgi?id=1427507
https://bugzilla.mozilla.org/show_bug.cgi?id=1427940
all hopefully for the better

That's a lot to read through. Just looking at the first and second one really quickly, they are "fixed". Well, definitely not here. On first run after the update I got a new trash folder outside the namespace, and that is what can not be deleted ever since.
I've manually deleted it from the local store of mail folders and purged from the imap as well, but it comes back under the "5CE5...random" name.
I have even completely removed the mailbox from the local store and deleted everything and recreated the mailbox from thunderbird, and it is still the same.

To resolve the issue, I suggest you do the following:

First make sure the Trash folder is properly selected. That needs to be done in the Account Settings under Server Settings.

As far as I can see, you only have one Trash folder called "Törölt elemek". Make sure it is selected. You should empty it, to get rid of the one inside. Then check that you're no longer subscribed to the deleted subfolder. Then restart.

Tried that now. Can't empty the "törölt elemek" (=trash) folder. Either i get a permission denied, or if i go into its contents and shift+delete them, they do disappear but come back upon new mail query.

The next issue seems to be a hierarchy of folders inside a DELETED folder. This doesn't appear to be recognised by TB as trash folder. So the easiest way to get rid of them is to unsubscribe from that hierarchy and then close TB. Then delete all these folders via webmail.

Also important to not return to TB 52 since that will destroy the Trash selection and bring some phantom folders back.

I've got no webmail but i'm the admin, I had a look at that. the DELETED folder and the random stuff is definitely there on the server, outside the namespace, and it's been created by TB with the new version. So much for it being better

Do I need to make changes to cyrus to make it "new thunderbird compatible" now? It's been fine since years.

[jorgk]

Best to file a new bug and we'll look into it. We have a volunteer doing IMAP stuff and he's familiar with all sorts of servers. You know there are advanced settings in TB related to namespace and such. Maybe they need tweaking.

[jorgk]

Oh, re. "Either i get a permission denied, or if i go into its contents and shift+delete them, they do disappear but come back upon new mail query."

How can the be "permission denied"? Also, if they come back, you haven't unsubscribed, have you?

[bsh]

Oh, re. "Either i get a permission denied, or if i go into its contents and shift+delete them, they do disappear but come back upon new mail query."

How can the be "permission denied"? Also, if they come back, you haven't unsubscribed, have you?

it comes in a notification from the taskbar area.
And I did unsubscribe, even from the entire trash folder. It has some greyed out stuff inside it, and this random named folder as well (it is black).
Advanced namespace settings are correct and haven't been changed since years and they were wokring before.
As for filing a bugreport, I have found this exact same bug (apparently) already filed and marked as fixed. Apparently it isn't.

on the server it looks like this now:
users/me/
users/me/Sent
users/me/Trash (but with a different icon)
users/me/Trash/5BE5DB2E (with privileges: anyone lrs) - this been done by TB. I have no privileges.
I'll delete it on the server side and rebuild, then recreate the trash folder with proper privileges and report back. I bet it will come back with a new random name...
Do i need to clean up the locally stored folders in the TB profile by hand?

[jorgk]

Well, if the bug wasn't 100% fixed, we need to file a new one. Looks like there's still a problem if folders are created outside the namespace. Perhaps the fix doesn't work for your type of server or made it worse. As I said, I can only get out IMAP person to look on Bugzilla, not here.

If the TB shows a "permission denied" errors, then something appears wrong in the setup. At least it couldn't delete what it wanted to delete.

Maybe best to remove the locally stored folders by hand, yes.

[bsh]

Hey, thanks for the help so far.
So the DELETED\ stuff is new, but as I see now, it's only partially TB's fault, i think this must be a few new or changed configuration option(s), because it didn't happen with earlier versions. Apparently TB now tags them as deleted and the server put's them in DELETED\, but doesn't prune them.
The other thing is, this DELETED shoudln't be visible because it's not subscribed to, but now under the special settings, this is now deactivated and it now show all folders even unsubscribed ones. That's gonna be fun to change for every user once they upgrade TB...
Anyhow, i manually deleted all that crap and rebuilt all databases and directories and delprune'd everything for now, recreated the trash folder and made sure the permissions were fine, and now all the DELETED\ stuff and the children of the Trash are gone for good and all is fine.
Except for one new thing: i'v put a message into inbox, i can move it around fine, as before. but when I delete it, it disappears for a split second, then reappears, and an error notification appears, saying, that the last operation on inbox failed, with "unkown code_______255". The message is copied into the trash and I can delete it from there normally, but the original message doesn't get removed from inbox (or from inbox\sent)
Any idea what it could be?

[jorgk]

Hi, I'm not sure whose configuration option(s) you're referring to. TB still has three delete schemes: Move to a trash folder, mark as deleted and delete immediately. That hasn't changed. But I'm glad it's working again.

That deleted messages spring back to life for a split-second is a known issue:
https://bugzilla.mozilla.org/show_bug.cgi?id=1293091
https://bugzilla.mozilla.org/show_bug.cgi?id=1495279
I've seen this on various occasions but no harm was done.

That you get an error and the message isn't deleted shouldn't happen. In cases like this you should run an IMAP log (https://wiki.mozilla.org/MailNews:Logging) and check what's happening.

All this is better discussed on Mozilla's Bugzilla since there we can get Gene's support, who is our IMAP man. He knows the quirks of many different IMAP servers. I'll e-mail Gene again, perhaps he will get an account here to answer (although I remember having trouble in opening one).

[gene smith]

I do have an account here. But, I agree, if there are still specific problems a bugzilla would be better to use.

[gene smith]

If there is still a problem and you submit a bug report, probably need to know the good folder structure before upgrade and the bad structure after upgrade to 60.*.

Also, need to know the namespace settings under Advanced Account Settings and are shared or "other' in use. Also, helpful to know the hierarchy delimiter (dot or slash).

Finally an imap.log attachment showing what happens, ideally, on 1st startup after upgrade would be helpful but probably not possible. So just the log with some activity with already updated 60 will be OK.

[bsh]

Well, i have good news and bad news.
The good news is, i seem to have fixed the mess on my other server, by a HELL lot of manual work and hair pulling and crying and swearing. Now everything seems to be working just as fine as before. Now the trash folder is qwhere it should be, with the proper acl's, it works, i can delete, it doesn't come back, no error popups (no errors on the server log either), no moe greyed out stuff, DELETED\ folder is not seen anymore, expunge works fine, I think i'm good now. I may have to do it again on the other server.
The bad news is, by doing this, i completely eradicated all the evidence and any clues to help you debug this. But I do think, based on what i've found, that there's nothing wrong with the server side, and the mess was indeed caused by the initial startup of thunderbid right after the update, and then me repeatedly trying to delete/fix the bogus stuff that's been created. By the looks of it, it created the Trash folder as a shared folder, probably because it was not using the correct naming conventions (which it is recognizing now correctly) and then not applying acl's on it so when i tried to delete it, it copied it into my trash folder but then could not flag/delete the original, maybe? Dunno.
btw this is on ubuntu 18.04 (upgraded from 16.04), cyrus 2.5.10, and apart from configuring ssl and tls and auth, and having some extra mail partitions, the only non-default config i thik is it is using unixhierarchseparator "\", but not altnamespace. Probably this is what TB interpreted wrong?

[jorgk]

Thanks for the update, glad it's all working again - Maybe Gene can run some tests with the details from the last paragraph.

[gene smith]

Not sure I understand at all what caused this or what the reporter did to fix it. Was it on the server or something tweaked in TB?.

If the user can provide clearer details I will be happy to look closer at this. Things like what the correct folder structure is vs. what he saw with tb 60 update. (Also, folder name translation to En would help since I am pretty much mono-lingual.) Also, info like tb namespace setting would be helpful since he mentions Trash getting set as shared. He also mention separator "\" (backslash) and "altnamespace". Surely he mean forward slash "/" and no idea what is meant by altnamespace (maybe the shared or public namespace)?