Searching for word not finding an email (Linux)

Discussion of general topics about Mozilla Thunderbird
Post Reply
chpnp
Posts: 16
Joined: May 4th, 2008, 3:44 pm

Searching for word not finding an email (Linux)

Post by chpnp »

Hi

I am doing a search on words and can see that a specific email is not found, eventhough it does contains those words
(I could track down 3 words failing to find for that specific email)

It very much looks like this specific email is not indexed.
I tried to rebuild the global message sqlite database, it did not help

This is imap 4

Any hint ?
Last edited by DanRaisch on July 3rd, 2020, 1:59 pm, edited 1 time in total.
Reason: (Linux) added to subject line.
User avatar
DanRaisch
Moderator
Posts: 127187
Joined: September 23rd, 2004, 8:57 pm
Location: Somewhere on the right coast

Re: Searching for word not finding an email (Linux)

Post by DanRaisch »

How are you initiating the search? Ctrl+K or Ctrl+Shift+K or something else?
What version of Thunderbird? Is it from the Linux Distro or from Mozilla?
chpnp
Posts: 16
Joined: May 4th, 2008, 3:44 pm

Re: Searching for word not finding an email (Linux)

Post by chpnp »

Hi,

This is from the Ubuntu distribution 68.8.0 (64 bits)
I click in the search window and enter the word I am looking for.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Searching for word not finding an email (Linux)

Post by tanstaafl »

I'd start by verifying you can find that message using the quick filter bar (limited to searching the current folder). If that succeeds you can ignore the possibility there is something wrong with the message in the remote folder etc. and should enable logging for gloda etc.

http://kb.mozillazine.org/Search_-_Thunderbird
http://kb.mozillazine.org/Session_logging_for_mail/news
https://wiki.mozilla.org/Thunderbird:Using_Gloda

If you were using version 60, you could use the add-on at https://addons.thunderbird.net/en-US/th ... en/?src=ss to add three columns to the Thunderbird message pane that are useful in debugging. Unfortunately it doesn't look like the author will rewrite it to be a WebExtension based add-on, which is needed to use it with version 68 and later.
User avatar
WaltS48
Posts: 5141
Joined: May 7th, 2010, 9:38 am
Location: Pennsylvania, USA

Re: Searching for word not finding an email (Linux)

Post by WaltS48 »

chpnp wrote:Hi,

This is from the Ubuntu distribution 68.8.0 (64 bits)
I click in the search window and enter the word I am looking for.
You still did not identify which search you are using.

If I select an email and use Ctrl+F I can search just that email for a word.
If I use Ctrl+K I can search all emails for a word.
If I use Quick Search (Ctrl+Shift+K) I can search just the Body of emails for a word. The user needs to have a folder selected in an account to use that method. It does not search all emails for all accounts.

Ubuntu version 68.8.0 on Ubuntu 18.04.4.
Linux Desktop - AMD Athlon(tm) II X3 455 3.3GHz | 8.0GB RAM | GeForce GT 630
Windows Notebook - AMD A8 7410 2.2GHz | 6.0GB RAM | AMD Radeon R5
chpnp
Posts: 16
Joined: May 4th, 2008, 3:44 pm

Re: Searching for word not finding an email (Linux)

Post by chpnp »

Apologies for the delay, i have been locked out of mozzillaZine

This is a search through the ctrl+k window, ie a search for a word in all exisitng emails
I have removed the sqlite index file and rebuild the index prior to testing.

Crl+shift+k -> ok
ctrlo+k -> ko, word find in some emails (but the email i was looking for does not show in list)

mailnews.database.global.indexer.enabled is set to true (but I have o idea how to exploit this, nor do I know where the debug file sits)

Any hint from then on ?

Of course in the meantime I found the email I was looking for; but I would like to track this down because it is very annoying that I am not sure Thunderbird can find my emails properly.
I am now feeling uncomfortable (as I have other email issues, probably tied to my provider, but this adds to the uncertainties).

Thanks !
Last edited by chpnp on July 28th, 2020, 5:53 am, edited 1 time in total.
User avatar
DanRaisch
Moderator
Posts: 127187
Joined: September 23rd, 2004, 8:57 pm
Location: Somewhere on the right coast

Re: Searching for word not finding an email (Linux)

Post by DanRaisch »

Close Thunderbird. Navigate to the profile folder ( http://kb.mozillazine.org/Profile_folder_-_Thunderbird
http://kb.mozillazine.org/Files_and_fol ... hunderbird ) and locate and delete the file global-messages-db.sqlite. Restart Thunderbird and allow several minutes for the program to rebuild that index file. Try the search again.
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Searching for word not finding an email (Linux)

Post by tanstaafl »

I repeat my suggestion: "I'd start by verifying you can find that message using the quick filter bar (limited to searching the current folder). If that succeeds you can ignore the possibility there is something wrong with the message in the remote folder etc. and should enable logging for gloda"
chpnp
Posts: 16
Joined: May 4th, 2008, 3:44 pm

Re: Searching for word not finding an email (Linux)

Post by chpnp »

DanRaisch wrote:Close Thunderbird. Navigate to the profile folder ( http://kb.mozillazine.org/Profile_folder_-_Thunderbird
http://kb.mozillazine.org/Files_and_fol ... hunderbird ) and locate and delete the file global-messages-db.sqlite. Restart Thunderbird and allow several minutes for the program to rebuild that index file. Try the search again.
I have, and already had.
No success.
Furthermore, I know can spot several email which cannot be found with different words


Really strange, it has been working in the past.
Could it be tied to a large amount of emails (unknown, several email addresses, way above 100 000 emails, maybe 2 or 3 times that).
the db size is bout 300 MB, I have dealt with larger sqlite database than that.
Last edited by chpnp on August 5th, 2020, 12:56 pm, edited 1 time in total.
chpnp
Posts: 16
Joined: May 4th, 2008, 3:44 pm

Re: Searching for word not finding an email (Linux)

Post by chpnp »

tanstaafl wrote:I repeat my suggestion: "I'd start by verifying you can find that message using the quick filter bar (limited to searching the current folder). If that succeeds you can ignore the possibility there is something wrong with the message in the remote folder etc. and should enable logging for gloda"
Apologies, good catch, I had checked that the email was found with ctrl+shift+k, and not with ctrl+k
(but forgot to mention).
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Searching for word not finding an email (Linux)

Post by tanstaafl »

What does gloda logging report?

I'm not aware of any limits on the amount of messages/folders that are specific to gloda/global search. I'm not aware of a limit on the maximum search index size (other than what would be imposed by using sqlite, which I'd expect to be the maximum file size supported by your operating system)

https://www.sqlite.org/limits.html
chpnp
Posts: 16
Joined: May 4th, 2008, 3:44 pm

Re: Searching for word not finding an email (Linux)

Post by chpnp »

tanstaafl wrote:What does gloda logging report?

I'm not aware of any limits on the amount of messages/folders that are specific to gloda/global search. I'm not aware of a limit on the maximum search index size (other than what would be imposed by using sqlite, which I'd expect to be the maximum file size supported by your operating system)

https://www.sqlite.org/limits.html

I have no idea how to check the logging (which is enabled per procedure provided above) -- Would you please instruct ??

An indexing process looks pretty straightforward to me - in particular when the index is rebuild from scratch, where it looks like reading a file, splitting in words, writing in some index table -- ok there is a bit more around this, but still). And I would think this process is very much debugged on TB, so I am really puzzled here, and barely believe what i am seeing - I wish someone would tell me I am wrong and just cant read, but it wont happen. I tried rebuilding several time, with the same consitent errors.
So debug is the way to go, if you can share some minimal instructions for me as of where to start with.

(as for sqlite, i hardly believe there is a limit here. I am a regular user of sqlite and have used databases of a few GB - and have read about databases in 2013 in the +100GB -- nothing close to my miserable 350 MB index).

Thanks
chpnp
Posts: 16
Joined: May 4th, 2008, 3:44 pm

Re: Searching for word not finding an email (Linux)

Post by chpnp »

hi --

i have made some progresses. - i understand how to view the debug log

At the same time, I can now spot several emails (all located in my main email inbox, it seems) containing a specific word (different, or same) which are not found by the search.
the email is found when doing a search on the sender name, but not when doing a search on a word in the content (body) of the email.

Furthermore, I can reproduce this at will with a given word.

At this stage, my assumption is now it is VERY likely that there is a bug which has excluded these emails from being indexed (or looked for, from what I understand from the search SELECT)

Is this forum read by developers ? is this the place to discuss what looks like real bugs (as opposed to simply looking for help to rebuild the global message database ?)

thanks
User avatar
tanstaafl
Moderator
Posts: 49647
Joined: July 30th, 2003, 5:06 pm

Re: Searching for word not finding an email (Linux)

Post by tanstaafl »

Developers normally don't visit this forum or the official support forums.

I suggest you file a bug report at https://bugzilla.mozilla.org/home against Thunderbird and provide both your debug log and the mbox file for the folder as attachments to the bug report so that somebody else can try to reproduce your problem. If you do that please post a link to the bug report. I suggest you first try to create a much smaller folder that duplicates the problem using messages that are less personal than whats in your inbox.

There should be a "Add an attachment link" on the bug report page.
Post Reply