Removing duplicate contacts

Discussion of features in Mozilla Thunderbird
User avatar
mbnoimi
Posts: 156
Joined: August 23rd, 2006, 1:25 pm
Location: Aleppo, Syria
Contact:

Removing duplicate contacts

Post by mbnoimi »

Hi All,
I've about 12000 address in my contacts but there are many duplicates included.

How I can remove duplicate contacts?

PS
I tried to use "Duplicate Contact Manager2 ver.1.2" extension but it's not practical at all because it works for number of addresses less than 1000 even it's not allow the user to modify any contact during the process in case of merging.
User avatar
DanRaisch
Moderator
Posts: 127187
Joined: September 23rd, 2004, 8:57 pm
Location: Somewhere on the right coast

Re: Removing duplicate contacts

Post by DanRaisch »

1. Have you tried this extension? https://addons.mozilla.org/en-US/thunderbird/addon/2505

2. Alternatively, export all contacts to a CSV format file. Load that file into a spreadsheet program and sort by appropriate columns to make identification of duplicates easier. Delete unwanted duplicates, save the file back to CSV format and import that file into Thunderbird. Delete all of the data in the existing Thunderbird address book and copy (drag and drop) the imported data to that book. After this you can delete the now empty imported address book.
User avatar
mbnoimi
Posts: 156
Joined: August 23rd, 2006, 1:25 pm
Location: Aleppo, Syria
Contact:

Re: Removing duplicate contacts

Post by mbnoimi »

Have you tried this extension

sure I mentioned it in my note (it seems that you read my post quickly)!!!

Alternatively, export all contacts to a CSV format file

Wow this is unbelievable!!! this procedure truly unpractical and exhausts I need a solution for at once procedure not a solution takes hours to run and it may cause disasters (for example removing un-duplicated contacts)
User avatar
DanRaisch
Moderator
Posts: 127187
Joined: September 23rd, 2004, 8:57 pm
Location: Somewhere on the right coast

Re: Removing duplicate contacts

Post by DanRaisch »

Perhaps you read too quickly. You mentioned Duplicate Contact Manager2 ver.1.2. The extension I posted a link to is Duplicate Contact Manager 0.6. That is a different extension from a different author.

I posted the spreadsheet process as an alternative only if you are not satisfied with the available extensions for removing duplications.
  • A) There would be no risk of disaster as it would always be possible to retain an unedited copy of the original data.
  • B) With 12,000 contacts to be reviewed, there is no instantaneous solution available. If the user it to be involved in reviewing the records that are considered duplicates, the process is going to be time consuming, whether the process is within Thunderbird using an extension or outside of Thunderbird using some other procedure.
User avatar
mbnoimi
Posts: 156
Joined: August 23rd, 2006, 1:25 pm
Location: Aleppo, Syria
Contact:

Re: Removing duplicate contacts

Post by mbnoimi »

Perhaps you read too quickly. You mentioned Duplicate Contact Manager2 ver.1.2. The extension I posted a link to is Duplicate Contact Manager 0.6. That is a different extension from a different author.

Although it's not from same Author but it did exactly same thing even it has same GUI, the worst thing is ...... it not working with TB 3.0.4!!!!!

With 12,000 contacts to be reviewed, there is no instantaneous solution available

I'm wondering how this happened!! why nobody faced duplications issue? does this problem related to old contacts (I'm using TB since earlier versions)? or it's TB simply can't deal with this issue?

PS
I think Outlook users don't suffer from this problem, while I'm very nervous now because of it... sorry.
User avatar
DanRaisch
Moderator
Posts: 127187
Joined: September 23rd, 2004, 8:57 pm
Location: Somewhere on the right coast

Re: Removing duplicate contacts

Post by DanRaisch »

How what happened? How you got 12,000 contacts? That would be something you would have to determine as I don't have any information about how you use (or have used) your installation of Thunderbird. Updating from an older version of Thunderbird to a newer does not cause duplication of contacts if performed correctly.
User avatar
mbnoimi
Posts: 156
Joined: August 23rd, 2006, 1:25 pm
Location: Aleppo, Syria
Contact:

Re: Removing duplicate contacts

Post by mbnoimi »

How you got 12,000 contacts?

In addition to manual adding I got many contacts from old TB contacts (exported them as LDIF), Email Address Crawler extension (currently it's not working with TB 3) and my old Outlook database from the work (currently we use TB instead) thus I got many duplicates.

Currently What I need is:
Automatic solution for removing duplicates without need to use manual removal because there are many duplicates and it's very very difficult to recognize them by exporting, filtering and manual removal by spreadsheet editors (unless if there is a way for automatic removal by shpreadsheet editors).
User avatar
mbnoimi
Posts: 156
Joined: August 23rd, 2006, 1:25 pm
Location: Aleppo, Syria
Contact:

Re: Removing duplicate contacts

Post by mbnoimi »

I discovered something new, Ooo Calc couldn't open the exported file because of "
The maximum number of rows has been exceeded. Excess raws, were not imported!"

Any way this is Calc issue and I'll ask about it in OpenOffice support forums but I preferred to point it for showing the challenges which any one could face them during removing duplicated contacts in TB.
User avatar
DanRaisch
Moderator
Posts: 127187
Joined: September 23rd, 2004, 8:57 pm
Location: Somewhere on the right coast

Re: Removing duplicate contacts

Post by DanRaisch »

Sorry but as far as I am aware there is nothing like that available.

You were worried about a disaster? Consider how risky it would be to let a piece of software decide, without user interaction, which of two contacts to consider the one to save. Is John Smith the same as John H. Smith? If the email address is the same but the first and last name are different, is it the same contact? Etc. etc.
User avatar
DanRaisch
Moderator
Posts: 127187
Joined: September 23rd, 2004, 8:57 pm
Location: Somewhere on the right coast

Re: Removing duplicate contacts

Post by DanRaisch »

What version of OOCalc are you using? Calc 3.0 can store a maximum of 65,536 rows with 1,024 columns in each sheet.
User avatar
mbnoimi
Posts: 156
Joined: August 23rd, 2006, 1:25 pm
Location: Aleppo, Syria
Contact:

Re: Removing duplicate contacts

Post by mbnoimi »

What version of OOCalc are you using?

I'm using version 3.2.0 but it's important to say that I'm using long UTF-8 strings (I'm Arabic user) for that most string items are long by comparing to Latin strings.
User avatar
couldabeen
Posts: 6729
Joined: September 9th, 2003, 11:24 am
Location: I'm Right Here

Re: Removing duplicate contacts

Post by couldabeen »

@Bashir Alnoimi, the first thing you need to do is go to the preferences and stop letting TB add contacts to those YOU email to.
Open the address book and bring up the Options panel. Under Composition, there is a box at the bottom to 'Automatically add outgoing address to'. Is it checked? If so, UN-Check it. That will stop future duplicates.

I can't hep you with the rest of them. I did that with OOoCalc. If you will take the time to do it manually, and keep the Outgoing addresses from being written, you should not continue to have this problem.
Free Your Mind, And Your A$$ Will Follow - Funkadelic, 1970
User avatar
mbnoimi
Posts: 156
Joined: August 23rd, 2006, 1:25 pm
Location: Aleppo, Syria
Contact:

Re: Removing duplicate contacts

Post by mbnoimi »

I found this tutorial it may fix the issue:
http://user.services.openoffice.org/en/ ... 60#p106040

but I faced a problem with connecting CSV with the Table as mentioned in my post:
http://user.services.openoffice.org/en/ ... 61&t=30372
User avatar
DanRaisch
Moderator
Posts: 127187
Joined: September 23rd, 2004, 8:57 pm
Location: Somewhere on the right coast

Re: Removing duplicate contacts

Post by DanRaisch »

Try opening the same CSV file in OpenOffice Calc rather than in Base.
User avatar
mbnoimi
Posts: 156
Joined: August 23rd, 2006, 1:25 pm
Location: Aleppo, Syria
Contact:

Re: Removing duplicate contacts

Post by mbnoimi »

stop letting TB add contacts to those YOU email to

Thanks I'm already unchecked it.
Any way I'm wondering why TB not smart enough in this point! it should fix this issue automatically I used Outlook before and I couldn't face this horrible problem before (although I hate Outlook enough cuz it's stupid enough specially without extensions).
Locked