Addressbook data loss in 102 update - howto retrieve data?

[martinr]

My TB just automatically updated to version 102.2.1.
It involved an address-book revamp.
The new address book is missing fields in its data model, that were in use by the old address book, that held important data to me.
Now the data is missing. How can I retrieve the missing data?
I would like to revert the update and be able to backup my old address-books, before I'm ready to migrate to the new format.
In order to be able to recover the now missing data.

Who knows a solution?

[martinr]

The address book export as .csv functionality doesn't work.

You can click it, but nothing is exported...

I retested all the export formats (csv, tab/txt, ldif) and none worked except for vcard format (.vcf), which is useless for my sync purposes.

I wish I would have known that the addressbook data fields would be lost in advance, then I could have postponed the update, run a final sync and saved the data in csv format beforehand.

[martinr]

When I look in the profile folder, then I can't find backup files.
I can only find:

$ ls abook*
abook-1.sqlite abook-2.sqlite abook-3.sqlite abook.sqlite
abook-1.sqlite-shm abook-2.sqlite-shm abook-3.sqlite-shm abook.sqlite-shm
abook-1.sqlite-wal abook-2.sqlite-wal abook-3.sqlite-wal abook.sqlite-wal
abook-1.v2.sqlite abook-2.v2.sqlite abook-3.v2.sqlite abook.v2.sqlite
abook-1.v3.sqlite abook-2.v3.sqlite abook-3.v3.sqlite abook.v3.sqlite

When I analyse them I get:
$ file abook*
abook-1.sqlite: SQLite 3.x database, user version 4, last written using SQLite version 3038003, page size 32768, writer version 2, read version 2, file counter 28, database pages 32, 1st free page 3, free pages 1, cookie 0x7, schema 4, UTF-8, version-valid-for 28
abook-1.sqlite-shm: data
abook-1.sqlite-wal: empty
abook-1.v2.sqlite: SQLite 3.x database, user version 2, last written using SQLite version 3031001, page size 32768, writer version 2, read version 2, file counter 12, database pages 32, cookie 0x6, schema 4, UTF-8, version-valid-for 12
abook-1.v3.sqlite: SQLite 3.x database, user version 3, last written using SQLite version 3036000, page size 32768, writer version 2, read version 2, file counter 27, database pages 32, 1st free page 3, free pages 1, cookie 0x7, schema 4, UTF-8, version-valid-for 27
abook-2.sqlite: SQLite 3.x database, user version 4, last written using SQLite version 3038003, page size 32768, writer version 2, read version 2, file counter 5, database pages 24, 1st free page 3, free pages 2, cookie 0x7, schema 4, UTF-8, version-valid-for 5
abook-2.sqlite-shm: data
abook-2.sqlite-wal: empty
abook-2.v2.sqlite: SQLite 3.x database, user version 2, last written using SQLite version 3031001, page size 32768, writer version 2, read version 2, file counter 3, database pages 24, cookie 0x6, schema 4, UTF-8, version-valid-for 3
abook-2.v3.sqlite: SQLite 3.x database, user version 3, last written using SQLite version 3036000, page size 32768, writer version 2, read version 2, file counter 4, database pages 24, 1st free page 3, free pages 2, cookie 0x7, schema 4, UTF-8, version-valid-for 4
abook-3.sqlite: SQLite 3.x database, user version 4, last written using SQLite version 3038003, page size 32768, writer version 2, read version 2, file counter 4, database pages 10, 1st free page 3, free pages 2, cookie 0x7, schema 4, UTF-8, version-valid-for 4
abook-3.sqlite-shm: data
abook-3.sqlite-wal: empty
abook-3.v2.sqlite: SQLite 3.x database, user version 2, last written using SQLite version 3031001, page size 32768, writer version 2, read version 2, file counter 2, database pages 10, cookie 0x6, schema 4, UTF-8, version-valid-for 2
abook-3.v3.sqlite: SQLite 3.x database, user version 3, last written using SQLite version 3036000, page size 32768, writer version 2, read version 2, file counter 3, database pages 10, 1st free page 3, free pages 2, cookie 0x7, schema 4, UTF-8, version-valid-for 3
abook.sqlite: SQLite 3.x database, user version 4, last written using SQLite version 3038003, page size 32768, writer version 2, read version 2, file counter 4, database pages 10, 1st free page 3, free pages 2, cookie 0x7, schema 4, UTF-8, version-valid-for 4
abook.sqlite-shm: data
abook.sqlite-wal: empty
abook.v2.sqlite: SQLite 3.x database, user version 2, last written using SQLite version 3031001, page size 32768, writer version 2, read version 2, file counter 2, database pages 10, cookie 0x6, schema 4, UTF-8, version-valid-for 2
abook.v3.sqlite: SQLite 3.x database, user version 3, last written using SQLite version 3036000, page size 32768, writer version 2, read version 2, file counter 3, database pages 10, 1st free page 3, free pages 2, cookie 0x7, schema 4, UTF-8, version-valid-for 3

They are all schema version 4, which is then new schema I suppose.

[morat]

Are you asking how to dump the address book?

Code: Select all

sqlite3.exe abook.sqlite .dump

Thunderbird 102 SQLITE_VERSION is 3.29.0.

Reference
http://searchfox.org/mozilla-esr102/sea ... TE_VERSION

SQLite Download Page
https://www.sqlite.org/download.html
https://www.sqlite.org/2019/sqlite-tools-win32-x86-3290000.zip

Similar thread
http://forums.mozillazine.org/viewtopic ... &t=3099228

[martinr]

Are you asking how to dump the address book?

Nope, I need a backup of my data before the TB v102.2.1 update destroyed it.
Even better yet would be a roll-back to the last version pre 102.2.1, so I could run a final sync and backup my data.
Thanks for your reply though.

[tanstaafl]

I can export my personal address book as a .csv file using Thunderbird 102.2.1. It looks fine using LibreOffice.

The address book conversion from *.mab to *.sqlite is normally done in version 78. When it did that it made backups of the *.mab files , and then deleted the data in the original files as it ported it to the sqlite files. I forget what file extension they used (it might have been *.sav) for the saved copy but it kept abook etc. as part of the file name etc.

What version of Thunderbird did you upgrade from?

[martinr]

What version of Thunderbird did you upgrade from?

I'm on the update channel and get regular updates, so it must have been recent. Is there a way to find out?

[tanstaafl]

help -> more troubleshooting information will list it (second line in Application Basics). help -> about Thunderbird will also list it, though if a update is available that typically forces an update regardless of your update settings.

[martinr]

help -> more troubleshooting information will list it (second line in Application Basics). help -> about Thunderbird will also list it, though if a update is available that typically forces an update regardless of your update settings.

Nope, that doesn't show the previous version that I upgraded from. It shows the version currently installed:

Code: Select all

Application Basics
Name 	Thunderbird
Version 	102.2.1
Build ID 	20220831174222

Ah, I found the update path below:

thanks.

Yet another update is ready for install:

Version 102.2.2, first offered to channel users on September 7, 2022

I switched off automatic updates and declined for the moment.

I tested address book export again and none of the export formats delivered a result (csv, tab/txt, ldif) except for the vcard format (.vcf). I need the old csv file format for sync with other tools.

[martinr]

I did an analysis of the vcard data to derive what information appears to be missing:
These are the results:

(old pre v102.2.1 data fields:)
1 First Name
2 Last Name
3 Display Name
4 Nickname
5 Primary Email
6 Secondary Email
7 Screen Name MISSING
8 Work Phone
9 Home Phone
10 Fax Number
11 Pager Number
12 Mobile Number
13 Home Address
14 Home Address 2
15 Home City
16 Home State
17 Home ZipCode
18 Home Country
19 Work Address
20 Work Address 2
21 Work City
22 Work State
23 Work ZipCode
24 Work Country
25 Job Title
26 Department
27 Organization
28 Web Page 1
29 Web Page 2
30 Birth Year MISSING (in another format now)
31 Birth Month MISSING (in another format now)
32 Birth Day MISSING (in another format now)
33 Custom 1
34 Custom 2
35 Custom 3
36 Custom 4
37 Notes

Screen Name was in use as a logical clock versioning field for my sync tool.
It is imperative that I can make a CSV export of my address books in the state they were in before the update.
I'm a heavy user of the address books (hundreds of entries, all updated regularly).

[martinr]

I reverse engineered the new data field that seem to have appeared in the new data model:
New fields as it seems post v102.2.1:

• Chat Accounts (+): IMPP:TEST_chat_Accounts, IMPP:TEST_chat_Accounts2
  (Org) Role: ROLE:TEST_Role
  Phone Numbers (+): TEL;VALUE=TEXT:TEST_phone_no_none
  Email (+): EMAIL:TEST-Add_email_address
  Website (+): URL:https://TEST-WEBSITE

(the + denotes 1 or more entries possible)

So it seems that the "Screen Name"field from the old data model was replaced by the "IMPP" in the new model.
Why was the data not migrated from the "Screen Name"field to the "IMPP" field? It broke my sync and more severely it resulted in data loss.

[martinr]

Is there a way to roll back the automatic Thunderbird update from v102.2.1 back to 91.13.0?
I need to get my address book data saved to a .CSV file (which is broken in v102.2.1).

I saw a link to article "Recover user data missing after Thunderbird update article." (souce).
But when I follow the link I get:

Page Not Found

Then I searched for the missing page. Couldn't find it, but saw this page instead: "DOWNGRADING Thunderbird Ver 78 to previous Ver 68 - APPEAL to ALL the great MOZILLA & THUNDERBIRD experts ...!".

Assuming you have windows OS then you can try this: go to run (search run, select that run app) and write on that small field this: thunderbird -p --allow-downgrade
...
All your data should be there except email addresses from the address book.
...
To add also those email addresses back follow below steps:
At the top right of the Thunderbird window, click the menu button (or use the regular menu at the top), then select Help: Troubleshooting Information. Click on Open Folder (if you have multiple options click on the one on the profile zone). Close TB.
On the folder opened by TB - you will find the old files which manage that information with extension .mab.bak

When I try that then I find out that NO .mab.bak EXISTS in my profile.

The address book data model has changed, the export address book as .CSV functionality is broken in TB v102.2.1 and address book backup files are missing.
It's seems like a total cluster f*ck.

How can I get the total content of my address book back and export it as a .CSV file?

[tanstaafl]

You can export the address book as a .csv file with version 102. They've just changed where that menu option is, probably due to their adding support for exporting/importing profiles as .zip files.

Right click on the address book name and select export from the context menu. It defaults to exporting it as a .csv file. I've done that and opened the .csv file using a spreadsheet program, it looks fine.

I have *.mab.bak files in a backup of my profile but not in my profile. I suspect I deleted them rather than it being done as a side effect of another update. That conversion occurred in version 78, over a year ago.

[martinr]

You can export the address book as a .csv file with version 102. They've just changed where that menu option is, probably due to their adding support for exporting/importing profiles as .zip files.

Right click on the address book name and select export from the context menu. It defaults to exporting it as a .csv file. I've done that and opened the .csv file using a spreadsheet program, it looks fine.

I have *.mab.bak files in a backup of my profile but not in my profile. I suspect I deleted them rather than it being done as a side effect of another update. That conversion occurred in version 78, over a year ago.

.CSV export doesn't work in v102.2.1 (Windows 10) for me, see posting 2. I did exactly what you explained.
I tried over many times. Please check that you're testing on v102.2.1.

I did not touch my profile and I have no *.mab.bak files. You don't have them either. I suppose they should have been created with the recent upgrade to v102, so they weren't. Hence no backup...

[wsmwk]

You can export the address book as a .csv file with version 102. They've just changed where that menu option is, probably due to their adding support for exporting/importing profiles as .zip files.

Right click on the address book name and select export from the context menu. It defaults to exporting it as a .csv file. I've done that and opened the .csv file using a spreadsheet program, it looks fine.

I have *.mab.bak files in a backup of my profile but not in my profile. I suspect I deleted them rather than it being done as a side effect of another update. That conversion occurred in version 78, over a year ago.

.CSV export doesn't work in v102.2.1 (Windows 10) for me, see posting 2. I did exactly what you explained.
I tried over many times. Please check that you're testing on v102.2.1.

I did not touch my profile and I have no *.mab.bak files. You don't have them either. I suppose they should have been created with the recent upgrade to v102, so they weren't. Hence no backup...

*.mab.bak files are not created. They would be your original address book files, simply renamed after the migrate process was completed. And Thunderbird doesn't delete them.

As for export not working, please check error console after attempting the export.