[Ext] Preserve Download Modification Timestamp 2013.05.11.19

Announce and Discuss the Latest Theme and Extension Releases.
User avatar
Bluefang
Posts: 7857
Joined: August 10th, 2005, 2:55 pm
Location: Vermont
Contact:

[Ext] Preserve Download Modification Timestamp 2013.05.11.19

Post by Bluefang »

Preserve Download Modification Timestamp

When Firefox downloads a file, it sets the file modification time to be the current time. This is pretty much how every browser behaves. However, some will point out that this behavior is actually wrong (though it's a matter of opinion).

This extension allows Firefox to preserve the modification time, that is sent by the server, when you download a file. This should work for any file downloaded via http://, file://, or ftp:// (Firefox 4.0+ and SeaMonkey 2.1+) so long as the server sends the proper information.

Known Issues

  1. Not a bug. Ignores files downloaded by Save Page As "Web Page, complete". Doing this uses a different mechanism.
    • The page is not re-downloaded, so the server information is not available to the download manager
    • The downloaded page is changed by the browser, invalidating the server's modification time
    • Information for the additional files downloaded is not available through the download manager
  2. Firefox bug. If you click on a link to start a download, and the download finishes before you choose where to save it, the required server information is discarded before this extension is notified you downloaded a file (details here). To work around the problem, use the following methods to start a download:
    • File -> Save Page As...
    • Right-Click -> Save Page As...
    • Right-Click -> Save Link As...
    • Right-Click -> Save Image As...
    • Right-Click -> This Frame -> Save Frame As...
  3. The first time the SeaMonkey "download" preferences dialog is opened after installing PDMTS, the "Preserve file modification times when possible" check box does not have the correct state. However, toggling the check box will change the preference correctly. After closing and opening the dialog, or restarting SeaMonkey, the check box will show the correct state.
If you're having problems, be sure to check the Error Console to see what's going on. To see more information in the error console, change the log verbosity to "Info" or "Debug" in the extension preferences.

---------------------------------------------

Check out the code! PDMTS is now on GitHub:
https://github.com/SparkyBluefang/PDMTS

Want to provide a translation? Status-4-Evar can be found on BabelZilla:
http://www.babelzilla.org/forum/index.p ... topic=6531

Please do not email me for support/help/bug reports. I will not respond and will probably just delete it. The preferred way to contact me is to post in the official support topic on MozillaZine or create an issue on GitHub.

Developer Comments

Users that have installed the development/beta version: because you are on the beta channel, you will not receive automatic updates for non-beta releases. I will only post new beta releases when I start development on the next version. So if you want the most recent version, you will have to manually install it.

On that same note: be careful about installing the beta/development version. I know there is a tempting link just below, but it is often older than the current released version. I would hide the link, but Mozilla removed the option to do so...

Change Log

Version 2013.05.11.19b
* Drop support for Firefox 19 and older.
* Drop support for SeaMonkey 2.16 and older.
* Add support for per-window peivate browsing changes in Firefox 20.
* Add support for X-Archive-Orig-Last-Modified (used on archive.org).
* Move to S4E build system. Support install.rdf localization.
* Added locales ja-JP
* Updated translations for: de, fr, pt-BR, sr, sv-SE, zh-CN

Version 2011.03.21.22
* Added locales fr, zh-CN

Version 2011.03.07.00
* Added locales de, el, pt-BR, sr, sv-SE

Version 2010.09.12.18
* Handle when null request data is passed to the network state listener (see Known Issue #2).

Version 2010.09.12.01
* Default log verbosity is now "Info" (was previously "Warning")
* Simplify timestamp caching during download
* Support localization of both the preference UI and logging output
* Support Private Browsing. When private browsing, only show "Error" and "Warning" messages in the log. Also, do not show any download information (source URL and download destination)
* Added a preference to hide download information in the log

Version 2010.09.08.22
* Make sure debug output is still shown if the HTTP header is missing (catch the exception)
* Ignore files downloaded by Save Page As "Web Page, complete". It uses a different mechanism, file information is not available through the download manager, and it changes the downloaded page (invalidating the server's modification time).

Version 2010.09.08.18
* Fixed the "Windows bug" with browser.download.manager.scanWhenDone (details here). If you changed browser.download.manager.scanWhenDone to work around this bug, you can now change it back.

Version 2010.09.07.03
* Add support for Firefox 3.5+ and SeaMonkey 2.0

Version 2010.09.04.02
* Add some more debugging messages

Version 2010.09.01.19
* Added proper logging infrastructure and a preference to control verbosity.
* Added back the extension's options dialog. Can now be enabled/disabled in the extension's options dialog and in the downloads section of the application's options dialog.

Version 2010.08.31.20
* Add preference to SeaMonkey download preferences.

Version 2010.08.31.19
* Minor name change
* Move the preference to be with the rest of the download options in the main Firefox preferences

Version 2010.08.31
Initial Release
* Basic functionality
* Preference to enable/disable during runtime
Last edited by Bluefang on May 11th, 2013, 4:25 pm, edited 14 times in total.
There have always been ghosts in the machine... random segments of code that have grouped together to form unexpected protocols. Unanticipated, these free radicals engender questions of free will, creativity, and even the nature of what we might call the soul...
DERoss
Posts: 35
Joined: August 20th, 2004, 5:15 pm
Location: California
Contact:

Re: [Ext] Preserve Download Modification Timestamp 2010.09.0

Post by DERoss »

This is not working if I right-click on a link and select "Save Link Target As" from the pull-down context menu. The Modified date of the downloaded file reflects the date-time of the download. A log of HTTP headers shows the correct Modified date was sent by the server.

On the other hand, if I right click on an element (e.g., GIF image) on a page and select "Save Image As", I do indeed get the correct Modified date. Also, if I load a file (e.g., HTML, PDF) and select [File > Save Page As] from the menu bar, I get the correct Modified date. (I get the correct date for an HTML file only if it were not compressed (e.g., chunked, gzip); compression suppresses the HTTP header for Last Modified.)

My configuration:

Windows XP Home Edition Version 5.1 (Service Pack 3)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.12) Gecko/20100825 SeaMonkey/2.0.7
Extensions (enabled: 8)
* Adblock Plus 1.2.2 (http://adblockplus.org/)
* DOM Inspector 2.0.4 (http://www.mozilla.org/projects/inspector/)
* Flashblock 1.3.16 (http://flashblock.mozdev.org/)
* Live HTTP headers 0.16
* PrefBar 5.1.1 (http://prefbar.mozdev.org/)
* Preserve Download Modification Timestamp 2010.09.07.03
* Show Password On Input 0.1.3 (https://addons.mozilla.org/thunderbird/addon/6143/)
* ShowIP 0.8.19 (http://code.google.com/p/firefox-showip/)

user.js contains
user_pref("browser.download.manager.scanWhenDone", false);
SeaMonkey was restarted after setting this. Setting confirmed by viewing about:config.
User avatar
Bluefang
Posts: 7857
Joined: August 10th, 2005, 2:55 pm
Location: Vermont
Contact:

Re: [Ext] Preserve Download Modification Timestamp 2010.09.0

Post by Bluefang »

Status update: uploaded versions 2010.09.08.18 and 2010.09.08.22

DERoss, I can't recreate the problem you're having. I tested this with SeaMonkey 2.0.8pre on Linux. When ever I "Save Link Target As", either the file is updated properly or the header is missing.Some additional debugging information would be helpful:
* Change the extensions "log verbosity" to "Debug"
* Open the error console and clear it
* Try downloading a link
* Post the contents of the error console (all of the relevant entries will begin with PDMTS)

Also, regarding HTTP Chunked Transfer Coding, I don't think there is anything in the HTTP spec that says chunked transfers shouldn't have a Last-Modified header. It could just be that the server is not sending them.
There have always been ghosts in the machine... random segments of code that have grouped together to form unexpected protocols. Unanticipated, these free radicals engender questions of free will, creativity, and even the nature of what we might call the soul...
DERoss
Posts: 35
Joined: August 20th, 2004, 5:15 pm
Location: California
Contact:

Re: [Ext] Preserve Download Modification Timestamp 2010.09.0

Post by DERoss »

First of all, I upgraded to 2010.09.08.22. Then I changed the options per your reply. The problem no longer appears.
meywer
Posts: 1
Joined: September 11th, 2010, 4:03 am

Re: [Ext] Preserve Download Modification Timestamp 2010.09.0

Post by meywer »

Now I was able to install the addon in firefox 3.6.
But - when I download files or save websites, still I get the dowload-time instead of the modification time of the files.
DERoss
Posts: 35
Joined: August 20th, 2004, 5:15 pm
Location: California
Contact:

Re: [Ext] Preserve Download Modification Timestamp 2010.09.0

Post by DERoss »

Open the Add-on Manager. Select the Options button for the extension. Make sure there is a checkmark in the checkbox for "Preserve file modification times when possible". I'm also running with "4 - Debug" in the "Log verbosity" selection list. Then select the OK button.
User avatar
Bluefang
Posts: 7857
Joined: August 10th, 2005, 2:55 pm
Location: Vermont
Contact:

Re: [Ext] Preserve Download Modification Timestamp 2010.09.1

Post by Bluefang »

Status update: uploaded version 2010.09.12.01

meywer, please follow the instructions I previously gave to DERoss. Chances are the modification times, for the files your trying to download, are just unavailable.

If there's a PDMTS error in the Error Console, please report it. If you're certain the server is sending the modification time, but the file isn't being updated, post a link so I can try it.
There have always been ghosts in the machine... random segments of code that have grouped together to form unexpected protocols. Unanticipated, these free radicals engender questions of free will, creativity, and even the nature of what we might call the soul...
al_9x
Posts: 205
Joined: October 17th, 2005, 1:33 am

Re: [Ext] Preserve Download Modification Timestamp 2010.09.1

Post by al_9x »

Error logged at the end of the download, Fx 3.6.9, PDMTS 2010.09.12.01, new profile

debug log

Code: Select all

PDMTS: Download status changed - 5
   C:\...\firefox-4.0b6pre.en-US.win32.installer.exe
 ----------
PDMTS: Network status changed - 0x50001 (ERR: 0)
   https://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/firefox-4.0b6pre.en-US.win32.installer.exe
 ----------
PDMTS: File downloading via HTTP://
   C:\...\firefox-4.0b6pre.en-US.win32.installer.exe
 ----------
PDMTS: Download status changed - 0
   C:\...\firefox-4.0b6pre.en-US.win32.installer.exe
 ----------
PDMTS: Download status changed - 1
   C:\...\firefox-4.0b6pre.en-US.win32.installer.exe
 ----------
PDMTS: Download finished, updating timestamp
   C:\...\firefox-4.0b6pre.en-US.win32.installer.exe
 ----------
PDMTS: Last modified time now Sat, 11 Sep 2010 12:47:18 GMT
   C:\...\firefox-4.0b6pre.en-US.win32.installer.exe
 ----------
Error: req is null
Source file: file:///C:/.../extensions/pdmts@caligonstudios.com/components/pdmts.js
Line: 179
User avatar
Bluefang
Posts: 7857
Joined: August 10th, 2005, 2:55 pm
Location: Vermont
Contact:

Re: [Ext] Preserve Download Modification Timestamp 2010.09.1

Post by Bluefang »

Hm... According to the nsIDownloadProgressListener.onStateChange interface, that should never be null... But I guess that's not the case.

Simple enough fix. Thanks for reporting it :)
There have always been ghosts in the machine... random segments of code that have grouped together to form unexpected protocols. Unanticipated, these free radicals engender questions of free will, creativity, and even the nature of what we might call the soul...
al_9x
Posts: 205
Joined: October 17th, 2005, 1:33 am

Re: [Ext] Preserve Download Modification Timestamp 2010.09.1

Post by al_9x »

Bluefang wrote:Hm... According to the nsIDownloadProgressListener.onStateChange interface, that should never be null...


Perhaps you can file a gecko bug?
User avatar
Bluefang
Posts: 7857
Joined: August 10th, 2005, 2:55 pm
Location: Vermont
Contact:

Re: [Ext] Preserve Download Modification Timestamp 2010.09.1

Post by Bluefang »

No, it's that nsIDownloadProgressListener is really an extension of nsIWebProgressListener. So, if anything, it just looks like conflicting/incomplete documentation.

Though I will investigate into why it is null and it may very well be a bug.

But in the context of PDMTS, it doesn't matter (AFAIK) because the error is happening on an event we don't care about.
There have always been ghosts in the machine... random segments of code that have grouped together to form unexpected protocols. Unanticipated, these free radicals engender questions of free will, creativity, and even the nature of what we might call the soul...
User avatar
Bluefang
Posts: 7857
Joined: August 10th, 2005, 2:55 pm
Location: Vermont
Contact:

Re: [Ext] Preserve Download Modification Timestamp 2010.09.1

Post by Bluefang »

Status update: uploaded version 2010.09.12.18
There have always been ghosts in the machine... random segments of code that have grouped together to form unexpected protocols. Unanticipated, these free radicals engender questions of free will, creativity, and even the nature of what we might call the soul...
al_9x
Posts: 205
Joined: October 17th, 2005, 1:33 am

Re: [Ext] Preserve Download Modification Timestamp 2010.09.1

Post by al_9x »

A couple more issues:

  1. Is FTP (ftp://ftp.mozilla.org/pub/mozilla.org/ls-lR.gz) supposed to work?

    Code: Select all

     PDMTS: Last modified time unavailable C:\...\ls-lR.gz
  2. Error at the end of the download (Download Manager Tweak interaction, if either DMT or PDMTS is disabled, no error)

    Code: Select all

    Error: uncaught exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIObserverService.removeObserver]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: chrome://mozapps/content/downloads/downloads.js :: Shutdown :: line 442"  data: no]

    to repro:
    1. fx 3.6.9, new profile, PDMTS 2010.09.12.18, DMT 0.9.2
    2. check "options->downloads->File/Window->Close it when all downloads are finished"
    3. go to ftp://ftp.mozilla.org/pub/mozilla.org/
    4. download (click on) ls-lR.gz
User avatar
Bluefang
Posts: 7857
Joined: August 10th, 2005, 2:55 pm
Location: Vermont
Contact:

Re: [Ext] Preserve Download Modification Timestamp 2010.09.1

Post by Bluefang »

1. Only Firefox 4.0 and SeaMonkey 2.1 support modification times on FTP connections. See the extension description.

However, on Firefox 4, this is running into the null request data problem you previously pointed out. At this point, I'd like to point out that the fix that was landed, then backed out would have suffered from the same problems, because, as far as I can tell, it uses the same events that this extension does. But I'll file a bug on this.

2. You should pass this one over to the DMT people. All this extension does is register a listener and change the modification time of the file. I see no way this could be interfering with another extension.
There have always been ghosts in the machine... random segments of code that have grouped together to form unexpected protocols. Unanticipated, these free radicals engender questions of free will, creativity, and even the nature of what we might call the soul...
al_9x
Posts: 205
Joined: October 17th, 2005, 1:33 am

Re: [Ext] Preserve Download Modification Timestamp 2010.09.1

Post by al_9x »

Bluefang wrote:2. You should pass this one over to the DMT people. All this extension does is register a listener and change the modification time of the file. I see no way this could be interfering with another extension.


It's entirely possible that the fault is theirs or gecko's, however DMT is an established, "Featured" extension with lots of users and downloads, so the onus is more on you in the case of a conflict. If they are doing something wrong that causes this error, it would be helpful if you could point it out to them.
Last edited by al_9x on September 13th, 2010, 1:42 am, edited 1 time in total.
Post Reply