[Ext] ThumbsDown 0.15 - Bulk image downloader - 2010-01-21

[ake79]

ThumbsDown - Bulk/mass image downloader

Latest release: 0.15 - 2010-01-21 - ChangeLog
Compatibility: Firefox 3.6 - 3.6.*

Links: Homepage | Screenshots | User Guide | Forum | AMO

Description:
Say you have a web page with image thumbnails and you want to download the original full-size images with minimum hassle. With ThumbsDown you select the images you want to download simply by clicking their thumbnails and then just start the download. ThumbsDown will download the original images into a folder of your choice.

Things don't happen quite that automagically out-of-the-box. Before ThumbsDown is any use to you, it requires some configuration. Also, ThumbsDown is not a standalone extension. It uses an external download manager DownThemAll! to handle the actual downloading.

[schmooot]

what if the thumbnail image isn't a link to the original image?

I am trying to use this extension in similar circumstances where the page is loaded with thumbnail images but they don't link to the originals. However the original image is the same just in a different folder on the server (imaget instead of image) and the image name is preceded by a t for the thumb.

for example. The url for the thumbnail images are like this:

xxxxx.com/imaget/a/ta4.jpg
xxxxx.com/imaget/b/tb800.jpg
xxxxx.com/imaget/c/tc40.jpg

They don't actually link to the original images but the url for the original images are like this:

xxxxx.com/image/a/a4.jpg
xxxxx.com/image/b/b800.jpg
xxxxx.com/image/c/c40.jpg


I could go into each one and save each image individually but there are 2000+ of them and I know there is an easier way.

If there is a way to get this extension to do that or maybe there is a different extension that will can someone enlighten me?

[ake79]

If thumbnail isn't a direct link to the full size image, you need to use regexp or custom script.

From your description, I'm guessing, something like this might work.

type: regexp
pattern source: thumbnail
pattern: xxxxx\.com/imaget/(\w+)/t(\w+)\.jpg
template: xxxxx.com/image/$1/$2.jpg

This all is explained in the user guide. Not very well perhaps but still.

[schmooot]

alright I've done some things to make it easier.

I now have a page chock full of thumbnails (1440 to be exact) I made a list of all the ones I need to download and then made a page containing the links.

the thumbnail codes are like this:

<a href=http://www2.xxxx.com/image/z/zooa1000.jpg><img src="http://www2.xxxx.com/imaget/z/tzooa1000.jpg"></a>

<a href=http://www2.xxxx.com/image/t/trac4936.jpg><img src="http://www2.xxxx.com/imaget/t/ttrac4936.jpg"></a>

<a href=http://www2.xxxx.com/image/b/bacu7057.jpg><img src="http://www2.xxxx.com/imaget/b/tbacu7057.jpg"></a>

can I just make a matcher for this page and then use the direct method?

[ake79]

Direct, yes: http:/ /www2\.xxxx\.com/image/\w+/\w+\.jpg (remove extra space)

[schmooot]

crap it won't let me link the images from another server...imma have to go a different route. Thanks

[rayser]

Please add POSTLOAD() so we can load a page with POST-Data and also RETURNREF() so we can download sending special REFERERS that would be great, keep up the good work. Kivä Ohjelma.

[ake79]

Kiitoksia rayser.

Sounds doable. Can you provide an example where those are needed?

[rayser]

with flickr i was using some kind of UserJS Bookmarklet combination so here is the code i wanned to use with thumbsDown:

matcher: http\:\/\/www\.flickr\.com\/photos\/[\w@-]+\/\d+/

custom:
function makePOSTRequest(url, parameters) {
http_request = new XMLHttpRequest();
http_request.overrideMimeType('text/html');
http_request.onreadystatechange = alertContents;
http_request.open('POST', url, true);
http_request.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
http_request.setRequestHeader("Content-length", parameters.length);
http_request.setRequestHeader("Connection", "close");
http_request.send(parameters);
}

function alertContents() {
if (http_request.readyState == 4) {
if (http_request.status == 200) {
url = http_request.responseText.match(/\<iframe\ssrc=\"([^\"]+)/im);
//result = http_request.responseText;
//document.getElementById('myspan').innerHTML = result;

LOAD(url[1]);

} else {
alert('There was a problem with the request.');
}
}
}


switch (state) {
case 0:
stringa = linkUrl.split('/');
makePOSTRequest('http://www.flickr.com/services/api/explore/', 'method=flickr.photos.getSizes&magic_cookie=2cea3d04bb1cc1a4a14e2cf01105223a&enable_photo_id=on&param_photo_id='+stringa[stringa.length-2]+'&sign_call=full');

case 1:
tagg = document.getElementsByTagName('size');

RETURN(tagg[tagg.length-1].getAttribute('source'));
break;
}

only problem is ThumbsDown doesnt know XMLrequest-Handler.

I have no idea what referer ThumbsDown is sending right now, but i just thought that often when i tried to download pitures, the referer was a problem. i will tell you an example as soon as i have one

[neojam]

Hallo,
i wondering how can i solve this one:
http://img504.imageshack.us/img544/2499 ... 879po4.png

p.s.
thanks for this great extension.

[ake79]

Thanks neojam.

If the dynamic part (in your example "Xdftk") is not included in the thumbnail (or link) url, it's simply not possible to make it work with regexp script. So, you need to use custom script. It's explained in the last chapter of the user guide. It's a bit out of date, as there is also second code template included but it's not explained in the user guide.

Try figure it out yourself first. Ask, if you have troubles.

(I'm working on making the extension a bit user friendlier but let's see when I have the time to finish the next version.)

[neojam]

Yep, the custon script has worked fine, thanks for the tip.

Also, would it be possible to make Thumbsdown to create folders on its own, if they don't exist?

[ake79]

Also, would it be possible to make Thumbsdown to create folders on its own, if they don't exist?

The current download-everything-into-a-single-existing-folder is suppose to be a temporary thing. Just haven't got around to make anything better.

Can you describe how you would want it to work. I'm open to ideas.

[neojam]

Well I really didn't want anything special, just a feature for creation of folders/paths based on the input from "Save images to:" field, if those dont yet exist.
Currently TD(ThumbsDown!) cant create folders on its own, so you have to manually create a path yourself first, before it can be used in the "Save images to:" input field...


But on the other hand, since TD is supposed to be an "automated gallery leecher", maybe it would be a good idea to implement some sort of an automated subfolder creation feature as well.

Maybe like this one :
http://img518.imageshack.us/img518/3771/autoexc6.png
Which if activated, will generate subfolders based on the URL structure of the first thumbinail from the TD download session.
In other words: You start the TD Download, TD gets the URL from the first matching thumb and generates subdirectories from it by using the checkmarks/regex from the menu i've photoshoped above. ^_^

So what do you think?

p.s.
Or we could simply shrink the whole thing by using only the "Automated folder creation 2" part on my image, to generate subfolders by using backreference from the whole URL (and not just from the thumb name alone).
Though it will be more harder to use for people with little regex experience...

[xbadgerx]

1st: thanks for the extension!
2nd: I've tried to get some downloading working, but have only been able to with using a customised JavaScript Image RegExp code template. It may simply be because I don't understand the RegExp 'script' properly (I'm not much of a programmer, and not any sort of web programmer).
I have a page with thumbnails which link to other pages that contain the image I want to download. Page source says:
<a href="http://xxx.com/uploader/view/full/8839_zg990" target="_blank">
<img src="http://xxx.com/uploader/thumbs/small/8839_zg990/a01.jpg"></a>
full size image URL:http://xxx.com/uploader/files/8839_zg990/a01.jpg
The 8839_zg990 part varies for every set of link URL/thumbnail URL/full image URL.

I'd thought that something like the following would work:

Code: Select all

Matcher: http://xxx\.com/uploader/view/full/(\w+)
Type: RegExp
Pattern source: Thumbnail
Pattern: http://xxx\.com/uploader/thumbs/small/(\w+)/(\w+)
Template: http://xxx\.com/uploader/files/$1/$2

If I select Pattern source as thumbnail then the pattern should match the thumbnail img src URL? I don't think I should use a Pattern source of link, since the link URL (the <a href="..." one, right?) doesn't include the full size image filename, so I couldn't pattern match on it to use in the Template field.

I presume the Matcher field is meant to match the link URLs (not the initial/staring webpage URLs), correct?

I tried using a customised JS Image ID but the actual full size image didn't have an element ID as far as I could see. A regexp of ^http://xxx\\.com/uploader.+$ worked in a customised JS Image RegExp template.

Anyway, thanks for sharing your work. Cheers.