Userscript: lists fixed bugs of FF for desktop in Mercurial

[darkred]

I've written this script Mozilla Mercurial - Filters changelogs(pushlogs) (for Greasemonkey)

It generates a list (in a draggable jQuery dialog) of only the bugs related to Firefox for desktop in Mozilla Mercurial pushlogs.
It's basically for creating lists similar to the "The Official Win32 xxxxxxx builds" but for any date interval.

Screenshot for the pushlog for mozilla-inbounds from yesterday up to now:

and screenshot of logging in Web Console:


I hope you find it useful.

[Josa]

Awesome work! Way easier for me (or anyone) to create/edit the list!
Just 2 things:
- it only works when you're not logged in on Bugzilla - not a problem, just a warning to anyone who wants to try it.
- tickets that we don't have access are relevant, usually they are security bugs.

[darkred]

it only works when you're not logged in on Bugzilla - not a problem, just a warning to anyone who wants to try it.

No, it works ok in both cases (non-private bugs are available to anyone).
It didnt work though for plain http links (it only worked for m-i https links, I missed that!, sorry), so I replaced the `@include` rule with a regex - please redownload the script to get the updated version.

- tickets that we don't have access are relevant, usually they are security bugs.

You may find those bugs filtered out for being private, by opening Web Console and enter 'restricted access' in the 'Filter output' textbox.
Then you may click on the relevant links to open them (those that you have access to).
I think that (for the most people - which don't have access to such bugs) it's better to leave the script as it is, i.e. only showing these bugs in the Web Console. Don't you agree?

[Josa]

it only works when you're not logged in on Bugzilla - not a problem, just a warning to anyone who wants to try it.

No, it works ok in both cases (non-private bugs are available to anyone).
It didnt work though for plain http links (it only worked for m-i https links, I missed that!, sorry), so I replaced the `@include` rule with a regex - please redownload the script to get the updated version.

Maybe the problem is because I have edit privileges? I was getting 'status - undefined' if I remember correctly.

- tickets that we don't have access are relevant, usually they are security bugs.

You may find those bugs filtered out for being private, by opening Web Console and enter 'restricted access' in the 'Filter output' textbox.
Then you may click on the relevant links to open them (those that you have access to).
I think that (for the most people - which don't have access to such bugs) it's better to leave the script as it is, i.e. only showing these bugs in the Web Console. Don't you agree?

Well, ok.

[darkred]

it only works when you're not logged in on Bugzilla - not a problem, just a warning to anyone who wants to try it.

No, it works ok in both cases (non-private bugs are available to anyone).
It didnt work though for plain http links (it only worked for m-i https links, I missed that!, sorry), so I replaced the `@include` rule with a regex - please redownload the script to get the updated version.

Maybe the problem is because I have edit privileges? I was getting 'status - undefined' if I remember correctly.

I just tried again both pushlogs for m-i and m-c (since 1 day ago) and there's not even one entry in Web Console with 'status - undefined'.
Would you please try it while logging in, to make sure it works ok ?
Maybe in your case the HTML code is different?

For the status value, the script matches in the HTML code either:
a string split in two rows, eg.:

Code: Select all

      <span id="static_bug_status">VERIFIED
          FIXED
      </span>

or in single row, e.g:

Code: Select all

      <span id="static_bug_status">NEW
      </span>

[Josa]

Because I'm logged in - with edit privileges:

BugNo: 1192432
Title: Support custom icons in Web Extensions
Status: undefined
Product:
Component:
Platform: <input type="hidden" id="rep_platform_dirty">/<select id="rep_platform"
Whiteboard: undefined Mozilla_Mercurial_-_Filters_changelogs.user.js:201:9
"https://bugzilla.mozilla.org/show_bug.cgi?id=1192432
is IRRELEVANT because of it's Status --> undefined"

The page is totally different when you can edit it. It would be a lot of trouble to make it work. Don't worry.

[darkred]

For me it's parsed correctly, in either logged in or not.

So, it's apparent that the HTML changes when you have edit privileges.
Could you send me please via pm a saved page of that bug while you are logged in (upload it in e.g. mediafire or mega.co.nz, or whatever you got),
so that I may show you what lines (in fact what regular expessions in the script) to edit in the script?

[darkred]

After examining a response that the script retrieves versus the relevant saved page (saved pages that Josa sent me),
I saw that there are differences in newlines and encodings: https://i.imgur.com/GTNW7z0.jpg.
therefore, I can't make the script work in this case too, only based on a saved page.

So, the script works ok only if you either:*
- haven't login in Bugzilla, or
- have login (but without having edit privileges)


Also, I've renamed it into Mozilla Mercurial - generate list of recently fixed bugs related to Firefox for desktop

[johnp_]

Wow, this looks really nice!

But why not use the REST API?

And if you want to keep parsing HTML, the format=multiple GET-Parameter may give you better results (and also only one network request necessary, example).

Example REST API call: https://bugzilla.mozilla.org/rest/bug?i ... whiteboard

edit: Pseudo-code:

Code: Select all

    var base_url = "https://bugzilla.mozilla.org/rest/bug?include_fields=id,summary,status,resolution,product,component,platform,whiteboard&id="

    var id, bugIds = array();
    for each url in buglinks: // buglinks is like your linksTexts
        id = url.grep("id=?"); // too lazy to do proper regex, but you get the gist / already have code for that
        if (!bugIds.contains(id)) bugIds.push(id);

    var rest_url = base_url + bugIds.join();

    $.getJSON(rest_url, function(data) {
          // may need to sort data.bugs or split into irrelevant/relevant or sth like that
          $.each(data.bugs, function(bug) {
                // add html code for this bug
                // aka in your code
                // bugsComplete.push("<a href=" + url + ">#" + bug.id + "</a>........");
           });
           // edit2.5
           $.each(data.bugs, function(bug) {
                bugIds.remove(data.id);
           });
           for each id in bugIds:
                // everything remaining was not in the REST API output, so add them to HTML as "Access Denied / Not Found"
                // to be sure it's "Access Denied", and not "Not Found", call b.m.o/rest/bug/{id}
     });

disclaimer: I'm not a js guy, so take everything with a grain of salt

edit2: The REST API silently discards "Access denied" bugs in the mass_request done by base_url, so they have to get double checked

[darkred]

Thanks a lot for your suggestion and your help, johnp_!!
I just updated the script to use the REST API, i.e. the script now requests from Bugzila only those fields that it requires(not entire HTML pages),
therefore resulting in reduction to the procedure duration to less than half (less than 15 sec for parsing 165 bugs).
Also, the values are now in a standard format, so it's easier to extract from each response.

[johnp_]

Hmm... you're still making a network request for each bug.
I've had some time to write a version that retrieves all bugs in just one network request (although the discarding-issue workaround is only pseudo-code and still needs to be implemented properly).
Unfortunately, the BMO backend seems to be quite slow, when asked for dozens of bugs at once, so I'm not sure if it's a performance win.
I'd appreciate it if someone could test this with a better internet connection.

code is up at https://pastebin.mozilla.org/8851687 (and my disclaimer from above still stands)

edit: v4 is up

[darkred]

Hmm... you're still making a network request for each bug.
I've had some time to write a version that retrieves all bugs in just one network request (although the discarding-issue workaround is only pseudo-code and still needs to be implemented properly).
Unfortunately, the BMO backend seems to be quite slow, when asked for dozens of bugs at once, so I'm not sure if it's a performance win.
I'd appreciate it if someone could test this with a better internet connection.

code is up at https://pastebin.mozilla.org/8851431 (and my disclaimer from above still stands)

I'm also not a js guy (and I'm not familiar with JSON in particular). Your approach seemed more advanced for me to implement, that's why I followed the simpler (for me) way.
Your way is the best (and in essence much less complicated) in every aspect! It's always great learning new things!

I just tried your code, and it's much faster: the duration was now cut to half (8 sec vs 16sec for my approach for 106 bugs, in an 24/1 ADSL connection). Well done!


Sorting by product:component is not applied, though (only by product). To fix this, change line #62 into:

Code: Select all

return (a.product + ": " + a.component) > (b.product + ": " + b.component);

[darkred]

Also, something about the "enddate" value when you do a search:
instead of putting "now" you may put the changeset of your currently installed build (via about:buildconfig|click on that link)
so that you may view only the bugs that have been fixed till your current build - not those that have been fixed until the current time in the relevant branch.

For example: (for mozilla-inbound) instead of using

Code: Select all

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?startdate=1+day+ago&enddate=now

you may enter this: (it's for the changeset of my currently installed build: https://hg.mozilla.org/integration/mozi ... a4190c383e)

Code: Select all

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?startdate=1+day+ago&enddate=c4dd82aa903d

[johnp_]

my final version (with some fixes & improvements) is up now. feel free to use it as you like (as in CC BY 4.0).
https://pastebin.mozilla.org/8851687

[darkred]

Thanks a lot for your valuable contribution!
So, I've uploaded your version 4 here (screenshot)

I've further modified your version, by displaying the results as a sortable table, showing in an extra column the "Last Modified Date" of each bug, the values of which are converted to relative time ("1 hour ago") taking account the user's timezone.
(I've created a custom sorter for this extra column).
By default the results table is sorted by "Modified Date" and "Product:Component" : as a mozilla-inbound user I find this most useful because this way I can see which bugs have been just fixed, and therefore if it's worth downloading the latest m-i build.
You may also comment out line #305 and uncomment #304 in order the results to be sorted by default by "Product:Component" only.
So, my version 5 is here