[EXT] XMLTV Guide and Search

Announce and Discuss the Latest Theme and Extension Releases.
Post Reply
User avatar
Benjamin Markson
Posts: 397
Joined: November 19th, 2011, 3:57 am
Location: en-GB

Re: [EXT] XMLTV Guide and Search

Post by Benjamin Markson »

Yoss1960 wrote:I should have kept the log...
I'm always interested in any errors my add-ons throw. :)

But what I mean are any javascript errors appearing in the web console (ctrl-shift-k) - the console should be started from the add-on's own tab and then the add-on refreshed - or the browser console (ctrl-shift-j) - this one can be a bit overwhelming as it will include errors from all open tabs. The add-on's own log only reports SQL errors and not javascript errors.

Ben.
XUL is dead. Long live the Google Chrome Clones.
User avatar
Benjamin Markson
Posts: 397
Joined: November 19th, 2011, 3:57 am
Location: en-GB

Re: [EXT] XMLTV Guide and Search

Post by Benjamin Markson »

Okay, so the paint was barely dry on v1.0.8 and now here's v1.0.9 with yet another rewrite of the import. The new effort is a lot gentler. It no longer reads a complete image of the xmltv.xml file in one hit. The file itself is now being read and processed in smaller, more friendly, 200k chunks.

The net result is that we should be able to import files of any size without either consuming huge amounts of memory, or excessively lagging the browser. There is, however, a price to be paid. Importing the full Freeview lineup on my ageing Pentium 4, with 2 gig of memory, machine took 10 (now lag free) minutes. The Core 2, with 4 gig of memory, comes in at 3 minutes. Basically, computers that are less capable will take their time.

The more modest sized file that I routinely import on my Core 2 machine takes under... 15 seconds. So, I still recommend people steer clear of unnecessarily huge files. :)

It appears that this version is not currently supported by Pale Moon, and it also won't be supported by much older versions of Firefox. I hope this isn't going to a deal breaker.

Ben.
XUL is dead. Long live the Google Chrome Clones.
User avatar
Yoss1960
Posts: 119
Joined: January 6th, 2013, 11:23 am

Re: [EXT] XMLTV Guide and Search

Post by Yoss1960 »

I'm always interested in any errors my add-ons throw. :)

Okay, you asked for it......

I reinstalled v1.0.8, loaded everything up, called the JS console, cleared it, then tried an import. Here's the resulting long form the console (all one line of it):

Code: Select all

[20:09:14.983] InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable @ chrome://xmltvguide/content/grabber.js:250
Clicking on the link opened grabber.js with the following line highlighted:

Code: Select all

		req.responseType = "blob";
You'll not be surprised to know it's line 250. :-)
Regards,
¥ø$$1960
"The blindingly obvious is never immediately apparent"
User avatar
Yoss1960
Posts: 119
Joined: January 6th, 2013, 11:23 am

Re: [EXT] XMLTV Guide and Search

Post by Yoss1960 »

Bah!

"long form" should read "log from", natch.
Regards,
¥ø$$1960
"The blindingly obvious is never immediately apparent"
User avatar
Benjamin Markson
Posts: 397
Joined: November 19th, 2011, 3:57 am
Location: en-GB

Re: [EXT] XMLTV Guide and Search

Post by Benjamin Markson »

Yoss1960 wrote:Okay, you asked for it......
Oh no! You've discovered... The Blob - INDESCRIBABLE... INDESTRUCTIBLE! NOTHING CAN STOP IT!

I'm going to guess it has something to do with this: https://bugzilla.mozilla.org/show_bug.cgi?id=1110761
Nathan Vander Wilt 2015-02-13 10:48:58 PST wrote:The workaround is described in e.g. http://forums.udacity.com/questions/100 ... sponsetype — do not try to set `.responseType` before calling `.open()` lest you hit this out-of-spec behavior.
Ming-Chou Shih [:stone] 2016-03-08 19:59:10 PST wrote: It seems this bug was resolved by https://hg.mozilla.org/mozilla-central/ ... 69c8#l1.38
I verified the test case responsetype.html with my local build and it passed.
So, back in FF34 it was an error, but come FF38 it was fixed. Maybe Pale Moon is still using the older behaviour?

It suggests that if I had:

Code: Select all

var req = new XMLHttpRequest();
req.addEventListener("loadend", blobHandler, false);
req.addEventListener("error", blobHandler, false);
req.open("GET", fileURI.spec);
req.responseType = "blob";
req.send();
Instead of:

Code: Select all

var req = new XMLHttpRequest();
req.addEventListener("loadend", blobHandler, false);
req.addEventListener("error", blobHandler, false);
req.responseType = "blob";
req.open("GET", fileURI.spec);
req.send();
...then it would have worked in either case. I think this just goes to underline what a minefield this game can be. :shock:

Ben.
XUL is dead. Long live the Google Chrome Clones.
User avatar
Yoss1960
Posts: 119
Joined: January 6th, 2013, 11:23 am

Re: [EXT] XMLTV Guide and Search

Post by Yoss1960 »

So, back in FF34 it was an error, but come FF38 it was fixed. Maybe Pale Moon is still using the older behaviour?
It would seem so. OTOH, as time goes by, the common ground between PM & FF is decreasing. For example, PM uses the Goanna engine, not Gecko. I know that the PM people are trying to maintain compatibility, but they'll forever be playing 'catch up'. Pretty much in the same way AV programmers are always one step behind the virus writers. Not that I'm complaining about it, it's simply a fact of life.

On the plus side, your suggested code change means I now have a tweaked version of 1.0.8 working here. =D>
Regards,
¥ø$$1960
"The blindingly obvious is never immediately apparent"
User avatar
Benjamin Markson
Posts: 397
Joined: November 19th, 2011, 3:57 am
Location: en-GB

Re: [EXT] XMLTV Guide and Search

Post by Benjamin Markson »

Yoss1960 wrote:...as time goes by, the common ground between PM & FF is decreasing. For example, PM uses the Goanna engine, not Gecko ...your suggested code change means I now have a tweaked version of 1.0.8 working here.
I think the issue is more about the ECMAScript standard supported by the browser - where Javascript (and other scripting languages) track this standard. According to this Goanna uses JavaScript 1.8.5 (full ECMAScript 5.1 support, partial ECMAScript 6 support)

I just wish I could find a similarly silver bullet for v1.0.9 but I'm afraid I don't think that is possible as Goanna doesn't support the relevant ECMAScript 6 standard - in this case generators. There are probably other ways to skin this cat but I'm already at the very edge of my javascript knowledge on this one.

Incidently, your User Agent reports both Gecko and Goanna, and both Firefox 38.9 and Palemoon 26.3.3 - what a tangled web we browse. :p

Ben.
XUL is dead. Long live the Google Chrome Clones.
User avatar
Yoss1960
Posts: 119
Joined: January 6th, 2013, 11:23 am

Re: [EXT] XMLTV Guide and Search

Post by Yoss1960 »

I just wish I could find a similarly silver bullet for v1.0.9 but I'm afraid I don't think that is possible as Goanna doesn't support the relevant ECMAScript 6 standard - in this case generators. There are probably other ways to skin this cat but I'm already at the very edge of my javascript knowledge on this one.
I'm happy; I've got a working version of XMLTVGuide. Okay, it's not the latest but it's not far behind and in any case, you're doing this as a hobby, not to earn a living. I'm sure (well, hopeful) that PM will get the relevant abilities sooner or later.
Incidently, your User Agent reports both Gecko and Goanna, and both Firefox 38.9 and Palemoon 26.3.3 - what a tangled web we browse. :p
Yes, I'm aware. It's part of a contorted route I have to tread in an attempt to try and con web sites (put together by lazy programmers[1]) that my browser is capable of doing anything other than HTML.

As always, I'm very grateful for your work on this extension.

[1] A lazy way of determining a browser's capabilities is to ask for the version string and make assumptions based on that, rather than ask the browser what it can actually do. If I didn't spoof the version string, many banking (and several other) web sites simply wouldn't work.
Regards,
¥ø$$1960
"The blindingly obvious is never immediately apparent"
User avatar
Yoss1960
Posts: 119
Joined: January 6th, 2013, 11:23 am

Re: [EXT] XMLTV Guide and Search

Post by Yoss1960 »

Today I discovered yet another Firefox variant - Cyberfox(1). I'm still evaluating it for my personal use, but I can confirm that XMLTVGuide v1.0.9 works fine therein.

(1) I'd heard of a few, but this one had passed me by until today.
Regards,
¥ø$$1960
"The blindingly obvious is never immediately apparent"
User avatar
Benjamin Markson
Posts: 397
Joined: November 19th, 2011, 3:57 am
Location: en-GB

Re: [EXT] XMLTV Guide and Search

Post by Benjamin Markson »

Yoss1960 wrote:Today I discovered yet another Firefox variant - Cyberfox...
What is it that you are trying to achieve? My Firefox (v48) still looks and works pretty much the same as it did when it was Firefox2 - this does require several UI-altering add-ons.

Image

Mainly using:
Stylish [has many uses beyond altering the FF chrome]
Classic Theme Restorer
Tab Mix Plus
Status-4-Evar
[plus the hack to allow unsigned add-ons whose details have been removed by the mozillaZine mods]

...and also:
Downloads Window
Check for Updates in Help Menu
Classic Pop-up Alerts

Ben.
XUL is dead. Long live the Google Chrome Clones.
User avatar
Yoss1960
Posts: 119
Joined: January 6th, 2013, 11:23 am

Re: [EXT] XMLTV Guide and Search

Post by Yoss1960 »

What is it that you are trying to achieve?

Since this has nothing to do with XMLTVGuide, I've responded via PM.
Regards,
¥ø$$1960
"The blindingly obvious is never immediately apparent"
Mikhel
New Member
Posts: 1
Joined: February 14th, 2017, 12:36 am

Re: [EXT] XMLTV Guide and Search

Post by Mikhel »

Hi, I just began to use XMLTVGuide, I have a small XML file created with WebGrab++ which consists of about 20 channels. I am in México so a lot of the descriptions have characters like: á. é, í, ó, ú, ñ and their capital counterparts, so when I load the XML in XMLTVGuide it shows as garbled because it seems it does not support UTF-8 and instead the XML file needs to be saved as ISO-8859-1, I can do it manually using an XML editor but it would be cool for the add-on to read UTF-8.

A second issue I found is that even if I tell WebGrab++ to offset the list I download, XMLTVGuide doesn't seem to apply it properly.

The XML file has an entry like this one:

<programme start="20170214074500 -0800" stop="20170214092500 -0800" channel="01_Pánico">
<title lang="es">El diario de los muertos
(had to remove the closing argument of the TITLE tag because the forum didn't allow me to post it).

So even though it says: 20170214074500 -0800 ---> 07:45 am and an offset of -8 hours, XMLTVGuide shows it at 7:45 am.

Since the data is pulled from a website with a default -06 GMT and I want them in -8 GMT I thought of replacing the -0800 instruction with a -0200 but it did nothing (that's when I knew XMLTVGuide was not interpreting the time offset instruction). I do not know if there could be a way to add:

a) A way to manually offset a channel or all channels to the amount of hours or minutes we want to.
b) A way to chance and increase the size of the fonts (too small for my dad, which is for whom I am setting up the add-on).
c) A way to increase the size of the cells.
d) As of right now it only recognizes "drama" and "other" as genres, don't know if there's a way to add other genres or if it doesn't recognize those I have in my EPG data because they are in spanish:

Action = Acción or Accion
Suspense = Suspenso
Adventure = Aventura
Biography = Biográfica or Biografica or Biografía or Biografia
Terror = Terror
Horror = Horror
Drama = Drama

Some are the same but not even those appear (I wrote the different ways a word can be written since some websites do not use written accent like --> Á and A).

And that's basically it, the add-on is very very good even with the things I have found. If you want the file with the EPG data I can upload it to my google drive account and share you the link to download it.

Cheers.

Mike.
User avatar
Benjamin Markson
Posts: 397
Joined: November 19th, 2011, 3:57 am
Location: en-GB

Re: [EXT] XMLTV Guide and Search

Post by Benjamin Markson »

Hi Mike.

<rant> One thing worth noting is that this add-on will no longer work come Mozilla's Great XUL Purge (around FF57?) as these new 'modern' webextensions won't be supporting 'proper' relational databases (in this case SQLite) - besides which, attempting to translate all the complex grids and tables from XUL to HTML would be a nightmare. XUL is, and has always been, a superior mark-up language to HTML[5].</rant>

Anyway...
Mikhel wrote:...does not support UTF-8 and instead the XML file needs to be saved as ISO-8859-1, I can do it manually using an XML editor but it would be cool for the add-on to read UTF-8.
The import currently assumes ISO-8859-1 for no reason other than that mc2xml.exe defaults to that standard - with mc2xml there is a switch (-U) that can be used to change it to UTF-8. Does WebGrab+Plus have a similar choice to switch to ISO-8859-1?
Mikhel wrote:a) A way to manually offset a channel or all channels to the amount of hours or minutes we want to.
I may have this horribly wrong but doesn't the offset value indicate the amount by which the timestamp has been adjusted from UTC - so, in your example, it means that the 07:45 is 8 hours behind UTC. Therefore, arbitrarily changing just the offset value is meaningless? Basically, my add-on takes no notice whatsoever of the offset and simply assumes that the timestamp value is the local time.

Now, I'm not saying that it wouldn't be possible to adjust the timestamp using the offset value back to plain UTC, and then readjust it by a new offset value - but that would be a Really Big Ask. I think you need to get the original XMLTV file with the correct offset applied for your local time.
Mikhel wrote:b) A way to chance and increase the size of the fonts (too small for my dad, which is for whom I am setting up the add-on).
c) A way to increase the size of the cells.
The sizing can be adjusted using conventional Firefox zoom controls: View... Zoom... Zoom In (Ctrl++), Zoom Out (Ctrl+-), Reset (Ctrl+0).
Mikhel wrote:d) As of right now it only recognizes "drama" and "other" as genres, don't know if there's a way to add other genres or if it doesn't recognize those I have in my EPG data because they are in spanish:
The genres is an interesting one! The are in fact two genres, a Main, and a Sub, genre. The sub genres are the genres given by the XML file you import, whereas the main genres are a contrivance of the add-on. I created the list of main genres based on analysing tens of thousands of (English language) sub genres in a spreadsheet and picking out the most common. This is My list:

Code: Select all

['Children', 'Soap', 'Sitcom', 'Drama', 'Documentary', 'Interests', 'Arts & Crafts', 'Music & Arts', 'News', 'Sports', 'Game Show', 'Entertainment', 'Comedy', 'Crime Drama', 'History', 'Cooking', 'House & Garden', 'Reality']
...the front of the list has a higher precedence. A sub genre of 'Film' or 'Movie' makes the programme become a 'Film'. If a programme doesn't have any of these it is given a main genre of 'Other'. You should find that on the Search tab that the Genres drop-down has all the unidentified sub genres under the guise of 'Other'. I would suggest that the most important thing is that 'Film' is being recognised correctly. Without this the add-on will lose a lot of its functionality.
Mikhel wrote:And that's basically it, the add-on is very very good even with the things I have found.
I don't know how confident you are with changing add-ons - by that I mean unpacking them, changing their code, and repackaging them. The UTF-8, and genres would be trivial for you to change. See: grabber.js, line 265 for the UTF-8 thing, line 450 for the main genres, and line 508 for identifying films.

I think that adjusting the timestamp values goes beyond the scope of what is essentially a viewer. The zoom controls are already a part of Firefox.

Cheers,
Ben.
XUL is dead. Long live the Google Chrome Clones.
colin1951uk
Posts: 69
Joined: November 17th, 2014, 4:52 am

Re: [EXT] XMLTV Guide and Search

Post by colin1951uk »

Hello All,

Long time no see!

Can somebody check something for me please?

Since yesterday I've noticed that mc2xml has stopped grabbing listings for Movies4Men (channel id="I48.46306.schedulesdirect.org)

It picks up the channel id ok but I'm not getting any listings downloaded.

I use line up: GBR-0001225-DEFAULT Youview-Meridian (South) for my grab but if it is down I would imagine all line ups will be be down.

Can anyone tell me if they're getting listings for their lineup(s) please?

Cheers...Colin
User avatar
Yoss1960
Posts: 119
Joined: January 6th, 2013, 11:23 am

Re: [EXT] XMLTV Guide and Search

Post by Yoss1960 »

Different TV region, but same here. Also, at least one other is missing (11 PickTV). May be others, not fully checked yet.
Regards,
¥ø$$1960
"The blindingly obvious is never immediately apparent"
Post Reply