Learning to write extensions: the epic journey

Talk about add-ons and extension development.
Dewi Morgan
Posts: 8
Joined: October 1st, 2005, 8:35 am
Location: London, UK
Contact:

Learning to write extensions: the epic journey

Post by Dewi Morgan »

So I thought I'd learn to write a Thunderbird extension, driven by having found a couple of free email-to-fax gateways, and thinking that their email address syntax was a ruddy nightmare. I decided that if it took more than a day, I would give it up as a bad job.

Now, I don't know squat about this. I know javascript, but all I know about XUL is "the joke". I write web pages sometimes. Possibly it's incredibly arrogant of me to expect to be able to come along and learn how to write an extension in a day.

But that's what Firefox and Thunderbird are marketed as: slimline but extensible. If I can make me an extension for faxing stuff from thunderbird, then I'll agree that the bar is set suitably low.

So I decided to write about my learning process, as I go, in the hopes that the documentation people might find some useful hints from my confusion, about ways to try to improve things.

The gateways I'll be writing the extensions for, if I ever get that far, are:
http://www.tpc.int/
http://www-it.desy.de/support/help/uco_ ... ax.html.en


Now, where do we start? Oh, probably Google. "making thunderbird extensions"
That takes me to here:
http://www.mozilla.org/projects/thunder ... sions.html
Which has a link to a "mozilla.org tutorial" here:
http://www.mozilla.org/docs/tutorials/tinderstatus/

OK, so I have to extract a bunch of xpifiles in order to write an extension. Uh... where do I find them?
Well it says they're in the chrome directory, so let's search for that... oh, I've 50 directories called "chrome".

Wait. This page is awful.

On operating systems with DOS-like shells, the following command accomplishes this task:
for %file in (*.jar); do unzip %file
Note that there are platform-specific files--en-mac.jar, en-unix.jar, and en-win.jar--in that directory. Extract only the one appropriate for your platform.


Blithering useless contrariness, mixing esoteric for-loopiness with an exortation not to use it because it'll extract the wrong files. No thank you sir, I shall find my tutorials elsewhere.


So I Google again: "thunderbird extensions tutorial".

Hrm, every single extension plugin tutorial appears to demonstrate how to put a hello world popup hanging off a menu item. Why is that? Does everyone want their extensions to do this? I don't have ANY extensions that do this.

Still, I find a page about extensions:

http://kb.mozillazine.org/Dev_:_Extensions
Looks good, it's a wiki, claims to be part of some "extension development documentation project".

That page tells me to read the page about setting up the extension environment.

The dev-environment page gives me some settings, and tells me to read a page about how to set environment settings, which in turn tells me to use about:config to set them.

Back on the dev-environment page, it then gives the same instructions to extract the xpi files, again without saying where on earth they might be located. This time, it just lists them as a bunch of *nix commands, some perl, and the instructions "(Windows users can use Cygwin to run these commands)"... well, I see red there and then. I'm sorry, but if these are the official docs, then clearly they were written by people with no clue, or a great desire to make the hurdle for developing extensions as high as possible. "If you wish to accomplish task X, install Perl, and Cygwin. Now you have three problems."

Anyway, looks like that section is for earlier versions of Firefox and Thunderbird. There are instructions for the later one:
"Simply edit the chrome.manifest the same way as described above"... except, chrome.manifest is not mentioned anywhere else on the page, so that's a dead loss right there.

Thankfully though, this is a wiki, so I can submit a change deleting the retarded cygwin plug, and providing easy commands to do all that and better in Windows... oh wait, editing requires a registered login, and the wiki login/registration page says "register on the form below", but there is no registration form. So much for the wiki.

I find these forums instead, and start writing this post.


OK, that was an irritating little detour, but I've got the dev environment vars set up, and extracting those xpi files was in the "Non-essential" section anyway, so I carry on my merry way back at the Dev:extensions page.

Now that tells me to read the "getting started with extension development" page. Okeydoke, time for another detour.


Looks like this "getting started" page is for 1.5 and later. Fair enough, it makes it nice and clear right at the top. I'm good with that. I've got me Thunderbird 1.5, so I'm good to go.

Heh it's another one of those "hello world popup from a menu" tutorials. Fair enough.

Sweet, there's a stub zipfile I can download, with all those little required files already set up, with the right folder structure. *save to desktop*.

Oh wait, it says that's the "development structure", and to package it I'll need to put it in a different layout, the "package structure", with a link to yet another page... nah, screw that, I just want to get into developing my extension! I've been at this an hour and not even got hello world working! What kind of crazy development environment is this?

Now onto the syntax. Yay, finally, into the meat!
Yeah, that all looks fairly self-explanatory.

And it even gives me a link to a "profile folder" page, answering the question I had back when they first started talking about extracting stuff in it: where the smeg is it?

It turns out it's in a hidden, unsearchable folder, located randomly outside the application directory. Cool. Intuitive.

Now it doesn't say so anywhere in this tutorial, but I'm guessing that line 8 of overlay.xul there has the name of the XUL element that it's going to verlay. And I guess that's defined in some xul file somewhere in the browser, whether in the chrome or the application directory, or embedded in some xpi/jar file somewhere, who knows?

So. I want to add a "Fax:" label to the dropdown list of mail targets in thunderbird. That should then override the behaviour of the ajacent textbox to validate a fax number, and fill out any missing parts of the address. It shoudl pull the fax number from the addressbook if only a name is given.

So to my uneducated eyes, it seems I need to find the names of the chrome elements that I want to overlay: that dropdown, and the ajacent address textbox.

To do that, I need to find and read all xul files in the profile and the application directory of Thunderbird, whether they're packaged in xpi or jar files or not. Hrm. There MUST be a more sensible way of doing that.

A quick rummage finds me a listing of chrome URLs, which says "To overlay an existing XUL element, you first need the URL of it's chrome." (learn how to write your posessives: it's "its", when it's posessive):
http://kb.mozillazine.org/Chrome_URLs
Which is great for firefox and mozilla, useless for thunderbird.

The original crappy tutorial at least suggests using the DOM inspector, so let's try that!
Now, where can I get a DOM inspector that works on thunderbird 1.5?
https://addons.mozilla.org/thunderbird/1806/ apparently.

Yup, that works. We're cooking with gas now! So I open up the inspector from "tools -> dom inspector", click to compose a new email, click "file -> inspect a window" in the dom thingus, and select the composing email from the list.

Sweet, a DOM tree. Now how do I find the elements I want in that tree? Ooh, selecting elements in the tree makes them flash a red border on the window. That's positively sexy.

So I drill down: window -> hbox -> vbox -> toolbox(toolbar-top:headers-box) -> toolbar(MsgHeadersToolbar) -> hbox(msgheaderstoolbar-box) -> vbox(addresses-box) -> listbox(addressingWidget) -> xul:listrows (this one's in red, and it stops giving the cool flashy border hints here... but from here I can find my own way I think) -> xul:listboxbody -> listitem -> listcell -> menulist(addressCol1#1) -> xul:label.

So I've found my dropdown list.

Next to it is a textbox that probably holds the address.

The addressbar of the inspector reads:
chrome://messenger/content/messengercompose/messengercompose.xul

I guess that's where I need to look to find names to override.

But I've been at this for four hours, so it's time to go out for a kebab.
Yet another Firefox user.
old np
Posts: 0
Joined: December 31st, 1969, 5:00 pm

Post by old np »

Yeah, it's kinda hard to get started.
Dewi Morgan
Posts: 8
Joined: October 1st, 2005, 8:35 am
Location: London, UK
Contact:

Post by Dewi Morgan »

Yup. Needlessly, pointlessly hard, since the information I've required so far was spread out over a silly number of web pages, only about half of them linked together by anything other than google, when it could fairly easily have been gathered in one place, or even into an IDE.

I'm pretty scathing, but mostly that's for humour value. Fact is, the tutorials out there are actually pretty darn well written.

They aren't as scalable as I'd like, since they don't explain why they picked a value that they did, or how I should pick a value for my own plugin: instead, they focus on their own task (a hello world popup). But the information they /do/ give (at least in the two I linked to above) is really rather well written and thorough.

But they're aimed, generally, at the stuff that is new to the author, rather than what's new to the community. So XUL knowledge is taken as read, as is an installation of *nix: these, however are not generally things that the average mozilla, thunderbird or firefox user would have.

What hurdles did you encounter when starting out? Are they still there? Does it bug you, not having an API? Or are you accustomed now to using vi to create a bunch of human-unfriendly index files, because the system is too stupid to just look in the standard places for the standard files? Does it bug you, not being able to rightclick a specific component and edit its behaviour? Does it bug you, not having a reference all in one place with a searchable index?

WELL IT BUGS THE SNOT OUT OF ME!

OK, lunchtime is over. Time to tackle this again.
Yet another Firefox user.
old zeniko
Posts: 0
Joined: December 31st, 1969, 5:00 pm

Post by old zeniko »

Welcome to the world of open source software, where not only code depends on volunteers, but documentation as well. :)

Anyway, XULPlanet is a very good source of Mozilla API documentation and the Mozilla Cross-Reference usually pretty soon gets your friend (use the source, Luke - but seriously).

Finally, they are working on making the Mozilla Developer Center the one place for Mozilla information, unifying the MozillaZine Knowledge Base, XULPlanet and some more resources found in the web. But being short on contributors, it's going to take it's time (interestingly, there are far fewer individuals ready to provide documentation than those who provide code).

Oh, and most of the documentation works flawlessly under Windows as well. If you don't insist on packing the chrome files into it's own JAR (which isn't required - you can distribute your development environment as is), Notepad and the ZIP-Folder "Send To ..." target are enough.

Just the IDE might remain a dream for some more time - in the meantime, have a look at the Extension Developer's Extension.
User avatar
Pike
Posts: 2293
Joined: August 10th, 2003, 12:12 pm
Location: UK
Contact:

Post by Pike »

The official tutorial is here FWIW (it only covers Fx though):

http://developer.mozilla.org/en/docs/Bu ... _Extension
Dewi Morgan
Posts: 8
Joined: October 1st, 2005, 8:35 am
Location: London, UK
Contact:

Post by Dewi Morgan »

Sooo... when I left off, I'd found where I thought the bits I need to overlay are:

chrome://messenger/content/messengercompose/messengercompose.xul

So I search my drives for "messengercompose". Nothing, of course.
So rightclick the thunderbird icon -> "properties" -> "find target". I'm in a dir called "thunderbird", with a subdir "chrome", which contains a "content.jar" file - looks likely.

Ok, extract that, it creates a messenger dir, with a content subdir... but no "messengercompose" dir within that. There is a messenger subdir, and in that a messengercompose subdir, and in that a messengercompose.xul... it's not the path that the DOM inspector was showing, but it's close enough for a wild guess. I guess wildly, and edit that xul file.

...and then get dragged off for five hours of fixing web pages up after retards. Oh fun fun fun.

So now I'm in a lousy mood.

Oh, hey, posts since I last posted. Coolness, and thanks for the tutorial link, I'll check that out, but maybe tomorrow.
Yet another Firefox user.
old np
Posts: 0
Joined: December 31st, 1969, 5:00 pm

Post by old np »

It's really much easier to DOM Inspect to find what you want than go mucking around in resource files.
User avatar
Philip Chee
Posts: 6475
Joined: March 1st, 2005, 3:03 pm
Contact:

Post by Philip Chee »

Dewi Morgan wrote:So now I'm in a lousy mood.
Once you have got your thunderbird extension up and working I do hope that you might consider updating the knowledgebase wiki with your hard earned experience, if only to spare someone else the horrors you are currently facing.

I started the easy way. I first joined an existing project Flashblock with a working extension and learnt my ropes there.

Once I got the hang of things I started my own extension xSidebar. Of course by then I could simply reuse all the build and packaging scripts that I was using for Flashblock.

Phil
ericjung
Posts: 846
Joined: August 4th, 2003, 9:32 am

Post by ericjung »

Possibly it's incredibly arrogant of me to expect to be able to come along and learn how to write an extension in a day.
Possibly. It sounds like you'd be more comfortable with Greasemonkey for Thunderbird.
User avatar
Philip Chee
Posts: 6475
Joined: March 1st, 2005, 3:03 pm
Contact:

Post by Philip Chee »

grimholtz wrote:Possibly. It sounds like you'd be more comfortable with Greasemonkey for Thunderbird.
Is Greasemonkey available for Thunderbird? I have made a port for SeaMonkey here: greasemonkey-0.6.4.1-mod.xpi but as far as I know nobody has ported GM to Thunderbird.

Phil
old zeniko
Posts: 0
Joined: December 31st, 1969, 5:00 pm

Post by old zeniko »

Philip Chee wrote:Is Greasemonkey available for Thunderbird?

No. userChrome.js is currently probably the closest thing to that functionality.
ericjung
Posts: 846
Joined: August 4th, 2003, 9:32 am

Post by ericjung »

My comment was meant tongue-in-cheek.

Dewi wants to write an extension in one day with his existing JS knowledge. He doesn't realize JS and XUL are merely a means to interface with the Mozilla platform. IOW, he doesn't realize Mozilla is an entire platform. I think a lot of people make the same mistake: they think that since extensions are written in JS, all they need to do is learn XUL and they'll be experts. They don't realize conventional JS knowledge does not include Mozilla-specific platform knowledge; e.g., the XPCOM interfaces. Nor does conventional XML knowledge include things like overlays.

So my comment about Greasemonkey for Thunderbird was similar to saying he wants a 4GL language for extension programming; something for people who don't like to get under the hood.
Dewi Morgan
Posts: 8
Joined: October 1st, 2005, 8:35 am
Location: London, UK
Contact:

Post by Dewi Morgan »

Dang right. Greasemonkey would be an ideal solution.

I'm completely aware of the "it's a platform" issue - but it's deeply irritating that the platform is so hard to modify. I can modify the windows platform by writing and running what we call a "program" on it. I need to know nothing about the kernel architecture for this.

Yhe allegedly-extensible thunderbird doesn't actually have a mechanism for true plugins, as such (call the hooks, don't care about the internals): it just has a way to override the default behaviour with "extensions" that fondle the internals in an unpleasantly intimate, non-codehiding way.

I'd update the wiki, but it's non-editable, as I mentioned in my original thread (assuming I am looking at the right wiki). Anyway, I'm a bad choice for editing it ebcause I am annoyed and I don't like it (though, ye gods, imagine how I'd be if this were outlook I were trying to extend... I'd be spitting teeth!)

I haven't got back to it yet, been too busy with RL.
Yet another Firefox user.
ericjung
Posts: 846
Joined: August 4th, 2003, 9:32 am

Post by ericjung »

I can modify the windows platform by writing and running what we call a "program" on it. I need to know nothing about the kernel architecture for this.

I'm a software developer with over 15 years of Win32 kernel experience (and another 5-10 before that). And with all due respect, what you claim is bullshit. If you want to modify the windows platform, you must be knowledgable about the kernel architecutre. It is my opinion that you cannot be an effective programmer on any platform without learning "under the hood" topics. If you want to be a greasemonkey developer instead of learning the roots of mozilla development, perhaps you should lobby for a greasemoney extension for thunderbird instead of wasting everyone's time here. IOW, extension development is not something you can learn in a day -- nor can you learn win32 kernel development in a day. Set your expectations appropriately.
Dewi Morgan
Posts: 8
Joined: October 1st, 2005, 8:35 am
Location: London, UK
Contact:

Post by Dewi Morgan »

If you want to modify the windows platform, you must be knowledgable about the kernel architecutre.


Au contraire. I did not use the phrase "kernel extension", I used the word "program": with your over 15 years experience, I'm sure I needn't explain the difference, and that this clarification alone is enough. But I shall expand on my point anyway, for the other readers, who may also have missed my point.

In Windows (or any OS), it is trivial to "alter the behaviour of the system" by writing a "program".

Programs (generally speaking) do stuff that the OS does not do by default. Clearly, it was not necessary for every author of every program on your system to know the inner workings of the kernel in order to write their program. So, kernel knowledge is not a prerequisite to extend and alter the behaviour of the system.

Running programs is what operating systems are for. I know next to nothing about the Windows kernel, nor the linux kernel, but I can happily write programs to change the behaviour of either OS, to manipulate windows and create and modify GUIs.

That is my point: An "operating system" is a "system". So to excuse Mozilla for being hard to modify, merely on the basis that it is a "system", is not a valid argument, since we have an example of a far more complex system that is trivial enough to modofy that a sixyearold can do it. But Mozilla/ThunderBird/FireFox are hyped as being easy to extend and modify: and it is clear that this is a lie.

It is trivial to write a program to run in an operating system. The typical "hello world" program is one to four lines long and requires you to merely create a new text file, type in those lines, and possibly to compile it. Knowing the OS kernel is not required. The only code you ever need see, from initially starting work to finishing your program, is your own.

It is nontrivial to write a program for Mozilla. Unlike writing a program to run in an OS, you need to actually "grok the kernel" in Mozilla. Writing a "hello world" program involves crazy hoopjumping and even opening files of someone else's code, and then relying on that undocumented code not to change in future versions. This is not an API, it is not a cool hack, it is not easily-extensible, it is a kludge of the darkest possible kind.

It violates every programming precept by which programmers great and small have stood for the last three decades: code hiding, data hiding, isolation, modularity - all thrown out of the window in one fell swoop.

It's like the difference between writing a program to play tetris, and hacking the kernel to do it. One is the Right Way, the other is an Insane Kludge That Will Probably Break Your System.

...I hope that clarifies my point.

[And despite the above rant, yes, I do intend, eventually, to get my head around it and write the extension. But having a whole day free for such an endeavour is a rare and precious thing for me. It may take some time. Hopefully before the last fax machine leaves this earth, though.]
Yet another Firefox user.
Locked