read http headers & print subsets of it

Talk about add-ons and extension development.
Post Reply
n00b_k
Posts: 17
Joined: August 3rd, 2007, 10:40 am

read http headers & print subsets of it

Post by n00b_k »

Hi
I am trying to write an extension that reads in HTTP headers in real time (from activity of the person using Firefox). I do not need to modify or send any header requests..

I have gone over the LiveHTTPheaders plug in. I know this prints out all HTTP header information in real time, but due to the complexity of the code, I prefer to write what I need from scratch.

Like if someone performs a search on imdb, I want to be able to "see" or extract this from the headers:
http://www.imdb.com/find?s=all&q=jane

This is a very specific example, but I present it to explain that I want to be able to read the header information while someone is using FireFox, and using conditional statements on what I read, print that out.

I came across: XMLHttpRequest and getAllResponseHeaders but the examples were used for IE.

Please let me know what classes or functions I could use.

Thanks
einare
Posts: 159
Joined: October 22nd, 2006, 1:16 pm
Location: Iceland
Contact:

Post by einare »

Check out this tutorial: http://developer.mozilla.org/en/docs/Se ... st_headers

When you've got the nsIHttpChannel object you can call GetRequestHeader on it to get a specific header, or use the properties URI or originalURI to get the uri being requested.
Try my javascript card games, Hearts, Idiot and Crazy Eights
n00b_k
Posts: 17
Joined: August 3rd, 2007, 10:40 am

Post by n00b_k »

Thanks for the link. Here is what I tried:

Code: Select all

var httpChannel = subject.QueryInterface(Components.interfaces.nsIHttpChannel);
alert(httpChannel.originalURI);
alert(httpChannel.referrer);

I was trying to get it to print to an alert box for now. But all my code does not run, even the code that appears before these lines, so I am sure it is incorrect and I am missing something major.

I also saw how to use getRequestHeader:

Code: Select all

getRequestHeader: function(header){
        return this.httpChannel.getRequestHeader(header);
}


But I am not sure what to pass as a header object.

I get the general idea from the link. Perhaps if anyone can show me a link that has a running example.

Thanks a lot.
einare
Posts: 159
Joined: October 22nd, 2006, 1:16 pm
Location: Iceland
Contact:

Post by einare »

Here's a sample of a complete .js file, include it in your overlay and try it. It will alert the uri and user-agent header of every request.

Code: Select all


//Object to wrap our functions
var HeaderAlert = {


    //This is called when the window has finished loading
    onLoad : function(loadEvent) {
        var observerService = Components.classes["@mozilla.org/observer-service;1"]
                                .getService(Components.interfaces.nsIObserverService);
        observerService.addObserver(this, "http-on-modify-request", false);
    },

    //Called when this Mozilla window is closed
    onUnload : function(loadEvent) {
        var observerService = Components.classes["@mozilla.org/observer-service;1"]
                                .getService(Components.interfaces.nsIObserverService);
        observerService.removeObserver(httpRequestObserver, "http-on-modify-request");
    },

    //This function implements the nsIObserver interface
    observe : function(aSubject, aTopic, aData) {
        if (aTopic == "http-on-modify-request") {
            var httpChannel = aSubject.QueryInterface(Components.interfaces.nsIHttpChannel);
            alert("URI: " + httpChannel.originalURI.spec);
            alert("User-Agent: " + httpChannel.getRequestHeader('user-agent'));
        }
    }

};


//This is run when a new mozilla window loads
window.addEventListener("load", function(event) { HeaderAlert.onLoad(event); }, false);
window.addEventListener("unload", function(event) { HeaderAlert.onUnload(event); }, false);


The stuff from the article I linked to still applies, you should probably make a component so this isn't done for every window, but this should get you started.
Try my javascript card games, Hearts, Idiot and Crazy Eights
n00b_k
Posts: 17
Joined: August 3rd, 2007, 10:40 am

Post by n00b_k »

Thanks a lot. this did get me started and I was able to do some things. I have one question

Code: Select all

alert("Date: " + httpChannel.getRequestHeader('date'));


I am trying to get the Date header field but the above line of code does not occur at all. Is there another way to get the date/time stamp of the header or do I have a mistake in the line?


einare wrote:Here's a sample of a complete .js file, include it in your overlay and try it. It will alert the uri and user-agent header of every request.

Code: Select all



    //This function implements the nsIObserver interface
    observe : function(aSubject, aTopic, aData) {
        if (aTopic == "http-on-modify-request") {
            var httpChannel = aSubject.QueryInterface(Components.interfaces.nsIHttpChannel);
            alert("URI: " + httpChannel.originalURI.spec);
            alert("User-Agent: " + httpChannel.getRequestHeader('user-agent'));
        }
    }

};


einare
Posts: 159
Joined: October 22nd, 2006, 1:16 pm
Location: Iceland
Contact:

Post by einare »

The reason you don't get the date header with this is that this code is only examining the headers in requests, not respones. Requests dont have a Date header, it's the response from the remote server that has the Date header. If you also want to examine response headers you need to do the following:

1. Add this line in the onLoad function

Code: Select all

        observerService.addObserver(this, "http-on-examine-response", false); 


2. Change the observe function to this:

Code: Select all

//This function implements the nsIObserver interface
    observe : function(aSubject, aTopic, aData) {
        if (aTopic == "http-on-modify-request") {
            var httpChannel = aSubject.QueryInterface(Components.interfaces.nsIHttpChannel);
            alert("URI: " + httpChannel.originalURI.spec);
            alert("User-Agent: " + httpChannel.getRequestHeader('user-agent'));
        }
        else if (aTopic == "http-on-examine-response") {
            var httpChannel = aSubject.QueryInterface(Components.interfaces.nsIHttpChannel);
            alert("Date: " + httpChannel.getResponseHeader('date')); //note that here it's called getResponseHeader, but in the other place getRequestHeader
        }
    }
Try my javascript card games, Hearts, Idiot and Crazy Eights
n00b_k
Posts: 17
Joined: August 3rd, 2007, 10:40 am

Post by n00b_k »

Thanks, that worked for me.
n00b_k
Posts: 17
Joined: August 3rd, 2007, 10:40 am

Post by n00b_k »

Have a question regarding this code.

First on how the extension works. when installed, there is a menu item under "Tools" when user selects it, a new tab in firefox opens showing the relevant XUL file.

SO the code works, where it extracts the text I need from the HTTP headers and writes it to a file. When I close the tab, this stops. Sometimes, even when the tab is still opened, the code is not capturing the http packets and writing them to the file so I have to start the extension again (going into Tools and selecting the menu item).

Code: Select all

//This is run when a new mozilla window loads
window.addEventListener("load", function(event) { HeaderAlert.onLoad(event); }, false);
window.addEventListener("unload", function(event) { HeaderAlert.onUnload(event); }, false);


Understanding this might help me with the problem. I know this means to "wake up" when a window is open (I assume a new TAB as well because the code has worked with HTTP listening on individual TABs). So the load function adds the observer, and on unload it removes the observer.

What I am trying to figure out is, what would make the observer stop besides a TAB closing or the XUL TAB closing (the one opened when the menu item is selected).

Thanks
einare
Posts: 159
Joined: October 22nd, 2006, 1:16 pm
Location: Iceland
Contact:

Post by einare »

Where do you have this javascript? In the overlay for your own xul file? If so, it will be unloaded when you close your own window. I´m not really sure why it would stop working if you open or close tabs, it shouldn't happen, only when windows are opened or closed. If you give me some more details I might be able to figure it out.
Try my javascript card games, Hearts, Idiot and Crazy Eights
n00b_k
Posts: 17
Joined: August 3rd, 2007, 10:40 am

Post by n00b_k »

einare wrote:Where do you have this javascript? In the overlay for your own xul file? If so, it will be unloaded when you close your own window. I´m not really sure why it would stop working if you open or close tabs, it shouldn't happen, only when windows are opened or closed. If you give me some more details I might be able to figure it out.


The first xul file loaded creates the menu item, and the new tab (which basically is FF opening the 2nd XUL file into a FF tab). Then the 2nd XUL file creates the buttons on the tab and a tree. A javascript file is referenced in this 2nd XUL file. That Javascript file contains the functions for listening to the HTTP packets and creating the tree based on when those buttons are clicked (from the tab page).

From what I notice, the listener is listening and writing the HTTP data when this tab is opened. When it is closed, then it stops. IT will not start again unless the tab is reopened by selecting this menu item.
n00b_k
Posts: 17
Joined: August 3rd, 2007, 10:40 am

Post by n00b_k »

einare wrote:Where do you have this javascript? In the overlay for your own xul file? If so, it will be unloaded when you close your own window. I´m not really sure why it would stop working if you open or close tabs, it shouldn't happen, only when windows are opened or closed. If you give me some more details I might be able to figure it out.


The first xul file loaded creates the menu item, and the new tab (which basically is FF opening the 2nd XUL file into a FF tab). Then the 2nd XUL file creates the buttons on the tab and a tree. A javascript file is referenced in this 2nd XUL file. That Javascript file contains the functions for listening to the HTTP packets and creating the tree based on when those buttons are clicked (from the tab page).

From what I notice, the listener is listening and writing the HTTP data when this tab is opened. When it is closed, then it stops. IT will not start again unless the tab is reopened by selecting this menu item.
einare
Posts: 159
Joined: October 22nd, 2006, 1:16 pm
Location: Iceland
Contact:

Post by einare »

Ah, ok, there's your problem. Since you're loading your xul files in the tab, then everything is unloaded when you close the tab. You could try not removing the listener in the unload function, then it'll remain active even after you close the tab. Then you've got to watch out however, if someone opens that tab again you might end up with two listeners. Or you could do this:

1. Have the listener in a javascript file that's referenced in an overlay file for the browser.xul window, the main window. Then it'll keep listening while firefox is open.
2. Make your new tab just switch on some preference, something like extensions.httplistener.isactive = true.
3. The listener only writes to the file when the pref is set to true.

or something like that. The main issue is that your xul file is unloaded when you destroy the tab. If you do it in an overlay instead you won't have that problem.
Try my javascript card games, Hearts, Idiot and Crazy Eights
n00b_k
Posts: 17
Joined: August 3rd, 2007, 10:40 am

Post by n00b_k »

Hi

I had to put this off for a while, but I wanted to write and say that this worked. I just took out the code for creating the file and listeners and pasted into another Javascript file, included this in my overlay file and it was fine. Thanks
rr100
Posts: 1
Joined: November 20th, 2007, 11:20 am

how to automatically parse every URL that's loaded

Post by rr100 »

Hi,

I am building an extension with a single button on it. When certain sites are visited, I want the button to be disabled. To do this, I need to catch every URL that shows up in the URL bar and parse it. How do I do this automatically i.e. without needing the button to be clicked?

I am an extension newbie. Thanks for your help in advance.
deatul
Posts: 3
Joined: January 26th, 2010, 12:48 am

Re:

Post by deatul »

Hello einare,
Your this post has created lots of hope in me, I am building an extension which read all the HTTP-requests and modifies it according to referer & POST data. With the help of your code, I am able to read & write HTTP headers, but am stuck with reading & writing POST data.
I would be very great full if you may guide me.
Atul Gupta

einare wrote:Here's a sample of a complete .js file, include it in your overlay and try it. It will alert the uri and user-agent header of every request.

Code: Select all


//Object to wrap our functions
var HeaderAlert = {


    //This is called when the window has finished loading
    onLoad : function(loadEvent) {
        var observerService = Components.classes["@mozilla.org/observer-service;1"]
                                .getService(Components.interfaces.nsIObserverService);
        observerService.addObserver(this, "http-on-modify-request", false);
    },

    //Called when this Mozilla window is closed
    onUnload : function(loadEvent) {
        var observerService = Components.classes["@mozilla.org/observer-service;1"]
                                .getService(Components.interfaces.nsIObserverService);
        observerService.removeObserver(httpRequestObserver, "http-on-modify-request");
    },

    //This function implements the nsIObserver interface
    observe : function(aSubject, aTopic, aData) {
        if (aTopic == "http-on-modify-request") {
            var httpChannel = aSubject.QueryInterface(Components.interfaces.nsIHttpChannel);
            alert("URI: " + httpChannel.originalURI.spec);
            alert("User-Agent: " + httpChannel.getRequestHeader('user-agent'));
        }
    }

};


//This is run when a new mozilla window loads
window.addEventListener("load", function(event) { HeaderAlert.onLoad(event); }, false);
window.addEventListener("unload", function(event) { HeaderAlert.onUnload(event); }, false);


The stuff from the article I linked to still applies, you should probably make a component so this isn't done for every window, but this should get you started.
Post Reply