best way to read/write XML?

[ericjung]

Hi,
What's the best way to write/read (a.k.a. serialize/deserialize) XML to/from disk in my extension? I'm already using MonkeySage's jsio.js for non-XML reading/writing.

Thanks for any help,
grimholtz

Edit: this is a DOM tree which adheres to my own DTD/schema, not HTML or XHTML.

[Ted Mielczarek]

You want to use DOMParser and XMLSerializer to read and write XML respectively. See the XML Extras page for more details.

Code: Select all

dp = new DOMParser();
xs = new XMLSerializer();


[ericjung]

Thanks. Glad to see they'll be replaced by DOM Level 3 Load and Save.

[ericjung]

Along the same topic, I want to use XPath expressions against the document read with DOMParser. I found the documentation for the following:
nsIDOMXPathEvaluator, XPathEvaluator, XPathExpression, and XPathResult but no examples on how to use the evaluate() method, and then iterate over any results.

Can anyone provide and/or point me to some examples? I've used XPath with Java and C/C++, so a simple example should suffice.

Thank you in advance,
grimholtz

[ericjung]

Found this on XULPlanet here, contributed by wanderingstan@morethanwarm.mail.com:

Code: Select all

function xPathSearch(aXmlDoc, aXpath) {
  var pXPE = new XPathEvaluator();
  var foundNodes = new Array();

  var result = pXPE.evaluate(aXpath, aXmlDoc, pXPE.createNSResolver(aXmlDoc.documentElement), 0, null);
  while (res = result.iterateNext()) {
    foundNodes.push(res);
  }
  return foundNodes;
}

Still playing with it, combined with DOMParser(), to read a file then extract specific info. If anyone has clues, I'd appreciate it.

-grimholtz
p.s. This would make a great Knowledge Base article; I'll write it up when finished.

[jedbro]

Sounds interesting, I would love to see some code if you get it working.
Please excuse my ignorance, but what can you use XPath for? I've never used it before.
Thanks
-Jed

[ericjung]

Hi Jed & Ted. Got it working. Here's an example (I'll write an extensive knowledge base aritcle tomorrow):

Assume the following XML document is stored on disk at %ProfileDirectory/extensions/{5872365E-67D1-4AFD-9480-FD293BEBD20D}/test.xml

Code: Select all

<?xml version="1.0"?>
<people>
  <person first-name="eric" middle-initial="H" last-name="jung">
    <address street="321 south st" city="denver" state="co" country="usa"/>
    <address street="123 main st" city="arlington" state="ma" country="usa"/>
  </person>

  <person first-name="jed" last-name="brown">
    <address street="321 north st" city="atlanta" state="ga" country="usa"/>
    <address street="123 west st" city="seattle" state="wa" country="usa"/>
    <address street="321 south avenue" city="denver" state="co" country="usa"/>
  </person>
</people>


Now assume I have these two functions defined:

Code: Select all

function evalaute(aXmlDoc, aXpath) {
  // evaluate an XPath expression against a Document object, returning the results as an array
  // thanks wanderingstan at morethanwarm dot mail dot dom

  var xpe = new XPathEvaluator();
  var found = new Array();
  var result = xpe.evaluate(aXpath, aXmlDoc, xpe.createNSResolver(aXmlDoc.documentElement), 0, null);
  while (res = result.iterateNext())
    found.push(res);
  return found;
}

function readFile(fileName) {
  // read a file and return its contents
  // Note below my note to MonkeySage and his jsio.js class

  var file = Components.classes['@mozilla.org/file/directory_service;1']
      .getService(Components.interfaces.nsIProperties).get("ProfD", Components.interfaces.nsIFile);
  file.append("extensions");
  file.append("{5872365E-67D1-4AFD-9480-FD293BEBD20D}");
  file.append(fileName);

  // MonkeySage's read() & write() functions should be updated to use a finally block like this!
  var fiStream = null, siStream = null;
  try {
    fiStream = Components.classes["@mozilla.org/network/file-input-stream;1"]
      .createInstance(Components.interfaces.nsIFileInputStream);
    siStream = Components.classes["@mozilla.org/scriptableinputstream;1"]
      .createInstance("Components.interfaces.nsIScriptableInputStream");
    fiStream.init(file, 1, 0, false);
    siStream.init(fiStream);
    var data = new String(siStream.read(-1));
    return data;
  }
  catch (e) {
    return false;
  }
  finally {
    if (siStream)
      siStream.close();
    if (fiStream)
      fiStream.close();
  }
}


Now read the file...

Code: Select all

var fileContents = readFile("test.xml");


Jed, I can now "query" the document with XPath expressions. Although walking the DOM tree achieves the same thing, using XPath expressions is much simpler and more powerful. If you can rely on id attributes, document.getElementById() is still powerful, but it's not nearly as powerful as XPath. Here are some examples, but for a tutorial check out this (it's direct and to-the-point).

Code: Select all

// display the last names of all people in the doc
var results = evaluate(fileContents, "//person/@last-name");
for (var i=0; i<results.length; i++)
  alert("Person #" + i + " has the last name " + results[i]);


// get the 2nd person node
results = evaluate(fileContents, "/people/person[2]");

// get all the person nodes that have addresses in denver
results = evaluate(fileContents, "//person[address/@city='denver']);

//get all the addresses that have "south" in the street name
results = evaluate(fileContents,  "//address[contains(@street, 'south')]");


By the way, I found an XPath expression evaluator for Mozilla/Firefox. I'd seen these for IE, but not for yet for Mozilla/Firefox. With this html file, you can play around with XPath interactively to your heart's content -- similar to JS with Ted's Javascript Environment in his Extension Developer.

In fact -- Ted -- can we bundle up a version of this and get it in a release of Extension Developer?

[asqueella]

Just wondering, wouldn't using XMLHttpRequest's responseXML be easier?

[ericjung]

wouldn't using XMLHttpRequest's responseXML be easier?

For what? Reading the file into a DOM tree? I thought that didn't work for local files. If it does, that only replaces my readFile() function. It doesn't help with XPath at all.

You and Jed should find the above code interesting for use with QuickNote, or any application that must read/write files locally. Why not save those files as XML instead of however you're doing it now? (comma-delimited, pipe-delimited?) You'll get more power; you could, for example, enable the users to search all his QuickNotes using any of a variety of criteria (by date, by contents, by URL, whatever)--much better than regex searching. Maybe QuickNote already has a search feature--but I didn't see it as first glance.

Ted, I'm willing to convert the code on TopXML into a Firefox extension... any interest in putting in Extension Developer?

[jedbro]

grimholtz.
Thanks for the examples and explanation, I'm looking forward to your KB article.
Quick question, what firefox extension are you creating? Sounds like it will be something big and complex seeing all your questions and answers
(feel free to not answer if you cannot for nda reasons).
Cheers
-Jed

[ericjung]

Hi Jed. I've learned a lot from your code in the past (without you're realizing it, I'm sure), so I'm glad to return the favor if possible.

I'm looking forward to your KB article.

Unfortunately, I'm going away for the weekend so the article will have to wait until Monday.

what firefox extension are you creating

Believe it or not, almost all of these questions have been for PasswordMaker. This thread was started because v0.3 of PasswordMaker will auto-populate password fields without ever storing passwords (more secure if they are generated on-the-fly). But in order to do that, I need to start storing UI settings in a file instead of relying on persist attributes that get written to localstore.rdf. And storing data, if it's more than one or two pieces of info, in any other format besides XML seems just plain wrong!

Sounds like it will be something big and complex seeing

Not really!!

And by the way, since PasswordMaker is F/OSS, NDA does not apply.

I'm learning Firefox extension programming in leaps and bounds primarily because of you and asqueella, and I can't thank you enough. That's not to say others on this msg board haven't helped me, too, btw.

FWIW, I'm also an experienced COM/DCOM/C/C++ developer -- well over a decade of experience -- (although I do mostly Java these days), so learning the backend of Gecko has been a snap. I can't wait to write more of my own native XPCOM extensions. If you've got any ideas or have a need for something, let me know so I can return a favor to you!

[asqueella]

grimholtz, I was talking about reading and parsing XML files. afaik, XMLHttpRequest works with local files too.

re XML in QuickNote. As I understand it, we're not willing to store the notes themselves in XML. Storing metadata in a more advanced format (and not in prefs) would indeed make more sense, but I haven't yet got time to implement that.

[jedbro]

Yeah, storing quicknotes in XML would be great, but I for one (any many other users) need them stored in .txt files for use in other editors for easy access/portability, however there might be a way to address it, you are right though, xml would be ideal. hmm..

Thanks for the offer grimholtz, I just might take it up one of these days .
Cheers
-Jed

[Ted Mielczarek]

grim: Yes, this would be a useful addition to Extension Developer. It needs to be licensed MPL/GPL/LGPL tri-license, is the only caveat.

[jedbro]

Grim: were you ever able to write that article?