the problem is when the certificate is not valid or not known, the certificate dialog window does not pop up (I always get a response status 0)... is it normal ?
I think you're responsible for handling authentication errors and opening dialogs yourself. Password dialogs may be opened automatically though, depending on settings. Missing certificates do not return very helpful error codes. See this bug for background. For possible workarounds, see this page.
mozillaZine is an independent Mozilla community and advocacy site. We're not affiliated or endorsed by the Mozilla Corporation but we love them just the same.