Failed Core Validation

Yet more info. Dumped the public and private keys in GenDetached.java as well, as observed the same phenomenon as described above, i.e., the P and Q appear different going into the signing versus the signature produced. So that becomes an irrelevant observation.

Here is the XML generated by my application:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">

<SignedInfo>

<CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments"/>

<SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#dsa-sha1"/>

<Reference URI="file:/C:/JTDITransport_svn/foo.txt">

<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>

<DigestValue>HNq7BMuL4wdr6g6YJOuS8DlTcDA=</DigestValue>

</Reference>

</SignedInfo>

<SignatureValue>VnUgzbJHudZSG+/yeh5jfO3jBIJPfJga4NjBW2BILqgRSn7H/khJ+A==</SignatureValue>

<KeyInfo>

<KeyValue>

<DSAKeyValue>

<P>/KaCzo4Syrom78z3EQ5SbbB4sF7ey80etKII864WF64B81uRpH5t9jQTxeEu0ImbzRMqzVDZkVG9xD7nN1kuFw==</P>

<Q>li7dzDacuo67Jg7mtqEm2TRuOMU=</Q>

<G>Z4Rxsnqc9E7pGknFFH2xqaryRPBaQ01khpMdLRQnG541Awtx/XPaF5Bpsy4pNWMOHCBiNU0NogpsQW5QvnlMpA==</G>

<Y>MFc9VKzBT0ilHvURWYyFmwhPyCMdzVSqqkLe/SH+Y40vbZmUbyUboJHA/da0ad1CTjQjvCORXDb1wml6N+/DOQ==</Y>

</DSAKeyValue>

</KeyValue>

</KeyInfo>

</Signature>

And here is the XML generated by GenDetached.java for the exact same input file:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">

<SignedInfo>

<CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments"/>

<SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#dsa-sha1"/>

<Reference URI="file:/C:/JTDITransport_svn/foo.txt">

<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>

<DigestValue>HNq7BMuL4wdr6g6YJOuS8DlTcDA=</DigestValue>

</Reference>

</SignedInfo>

<SignatureValue>Uhf2xFWy3f6o0bXjZk3+3x5fkOOUwObfY+gfQyVFkC01X6C4AC7CGw==</SignatureValue>

<KeyInfo>

<KeyValue>

<DSAKeyValue>

<P>/KaCzo4Syrom78z3EQ5SbbB4sF7ey80etKII864WF64B81uRpH5t9jQTxeEu0ImbzRMqzVDZkVG9xD7nN1kuFw==</P>

<Q>li7dzDacuo67Jg7mtqEm2TRuOMU=</Q>

<G>Z4Rxsnqc9E7pGknFFH2xqaryRPBaQ01khpMdLRQnG541Awtx/XPaF5Bpsy4pNWMOHCBiNU0NogpsQW5QvnlMpA==</G>

<Y>MFc9VKzBT0ilHvURWYyFmwhPyCMdzVSqqkLe/SH+Y40vbZmUbyUboJHA/da0ad1CTjQjvCORXDb1wml6N+/DOQ==</Y>

</DSAKeyValue>

</KeyValue>

</KeyInfo>

</Signature>

As you can see, the information is 100% identical except for the <SignatureValue>. Validate.java approves of the XML created by GenDetached.java, but claims "failed core validation" for the XML created by my application, which as far as I can tell is statement-for-statement identical to GenDetached.java.

How to diagnose "failed core validation"? Thanks.

Frank_Natolia at 2007-7-29 17:11:56 >

Thanks, Sean, and too bad you weren't at Sun Tech Days in DC this past June. I would have bought you as many adult beverages as it took for me to understand Java digital signatures.

Had seen your first blog entry, and since that was more or less a restatement of the JWSDP 2.0 Validate.java demo code, I had already implemented the relevant statements. Therefore, I already knew that although "core validation" had failed, the single reference item had succeded, see below log entry:

[junit] 2007-07-26 09:47:54,750 [main] INFO com.cryptek.jtdi.dim.dmt.JTDITransportImpl - Signature failed core validation

[junit] 2007-07-26 09:47:54,750 [main] INFO com.cryptek.jtdi.dim.dmt.JTDITransportImpl - Signature validation status: false

[junit] 2007-07-26 09:47:54,750 [main] INFO com.cryptek.jtdi.dim.dmt.JTDITransportImpl - ref[0] validity status: true

Every example I could find on the web of a failed core validation also included at least one failed reference validation. But not mine!

Courtesy of your second blog, I enabled detail logging, which of course produced an avalanche of information. Perhaps significantly, the "expected" and "actual" digests are the same:

[junit] Jul 26, 2007 9:47:54 AM org.jcp.xml.dsig.internal.dom.DOMReference validate

[junit] FINE: Expected digest: HNq7BMuL4wdr6g6YJOuS8DlTcDA=

[junit] Jul 26, 2007 9:47:54 AM org.jcp.xml.dsig.internal.dom.DOMReference validate

[junit] FINE: Actual digest: HNq7BMuL4wdr6g6YJOuS8DlTcDA=

Have some additional information. My app is generating a JAXB object for the digital signature, then inserting the generated JAXB object into a schema for which the digital signature is only one element. Here is the key point: it appears that the <SignatureValue> produced by the JAXB object is NOT the same as the <SignatureValue> produced by a stream output. My app includes the following code:

// output the resulting document to a JAXBResult

TransformerFactory tf = TransformerFactory.newInstance();

Transformer trans = tf.newTransformer();

JAXBContext jcs = JAXBContext.newInstance("com.cryptek.jtdi.dim.dmt.jaxb");

JAXBResult jaxbResult = new JAXBResult(jcs);

trans.transform(new DOMSource(doc), jaxbResult);

JAXBElement<SignatureType> jaxbElement = (JAXBElement<SignatureType>)jaxbResult.getResult();

// output the resulting document to file

trans.transform(new DOMSource(doc), new StreamResult(new FileOutputStream("signature.xml")));

The JAXB object is inserted into a complex JAXB object that after marshalling produces the following <Signature>:

<Signature>

<SignedInfo>

<CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments"></CanonicalizationMethod>

<SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#dsa-sha1"></SignatureMethod>

<Reference URI="file:/C:/JTDITransport_svn/foo.txt">

<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"></DigestMethod>

<DigestValue>HNq7BMuL4wdr6g6YJOuS8DlTcDA=</DigestValue>

</Reference>

</SignedInfo>

<SignatureValue>QQAID7o2T1THuz9m7P6w80gSoiN5nAfvD+iW5K+XEI71EYyMm0qoSw==</SignatureValue>

<KeyInfo>

<KeyValue>

<DSAKeyValue>

<P>/KaCzo4Syrom78z3EQ5SbbB4sF7ey80etKII864WF64B81uRpH5t9jQTxeEu0ImbzRMqzVDZkVG9xD7nN1kuFw==</P>

<Q>li7dzDacuo67Jg7mtqEm2TRuOMU=</Q>

<G>Z4Rxsnqc9E7pGknFFH2xqaryRPBaQ01khpMdLRQnG541Awtx/XPaF5Bpsy4pNWMOHCBiNU0NogpsQW5QvnlMpA==</G>

<Y>MFc9VKzBT0ilHvURWYyFmwhPyCMdzVSqqkLe/SH+Y40vbZmUbyUboJHA/da0ad1CTjQjvCORXDb1wml6N+/DOQ==</Y>

</DSAKeyValue>

</KeyValue>

</KeyInfo>

</Signature>

Whereas the output stream generated the following:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">

<SignedInfo>

<CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments"/>

<SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#dsa-sha1"/>

<Reference URI="file:/C:/JTDITransport_svn/foo.txt">

<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>

<DigestValue>HNq7BMuL4wdr6g6YJOuS8DlTcDA=</DigestValue>

</Reference>

</SignedInfo>

<SignatureValue>Ml08ontriC9kmiybhlh/DOK61P1mnR326YcnKt6bZqrL7GKoZfZIMA==</SignatureValue>

<KeyInfo>

<KeyValue>

<DSAKeyValue>

<P>/KaCzo4Syrom78z3EQ5SbbB4sF7ey80etKII864WF64B81uRpH5t9jQTxeEu0ImbzRMqzVDZkVG9xD7nN1kuFw==</P>

<Q>li7dzDacuo67Jg7mtqEm2TRuOMU=</Q>

<G>Z4Rxsnqc9E7pGknFFH2xqaryRPBaQ01khpMdLRQnG541Awtx/XPaF5Bpsy4pNWMOHCBiNU0NogpsQW5QvnlMpA==</G>

<Y>MFc9VKzBT0ilHvURWYyFmwhPyCMdzVSqqkLe/SH+Y40vbZmUbyUboJHA/da0ad1CTjQjvCORXDb1wml6N+/DOQ==</Y>

</DSAKeyValue>

</KeyValue>

</KeyInfo>

</Signature>

Note that everything appears to be identical EXCEPT for <SignatureValue>. Curiously, if I insert consecutive calls to "trans.transform" to generate multiple output stream copies, all copies bear the same <SignatureValue> result, and all copies pass Validate.java testing. But Validate.java complains of "failed core validation" for the JAXB object though approving its single reference item.

I have also found that the very same output stream produced by "trans.transform" that Validate.java approves, becomes "failed core validation" if I manually edit the XML to put CR-LF after certain XML tags.

Thanks for your time.

Frank_Natolia at 2007-7-29 17:11:56 >

Ok, as I understand it, the signature over the SignedInfo element is invalid, right?

My best guess is that when validating the XML Signature, the canonicalized bytes

of the SignedInfo element are not the same (as when you generated the signature), and thus the signature check fails. Probably in the process of serializing the JAXB object,

it changed the content such that the signature broke.

You can use the debug logging to dump the canonicalized bytes of the SignedInfo

element. Look for the following message:

[java] FINE: Canonicalized SignedInfo: ...

Now, since you are generating the SignatureValue yourself, you will have to find a way to print the same bytes when generating. Once you do that, carefully inspect the

output for differences. Often it is something very subtle.

smullana at 2007-7-29 17:11:57 >

Here are the relevant log entries:

[junit] FINE: Canonicalized SignedInfo:

[junit] <SignedInfo xmlns="http://www.w3.org/2000/09/xmldsig#"><CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315#WithComments"></CanonicalizationMethod><SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#dsa-sha1"></SignatureMethod><Reference URI="file:/C:/JTDITransport_svn/foo.txt"><DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"></DigestMethod><DigestValue>HNq7BMuL4wdr6g6YJOuS8DlTcDA=</DigestValue></Reference></SignedInfo>

[junit] Jul 27, 2007 5:55:36 PM org.jcp.xml.dsig.internal.dom.DOMSignedInfo canonicalize

Regarding your comment about changing line breaks, I confess to have inserted line breaks in the above forum posts but only for forum readability purposes and most certainly did NOT manipulate any XML data in the application itself. All XML writing was done by the Java API marshalling calls.

Here is the code that generates the signature. You will recognize its remarkable similarity to GenDetached.java:

// create digital signature objects

String providerName = System.getProperty("jsr105Provider", "org.jcp.xml.dsig.internal.dom.XMLDSigRI");

XMLSignatureFactory fac = XMLSignatureFactory.getInstance("DOM", (Provider) Class.forName(providerName).newInstance());

KeyInfoFactory kif = fac.getKeyInfoFactory();

DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();

dbf.setNamespaceAware(true); // must be set

TransformerFactory tf = TransformerFactory.newInstance();

JAXBContext jcs = JAXBContext.newInstance("com.cryptek.jtdi.dim.dmt.jaxb");

...

// create reference to external/detached file

Reference ref = fac.newReference(

new File(srcFile).toURI().toString(),

fac.newDigestMethod(DigestMethod.SHA1, null));

// Create the SignedInfo

Signed Info si = fac.newSignedInfo(

fac.newCanonicalizationMethod(CanonicalizationMethod.INCLUSIVE_WITH_COMMENTS, (C14NMethodParameterSpec) null),

fac.newSignatureMethod(SignatureMethod.DSA_SHA1, null),

Collections.singletonList(ref));

// Create a KeyValue containing the DSA PublicKey that was generated

KeyValue kv = kif.newKeyValue(publicKey);

// Create a KeyInfo and add the KeyValue to it

KeyInfo ki = kif.newKeyInfo(Collections.singletonList(kv));

// Create the XMLSignature (but don't sign it yet)

XMLSignature signature = fac.newXMLSignature(si, ki);

// Create the Document that will hold the resulting XMLSignature

Document doc = dbf.newDocumentBuilder().newDocument();

// Create a DOMSignContext and set the signing Key to the DSA

// PrivateKey and specify where the XMLSignature should be inserted

// in the target document (in this case, the document root)

DOMSignContext signContext = new DOMSignContext(privateKey, doc);

// Set the BaseURI to bypass a bug in the JKD. See the following URL

// for more information:

//http://forum.java.sun.com/thread.jspa?threadID=745105

signContext.setBaseURI("file:/dummy");

// Marshal, generate (and sign) the detached XMLSignature. The DOM

// Document will contain the XML Signature if this method returns

// successfully.

signature.sign(signContext);

// output the resulting document to a JAXBResult

Transformer trans = tf.newTransformer();

JAXBResult jaxbResult = new JAXBResult(jcs);

trans.transform(new DOMSource(doc), jaxbResult);

JAXBElement<SignatureType> jaxbElement = (JAXBElement<SignatureType>)jaxbResult.getResult();

// output the resulting document to stdout

trans.transform(new DOMSource(doc), new StreamResult(new FileOutputStream("signatureJtdiData.xml")));

// output the resulting document to a DOMResult

domResult = new DOMResult();

trans.transform(new DOMSource(doc), domResult);

debugLogger.info("dom " + domResult.toString());

DumpNode(domResult.getNode(), 0);

Key point, key point: for debug purposes, as you see above, I have called "trans.transform" three times, always with the same input, to generate three different outputs, one JAXBResult, one OutputStream and one DOMResult. Indications are, the JAXBResult and DOMResult generate the same <SignatureValue> that is declared incorrect by validation, but the OutputStream generates a different <SignatureValue> that is declared correct by validation.

How can this be?

I understand your suspicion that the <SignedInfo>...</SignedInfo> data has changed but given the above code, three consecutive calls to "trans.transform" with absolutely positively no fiddling with input data in between the calls, how can the <SignedInfo>...</SignedInfo> data be changed?

Thanks for your time.

Frank_Natolia at 2007-7-29 17:11:57 >