XML deserialize and decrypting encoding problem. Please help me
This is my first topic here, so at first I'd like to say "Hi" everyone and apologise for my bad english ;)
I have just finished my new application about signing/checking and encrypting/decrypting XML files. I use Apache XML Security packages to do this.
Everything works fine, instead of one...
I'm Polish and sometimes I have to encrypt or decrypt XML which includes polish letters like: 'ą' , 'ę', 'ł' and some others... If I encrypt such file, it succeeds. The problem is when I try to decrypt such an encrypted file. I recieve an error like :
"[Fatal Error] :2:7: An invalid XML character (Unicode: 0x19) was found in the element content o
f the document.
gov.mf.common.exceptions.SenderException: E_SENDER_DECRYPTION
at gov.mf.common.xml.encryption.EncryptTool.decrypt(Unknown Source)
at gov.mf.CERBER.TestCBR.main(Unknown Source)
Caused by: org.apache.xml.security.encryption.XMLEncryptionException: An invalid XML character
(Unicode: 0x19) was found in the element content of the document.
Original Exception was org.xml.sax.SAXParseException: An invalid XML character (Unicode: 0x19)
was found in the element content of the document.
at org.apache.xml.security.encryption.XMLCipher$Serializer.deserialize(Unknown Source)
at org.apache.xml.security.encryption.XMLCipher.decryptElement(Unknown Source)
at org.apache.xml.security.encryption.XMLCipher.doFinal(Unknown Source)
... 2 more
"
What's wrong? My XML document is UTF-8 encoded, with or without BOM. I wrote in in Notepad++ or any other editior which has UTF-8 encoding.
I'm parsing my XML with DOM. There is an interesting line in an error above like: " at org.apache.xml.security.encryption.XMLCipher$Serializer.deserialize(Unknown Source)" , do you know that?
Everything is fine when I try to encrypt/decrypt '? or 'ń', but things go wrong with 'ą', 'ę', 'ł' and others... I also managed to encrypt and decrypt 'ł' but unfortunately, after decryption 'ł' turns into 'B'. It obviously an encoding problem, but how to fix it?
I would be really thankfull if some of You guys would help me.
Looking forward fo any answers.
Matthew
Message was edited by:
matthew_pl
# 1
I think you need to post your code for us to look at.
# 2
Hi once again.
I still don't havy any solution to my problem. I used Apache XML Security examples to encrypt/decrypt my XML document with Polish charaters but I also recieve the same error. What's wrong?
Here is some code:
-- Parsing XML do Document
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
//Bardzo wazna linijka - bless TEK ;)
factory.setNamespaceAware(true);
DocumentBuilder builder;
builder = factory.newDocumentBuilder();
File f = new File(Const.FILE_IN_PATH + File.separator + Const.FILE_IN);
org.w3c.dom.Document doc = builder.parse(f);
- Encrypting & Decrypting XML document (whole class) -
import java.io.*;
import java.security.*;
import javax.crypto.SecretKey;
import javax.crypto.SecretKeyFactory;
import javax.crypto.spec.DESedeKeySpec;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.apache.xml.security.keys.KeyInfo;
import org.apache.xml.security.utils.EncryptionConstants;
import org.apache.xml.security.encryption.XMLCipher;
import org.apache.xml.security.encryption.EncryptedData;
import org.apache.xml.security.encryption.EncryptedKey;
[...]
public class EncryptTool
{
private PublicKey publicKey;
private PrivateKey privateKey;
static
{
org.apache.xml.security.Init.init();
}
public EncryptTool()
{
publicKey = KeyStores.getCerberPublicKey();
privateKey = KeyStores.getCerberPrivateKey();
}
public Document encrypt(Document doc, String sufix)
{
try
{
byte[] passPhrase = "24 Bytes per DESede key!".getBytes("UTF-8");
DESedeKeySpec keySpec = new DESedeKeySpec(passPhrase);
SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("DESede");
SecretKey secretKey = keyFactory.generateSecret(keySpec);
XMLCipher keyCipher = XMLCipher.getInstance(XMLCipher.RSA_v1dot5);
keyCipher.init(XMLCipher.WRAP_MODE, publicKey);
EncryptedKey encryptedKey = keyCipher.encryptKey(doc, secretKey);
Element elementToEncrypt = (Element) doc.getDocumentElement();
System.out.println("Szyrfuję: " + elementToEncrypt.getTextContent());
XMLCipher xmlCipher = XMLCipher.getInstance(XMLCipher.TRIPLEDES);
xmlCipher.init(XMLCipher.ENCRYPT_MODE, secretKey);
EncryptedData encryptedDataElement = xmlCipher.getEncryptedData();
KeyInfo keyInfo = new KeyInfo(doc);
keyInfo.add(encryptedKey);
encryptedDataElement.setKeyInfo(keyInfo);
boolean encryptContentsOnly = true;
xmlCipher.doFinal(doc, elementToEncrypt, encryptContentsOnly);
// output the resulting document
String [] parts = Const.FILE_IN.split("\\.");
String saveAs = Const.FILE_OUT_PATH + File.separator + parts[0] + sufix + "." + parts[1];
OutputStream os = new FileOutputStream(saveAs);
XMLUtil.sameXMLtoFile(doc, os);
} catch (Exception ex)
{
throw new TestCBRException("E_CERBER_ENCRYPTION", ex);
}
return doc;
}
public void decrypt(Document doc, String sufix) throws SenderException
{
try
{
String namespaceURI = EncryptionConstants.EncryptionSpecNS;
String localName = EncryptionConstants._TAG_ENCRYPTEDDATA;
int ile = doc.getElementsByTagNameNS(namespaceURI, localName).getLength();
if (ile == 0) throw new SenderException("E_SENDER_DECRYPTION_NEEDED");
for(int i=0; i < ile; i++)
{
Element encryptedDataElement = (Element) doc.getElementsByTagNameNS(namespaceURI, localName).item(0);
XMLCipher xmlCipher = XMLCipher.getInstance();
xmlCipher.init(XMLCipher.DECRYPT_MODE, null);
xmlCipher.setKEK(privateKey);
xmlCipher.doFinal(doc, encryptedDataElement);
String [] parts = Const.FILE_IN.split("\\.");
String saveAs = Const.FILE_OUT_PATH + parts[0] + sufix + "." + parts[1];
OutputStream os = new FileOutputStream(saveAs);
XMLUtil.saveXMLtoFile(doc, os);
}
} catch (SenderException ex) {
throw ex;
} catch (Exception ex) {
throw new SenderException("E_SENDER_DECRYPTION", ex);
}
}
}
Please help me. I'm going into madness what's wrong with it...
# 3
You need to put [code] [/code]tags round the code.
# 4
Ok, so once again, because I can't edit previous message. Sorry for flooding ;)
-- Parsing XML do Document
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setNamespaceAware(true);
DocumentBuilder builder;
builder = factory.newDocumentBuilder();
File f = new File(Const.FILE_IN_PATH + File.separator + Const.FILE_IN);
org.w3c.dom.Document doc = builder.parse(f);
- Encrypting & Decrypting XML document (whole class) -
import java.io.*;
import java.security.*;
import javax.crypto.SecretKey;
import javax.crypto.SecretKeyFactory;
import javax.crypto.spec.DESedeKeySpec;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.apache.xml.security.keys.KeyInfo;
import org.apache.xml.security.utils.EncryptionConstants;
import org.apache.xml.security.encryption.XMLCipher;
import org.apache.xml.security.encryption.EncryptedData;
import org.apache.xml.security.encryption.EncryptedKey;
[...]
public class EncryptTool
{
private PublicKey publicKey;
private PrivateKey privateKey;
static
{
org.apache.xml.security.Init.init();
}
public EncryptTool()
{
publicKey = KeyStores.getCerberPublicKey();
privateKey = KeyStores.getCerberPrivateKey();
}
public Document encrypt(Document doc, String sufix)
{
try
{
byte[] passPhrase = "24 Bytes per DESede key!".getBytes("UTF-8");
DESedeKeySpec keySpec = new DESedeKeySpec(passPhrase);
SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("DESede");
SecretKey secretKey = keyFactory.generateSecret(keySpec);
XMLCipher keyCipher = XMLCipher.getInstance(XMLCipher.RSA_v1dot5);
keyCipher.init(XMLCipher.WRAP_MODE, publicKey);
EncryptedKey encryptedKey = keyCipher.encryptKey(doc, secretKey);
Element elementToEncrypt = (Element) doc.getDocumentElement();
XMLCipher xmlCipher = XMLCipher.getInstance(XMLCipher.TRIPLEDES);
xmlCipher.init(XMLCipher.ENCRYPT_MODE, secretKey);
EncryptedData encryptedDataElement = xmlCipher.getEncryptedData();
KeyInfo keyInfo = new KeyInfo(doc);
keyInfo.add(encryptedKey);
encryptedDataElement.setKeyInfo(keyInfo);
boolean encryptContentsOnly = true;
xmlCipher.doFinal(doc, elementToEncrypt, encryptContentsOnly);
// output the resulting document
String [] parts = Const.FILE_IN.split("\\.");
String saveAs = Const.FILE_OUT_PATH + File.separator + parts[0] + sufix + "." + parts[1];
OutputStream os = new FileOutputStream(saveAs);
XMLUtil.sameXMLtoFile(doc, os);
} catch (Exception ex)
{
throw new TestCBRException("E_CERBER_ENCRYPTION", ex);
}
return doc;
}
public void decrypt(Document doc, String sufix) throws SenderException
{
try
{
String namespaceURI = EncryptionConstants.EncryptionSpecNS;
String localName = EncryptionConstants._TAG_ENCRYPTEDDATA;
int ile = doc.getElementsByTagNameNS(namespaceURI, localName).getLength();
if (ile == 0) throw new SenderException("E_SENDER_DECRYPTION_NEEDED");
for(int i=0; i < ile; i++)
{
Element encryptedDataElement = (Element) doc.getElementsByTagNameNS(namespaceURI, localName).item(0);
XMLCipher xmlCipher = XMLCipher.getInstance();
xmlCipher.init(XMLCipher.DECRYPT_MODE, null);
xmlCipher.setKEK(privateKey);
xmlCipher.doFinal(doc, encryptedDataElement);
String [] parts = Const.FILE_IN.split("\\.");
String saveAs = Const.FILE_OUT_PATH + parts[0] + sufix + "." + parts[1];
OutputStream os = new FileOutputStream(saveAs);
XMLUtil.saveXMLtoFile(doc, os);
}
} catch (SenderException ex) {
throw ex;
} catch (Exception ex) {
throw new SenderException("E_SENDER_DECRYPTION", ex);
}
}
}
So, can anybody help me with that?
# 5
> Ok, so once again, because I can't edit previous
> message. Sorry for flooding ;)
>
>
> -- Parsing XML do Document
> >
> DocumentBuilderFactory factory =
> DocumentBuilderFactory.newInstance();
>
> factory.setNamespaceAware(true);
>
> DocumentBuilder builder;
> builder = factory.newDocumentBuilder();
> File f = new File(Const.FILE_IN_PATH + File.separator
> + Const.FILE_IN);
> org.w3c.dom.Document doc = builder.parse(f);
>
>
>
> - Encrypting & Decrypting XML document
> (whole class) -
> >
> import java.io.*;
> import java.security.*;
>
> import javax.crypto.SecretKey;
> import javax.crypto.SecretKeyFactory;
> import javax.crypto.spec.DESedeKeySpec;
>
> import org.w3c.dom.Document;
> import org.w3c.dom.Element;
>
> import org.apache.xml.security.keys.KeyInfo;
> import
> org.apache.xml.security.utils.EncryptionConstants;
> import org.apache.xml.security.encryption.XMLCipher;
> import
> org.apache.xml.security.encryption.EncryptedData;
> import
> org.apache.xml.security.encryption.EncryptedKey;
> [...]
>
> public class EncryptTool
> {
> private PublicKey publicKey;
> private PrivateKey privateKey;
>
> static
> {
> org.apache.xml.security.Init.init();
> }
>
> public EncryptTool()
> {
>
> publicKey = KeyStores.getCerberPublicKey();
> privateKey = KeyStores.getCerberPrivateKey();
>
> }
>
>
> public Document encrypt(Document doc, String sufix)
> {
> try
> {
> byte[] passPhrase = "24 Bytes per DESede
> key!".getBytes("UTF-8");
> DESedeKeySpec keySpec = new
> DESedeKeySpec(passPhrase);
> SecretKeyFactory keyFactory =
> SecretKeyFactory.getInstance("DESede");
> SecretKey secretKey =
> keyFactory.generateSecret(keySpec);
>
>
> XMLCipher keyCipher =
> XMLCipher.getInstance(XMLCipher.RSA_v1dot5);
> keyCipher.init(XMLCipher.WRAP_MODE, publicKey);
>
> EncryptedKey encryptedKey = keyCipher.encryptKey(doc,
> secretKey);
>
> Element elementToEncrypt = (Element)
> doc.getDocumentElement();
>
> XMLCipher xmlCipher =
> XMLCipher.getInstance(XMLCipher.TRIPLEDES);
> xmlCipher.init(XMLCipher.ENCRYPT_MODE, secretKey);
>
> EncryptedData encryptedDataElement =
> xmlCipher.getEncryptedData();
> KeyInfo keyInfo = new KeyInfo(doc);
> keyInfo.add(encryptedKey);
> encryptedDataElement.setKeyInfo(keyInfo);
>
> boolean encryptContentsOnly = true;
> xmlCipher.doFinal(doc, elementToEncrypt,
> encryptContentsOnly);
>
> // output the resulting document
> String [] parts = Const.FILE_IN.split("\\.");
> String saveAs = Const.FILE_OUT_PATH + File.separator
> + parts[0] + sufix + "." + parts[1];
> OutputStream os = new FileOutputStream(saveAs);
> XMLUtil.sameXMLtoFile(doc, os);
>
> } catch (Exception ex)
> {
> throw new TestCBRException("E_CERBER_ENCRYPTION",
> ex);
> }
>
> return doc;
> }
>
>
> public void decrypt(Document doc, String sufix)
> throws SenderException
> {
> try
> {
> String namespaceURI =
> EncryptionConstants.EncryptionSpecNS;
> String localName =
> EncryptionConstants._TAG_ENCRYPTEDDATA;
>
>
> int ile = doc.getElementsByTagNameNS(namespaceURI,
> localName).getLength();
> if (ile == 0) throw new
> SenderException("E_SENDER_DECRYPTION_NEEDED");
>
> for(int i=0; i < ile; i++)
> {
>
> Element encryptedDataElement = (Element)
> doc.getElementsByTagNameNS(namespaceURI,
> localName).item(0);
>
> XMLCipher xmlCipher = XMLCipher.getInstance();
> xmlCipher.init(XMLCipher.DECRYPT_MODE, null);
>
> xmlCipher.setKEK(privateKey);
>
>
> xmlCipher.doFinal(doc, encryptedDataElement);
>
>
> String [] parts = Const.FILE_IN.split("\\.");
> String saveAs = Const.FILE_OUT_PATH + parts[0] +
> sufix + "." + parts[1];
> OutputStream os = new FileOutputStream(saveAs);
> XMLUtil.saveXMLtoFile(doc, os);
> }
>
> } catch (SenderException ex) {
> throw ex;
> } catch (Exception ex) {
> throw new SenderException("E_SENDER_DECRYPTION",
> ex);
> }
> }
> }
>
>
> So, can anybody help me with that?
Witam,
Mialem ten sam problem. Winna jest biblioteka xml-sec a wlasciwie jej procedury konwersji char (int) do byte[]. ZMiana na prawidlowy encoding UTF8 zalatwia problem. O szczegoly prosze pytac na priv.
# 6
> So, can anybody help me with that?There is too much missing code for me to be able to diagnose the problem. Others may be more able to help.
# 7
Well, that's all what is essential. What more would you like to see? These are two methods to encrypt and decrypt parsed document and sth is wrong with them, i think.
To Rafał31
Możesz dać jakieś namiary na ten priv? Z tego co widzę nie udostępniasz swojego maila, więc nie mam jak się z Tobą skontaktować.
Możesz rozwinąć myśl z kodowaniem UTF-8? Moje pliki są (jak twierdzi XMLSpy i inne narzędzia) w UTF-ie.
# 8
Mostly I suspect, it is something to so with the encrypt/decrpyt method is not compatible with the specific character(s) you have mentioned here. May be yoca try any other or your own decrpyt/encrypt method(I am not sure how to do that but have seen ppl saying it ;-))
I have not found a direct solution but see this link if it helps
Java supports UTF in four main ways.
1. DataOutputStream.writeUTF/ DataInputStream/readUTF: modified UTF-8.
2. OutputStreamWriter/InputStreamReader: any encoding including UTF-8, UTF-16, UTF-16BE, UTF-16LE.
3. String.getBytes/new String: any encoding including UTF-8, UTF-16, UTF-16BE, UTF-16LE.
4. CharBuffer & CharsetEncoder. encoder/ByteBuffer & CharsetDecoder. decode : any encoding including UTF-8, UTF-16, UTF-16BE, UTF-16LE.
I wrote this program to discover if Java conformed with the UTF standards. It demonstrates these four ways or writing and reading UTF. It turns out Java conforms, with the exception of writeUTF.
I lay aside for now the issue of dealing with 32-characters.
http://mindprod.com/jgloss/utf.html#UTF8DECODER
# 9
1. You can try to cheat for the character in trouble for example, if character "A" is in trouble you can substitute it with DECRYPTABLE "B" or "BB" or "~B" use your imagintion
2. Alternatively you shoud find a encrypt/decrypt decrypt package that is totally comaptible with the Language in issue and JAVA
# 10
More related thread, http://forum.java.sun.com/thread.jspa?threadID=5114959
# 11
Thanks a lot for any answers.
I'm closer and closer to think that Apache XML Security package is not 100% compatible with the charset I used. Whenever I try to encrypt, before encryption I can read my document as it is in real, with all characters. Unfortunately, after decryption I loose my foreign characters, and I don't know why. I'm doing that on document, I even don't write document to XML file.
I'm also looking forward too hearing hrom Rafal31. He told (in Polish), some posts earlier that he had the same problem and the couse was the xml-sec package and it's char (int) to byte[] conversion methods... I don't know exactly how to solve the problem, but thanks for all advice in advance.
# 12
I regret to say I haven't found the solution yet. Does anyone of you had any idea? Rafal31 - please contact me if You can!
# 13
Ok, that's all. I have finally solved that problem so a few words below...
Generally there are two ways to fix this issue:
1) Just use xmlsec-1.3.0.jar instead of xmlsec-1.4.0.jar. It seems to me that everything works fine with v.1.3.0 and no further changes are required.
2) If someone has to use xmlsec-1.4.0.jar, you have to download sources (here: http://xml.apache.org/security/dist/java-library/), and open CanonicalizerBase.java and UtfHelpper.java in org.apache.xml.security.c14n.implementations package
You have to change lines:
=> CanonicalizerBase.java <=
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
line 696: if( (c & 0x80) ==0) {
into if(c < 0x80){
line 731: if( (c & 0x80) ==0) {
into if(c < 0x80) {
line 783: if( (c & 0x80) ==0) {
into if(c < 0x80) {
=> UtfHelpper.java <=
^^^^^^^^^^^^^^^^^^^^^^^
line 21: if ( (c & 0x80) ==0) {
into if(c < 0x80) {
line 56: if ((c & 0x80) == 0) {
into if(c < 0x80) {
line 96: if ((c & 0x80) == 0) {
into if(c < 0x80) {
Be aware of some extra spaces between "(" so don't use Find function in your editor, and change every line manually.
After making changes in those two files you compile them by:
javac CanonicalizerBase.java UtfHelpper.java
Now you should get xmlsec-1.4.0.jar end extract it using following command from your command line:
jar -xvf xmlsec-1.4.0.jar
You should recieve three folders: org, jawax and META-INF.
Get two compiled files (CanonicalizerBase.class UtfHelpper.class) and put them in org/apache/xml/security/c14n/implementations.
Finally, let's build fixed xmlsec-1.4.0.jar library by putting in a cmd:
jar -cvf xmlsec-1.4.0-utf_bug_fixed.jar META-INF\MANIFEST.MF *.class org/ javax/
That's all folks. Everything you have to do is to remove xmlsec-1.4.0.jar and replace it with xmlsec-1.4.0-utf_bug_fixed.jar (Remember about new filename of your .jar )
Useful links:
^^^^^^^^^^^^^
ASF Bug no. 41472 (bacis info): http://issues.apache.org/bugzilla/show_bug.cgi?id=41472
The same, but more info: http://www.mail-archive.com/security-dev@xml.apache.org/msg03748.html
My thread at nabble Apache XML Security Dev mailing list (further information about the problem): http://www.nabble.com/Apache-XML-Security-Encoding-problem-in-encrypting-decrypting-XML-tf3141650.html
That's all, folks!
Thanks guys for your help!
Matthew
