Garbage when Compressing a Byte array
Hi all guys
I am sure you can help me with this, I am using jazzlib (http://jazzlib.sourceforge.net/) ZIP library for deflating and inflating data, I was all day trying to find examples and I came with this:
publicbyte[] compress(byte[] input)throws IOException
{
Deflater compressor =new Deflater();
compressor.setLevel(Deflater.BEST_COMPRESSION);
byte[] buf =newbyte[input.length];
ByteArrayOutputStream bos =new ByteArrayOutputStream(buf.length);
compressor.setInput(input);
compressor.finish();
while (!compressor.finished()){
int count = compressor.deflate(buf);
bos.write(buf, 0, count);
}
bos.close();
byte[] compressedData = bos.toByteArray();
return compressedData;
}
that works very fine (I uncompressed the data and it's the same as the original) except that the compressed byte[] is longer than the uncompressed.
As you see this is unacceptable, and I think that these lines are the cause:
byte[] buf =newbyte[input.length];
[...]
ByteArrayOutputStream bos =new ByteArrayOutputStream(buf.length);
I am creating a buffer of length=input.length and that's obviusly bigger than compressed data, How can I resolve this if I can't dinamically allocate buf array and bos object?
Thanks guys!
JDK 6 - Latest Version of Jazzlib
[2067 byte] By [
Kurdta] at [2007-11-27 9:53:47]
# 1
> I am creating a buffer of length=input.length and that's
> obviusly bigger than compressed data, How can I
> resolve this if I can't dinamically allocate buf array
> and bos object?
Because there is no guarantee that the compressed data is smaller then the uncompressed data. Especially when you're trying to compress small amounts of data or already compressed data.
> ...
> JDK 6 - Latest Version of Jazzlib
The code you posted is not using any Jazzlib classes AFAIK. You're just using the classes in the standard java.util.zip package.
Here are various examples:
http://www.exampledepot.com/egs/java.util.zip/pkg.html
# 2
Hi, It is using jazzlib, I did not include the imports .It seems that the class names are the same, but trust me I am using jazzlib.Ok, I'll try with a big chunk of data , but anything about allocating dynamically the byte array?
# 3
> Hi, It is using jazzlib, I did not include the
> imports .
>
> It seems that the class names are the same, but trust
> me I am using jazzlib.
I just checked, and we both are right: Jazzlib seems to be in the standard java.util.zip package.
> but anything about allocating dynamically the byte array?
I don't know what you mean by that. The byte array which gets returned is automatically adjusted to the size of the compressed data. You are perhaps talking about the buffer?
Anyway, here's an implemented example form some code scraped from http://exampledepot.com:
import java.util.zip.*;
import java.io.*;
import java.util.Arrays;
public class ZipDemo {
public static byte[] compress(byte[] uncompressed) {
Deflater compressor = new Deflater();
compressor.setLevel(Deflater.BEST_COMPRESSION);
compressor.setInput(uncompressed);
compressor.finish();
ByteArrayOutputStream bos = new ByteArrayOutputStream(uncompressed.length);
byte[] buf = new byte[1024];
while (!compressor.finished()) {
int count = compressor.deflate(buf);
bos.write(buf, 0, count);
}
try {
bos.close();
} catch (IOException e) {
e.printStackTrace();
}
return bos.toByteArray();
}
public static byte[] decompress(byte[] compressed) {
Inflater decompressor = new Inflater();
decompressor.setInput(compressed);
ByteArrayOutputStream bos = new ByteArrayOutputStream(compressed.length);
byte[] buf = new byte[1024];
while (!decompressor.finished()) {
try {
int count = decompressor.inflate(buf);
bos.write(buf, 0, count);
} catch (DataFormatException e) {
e.printStackTrace();
}
}
try {
bos.close();
} catch (IOException e) {
e.printStackTrace();
}
return bos.toByteArray();
}
public static void main(String[] args) {
String text = "aaaaaaaaaaaaabbbbbbbbbbbbbbbbbbccccccccccccccc";
byte[] compressed = compress(text.getBytes());
byte[] decompressed = decompress(compressed);
System.out.println("bytes original text : "+Arrays.toString(text.getBytes()));
System.out.println("bytes compressed: "+Arrays.toString(compressed));
System.out.println("bytes decompressed : "+Arrays.toString(decompressed));
}
}
# 4
Hi prometeuz
> I don't know what you mean by that. The byte array
> which gets returned is automatically adjusted to the
> size of the compressed data.
Didn't know that! I based my conclusions on this:
this is my code:
public class JazzLibStreamTester {
private static Utils utils;
public static void main(String args[])
{
JazzLibStreamTester t = new JazzLibStreamTester();
String texto = "java forums!!";
System.out.println("Input to Hex " + utils.toHex(texto.getBytes()) + " " + texto.length());
byte ble[]={};
try {
ble = t.compressB(texto.getBytes());
} catch (IOException ex) {
ex.printStackTrace();
}
System.out.println("deflated to hex " + utils.toHex(ble) + " " + ble.length);
byte asd[]={};
try {
asd = t.decompressB(ble);
} catch (Exception ex) {
ex.printStackTrace();
}
System.out.println("inflated to Hex " + utils.toHex(asd)+ " " + asd.length);
System.out.println("inflated to String " + new String(asd)+ " " + new String(asd).length());
}
public byte[] compressB(byte[] input) throws IOException
{
Deflater compressor = new Deflater();
compressor.setLevel(Deflater.BEST_COMPRESSION);
byte[] buf = new byte[input.length];
ByteArrayOutputStream bos = new ByteArrayOutputStream(buf.length);
compressor.setInput(input);
compressor.finish();
while (!compressor.finished()) {
int count = compressor.deflate(buf);
bos.write(buf, 0, count);
}
bos.close();
// Get the compressed data
byte[] compressedData = bos.toByteArray();
return compressedData;
}
public byte[] decompressB(byte[] input) throws DataFormatException, IOException
{
byte[] buf = new byte[input.length];
ByteArrayOutputStream bos = new ByteArrayOutputStream(buf.length);
Inflater inf = new Inflater();
inf.setInput(input);
while (!inf.finished()) {
int count = inf.inflate(buf);
bos.write(buf, 0, count);
}
bos.close();
// Get the compressed data
byte[] uncompressedData = bos.toByteArray();
return uncompressedData;
}
This is the result:
Input to Hex 6a61766120666f72756d732121 13
deflated to hex 78dacb4a2c4b5448cb2f2acd2d56540400229504a1 21
inflated to Hex 6a61766120666f72756d732121 13
inflated to String java forums!! 13
Maybe I am totally wrong but as you see the deflated lenght is longer than the input, am I doing this relation right?
Thanks!!
# 5
Like I said: there is no guarantee that the compressed data is smaller than the uncompressed data, especially when you're trying to compress small amounts of data. Try compressing larger Strings.
