A challenge for anyone who wants one : )

Hi, I am trying to write a method that basically takes in a vector containing short objects which have variable legnth codes in them. I want to manipulate these shorts and create a new Vector of shorts that contain fixed length code versions of these;

Right now you can think of the shorts as overlapping on its neighbour in the vector, some of the bits are spilling over into the next short.

Heres an example to make it clearer

every short should have two parts, a 3 bit 'id' that tells the algorithm how many bits folling these 3 bits are to be associated with the id. The id can be used to get this number of bits from an array:

indexlengths[] = {7, 6, 8, 5, 4, 7, 9, 8}

so, when the algorithm encounters id #2 (the in the 3 leftmost bits) it looks in this array in position 2 and then knows that following the id is 6 bits to be extracted. once this has been extracted a new short should created and, these 6 bits should be added but shifted over to the very right of the short(shifted 13-6 in this case) and this id should added but to the very leftmost 3 bits.

so:

0101111110100000

(id:2)(6 bits)

should go to:

0100000000111111

now this wouldn't be too much of a problem BUT the remaining bits that were discarded (in this case 0100000) must be kept and plugged to the next short in the vector

so if the next short in the vector is

1010000011111110

it will have this remainder from the previous short plugged into the right of it

010 0000101000001and 'pushed' off is:1111110

now this process must continue making sure that this remainder that was 'pushed' off in plugged in to the next short BUT FIRST the process that we started with must be applied, a new short is created(for which the resulting fixed legnth code will be stored in), the ID is extracted from the leftmost 3 bits (id: 0) of the current short and the number of bits to follow is found from the above array indexLengths[0] which is 7 bits. these 7 bits are shifted by (16-3)-7 = 6 positions to the right so they are stored in to the rightmost part of this new short and the id will stay at the leftmost 3 bits:

010 0000000000101 'pushed off: 000001'

so now we have both 1111110 and 000001 'pushed' off of the same short, as the length of bits of this remainder gets longer and reaches 16, a new short should be created and added to the result vector.

One more example just to show it visually:

this time the indexes after the 'id' part are all 6

int[] indexLengths = {6, 6, 6, 6, 6, 6, 6, 6};

a vector of 3 shorts:

111 1000011111111, 011 0111111100000, 101 1000000000000,

results in:

111 0000000100001, 111 1111011 011111, 110 00001011000000, 000000

looking at the first resulting short:

(id) (6 bits that followed the index shifted to the right most 6 bits)

1110000000100001

the remiander that had to be 'pushed off' is inserted in to the start of the next short and the process starts again.

I know this is alot to ask, but any tips would be great, java isn't being any help when working with bits and shorts, for example when extracting the id (using a mask and then shifting to the right 13 places) I get negative numbers sometimes!

Any suggestions on how to break this down into a managable problem to solve would be great too : )

Thanks!

Message was edited by:

digiology

[3479 byte] By [digiologya] at [2007-11-26 19:53:35]
# 1

this might help, heres a method doing the opposite, turning fixed legnth codes into variable length codes, the whole idea of this method is to splim down the array of shorts before sending it in a message. This works fine, however the problem above of doing it the other way around has caused me alot of grief after 3 attempts!

public Vector convertVectorToSlimmerVector(Vector v)

{

System.out.println("initial:");

for(int i=0;i<v.size();i++)

{

System.out.print(toBinaryString(((Short)v.get(i)).shortValue())+", ");

}

System.out.println();

Vector result = new Vector();

//assume vector starts at message part for now

//seems to be fine for legnths

int[] indexLengths = {6, 6, 6, 6, 6, 6, 6, 6}; //this will be passes as a parameter or stored in the incomming vector later!

// above values cant be greater than 13!

short mask1 = 0x1;

short mask2 = 0x3;

short mask3 = 0x7;//000 0000000000111;

short mask4 = 0xF; //000 0000000001111;

short mask5 = 0x1F; //000 0000000011111;

short mask6 = 0x3F; //000 0000000111111;

short mask7 = 0x7F; //000 0000001111111;

short mask8 = 0xFF; //000 0000011111111;

short mask9 = 0x1FF; //000 0000111111111;

short mask10 = 0x3FF; //000 0001111111111;

short mask11 = 0x7FF; //000 0011111111111;

short mask12 = 0xFFF; //000 0111111111111;

short mask13 = 0x1FFF;//000 1111111111111;

short mask14 = 0x3FFF;//001 1111111111111;

short mask15 = 0x7FFF;//011 1111111111111;

//short mask16 = 0xFFFF;//111 1111111111111;

long longMask1 = 0x8000;//100 0000000000000;

long longMask2 = 0xC000;//110 0000000000000;

long longMask3 = 0xE000;//111 0000000000000;

long longMask4 = 0xF000;

long longMask5 = 0xF800;

long longMask6 = 0xFC00;

long longMask7 = 0xFE00;

long longMask8 = 0xFF00;

long longMask9 = 0xFF80;

long longMask10 = 0xFFC0;

long longMask11 = 0xFFE0;

long longMask12 = 0xFFF0;

long longMask13 = 0xFFF8;

long longMask14 = 0xFFFC;

long longMask15 = 0xFFFE;

//others omitted for now

Vector redundantShortPositions = new Vector();

//short lengthMask=0xE000;

long lengthMask = (long)0xF000; //111 0000000000000;

short[] masks = {mask1, mask2, mask3, mask4, mask5, mask6, mask7, mask8, mask9, mask10, mask11, mask12, mask13, mask14, mask15};

long[] longMasks = {longMask1, longMask2, longMask3, longMask4, longMask5, longMask6, longMask7, longMask8, longMask9, longMask10, longMask11, longMask12, longMask13, longMask14, longMask15};

short currentShort;

Short currentShortObject;

long currentWordLength;

int indexLength;

int currentBitLength;

int numberOfCarriedBits = 0;

int numberOfBitsToSpillOver = 0; //number of bits that cant be stored in this iteration and will spill over into the next

int numberOfBitsLeftOver = 0; //number of redundant bits at the end of the current byte(/short?)

short valueCarriedOver = 0;

long currentLong;

String currentProgress;

Short lastShort = null;

short l;

System.out.println();

for(int index=0; index><v.size(); index++)

{

System.out.println("index:"+index);

currentShort = ((Short)v.get(index)).shortValue();

currentProgress = toBinaryString(currentShort);

System.out.println("progress: "+currentProgress);

currentShortObject = (Short)(v.get(index));

currentLong = currentShortObject.longValue();

currentWordLength = (long)(currentLong & lengthMask);

currentWordLength = (currentWordLength>>13);

//currentShort = (short)currentLong;

//short l2 = (short)currentWordLength;

indexLength = indexLengths[(int)currentWordLength]; //make sure that the index length for word legnth 0 is stored above somewhere

//System.out.print("isolated legnth: "+toBinaryString((short)currentWordLength)+", ");

//l = (short)(currentShort & lengthMask);

//currentWordLength = (short)(currentShort & lengthMask);

//l = (short)currentWordLength;

//currentWordLength >> 13;

//System.out.println(toBinaryString((short)l));//currentWordLength));

//that 1 is added to it

if(currentWordLength != 0)

{

currentShort = (short)(currentShort & masks[indexLength-1]); //extract index

currentShort = (short)(currentShort << ((16-3)-indexLength));//shift index over (just 3 bits from the left most bit, next to where the length will go)

//number of bits from this short placed in previous short

currentBitLength = (3 + indexLength + numberOfCarriedBits)-numberOfBitsLeftOver;

currentWordLength = currentWordLength<<13;

currentShort = (short)(currentWordLength + currentShort);

currentProgress = toBinaryString(currentShort);

System.out.println("progress: "+currentProgress);

if(index!=0)

{

//number of free bits from last short is > 0

if(numberOfBitsLeftOver > 0)

{

System.out.println("last short: "+toBinaryString(lastShort));

System.out.println("current short: "+toBinaryString(currentShort));

//isolate left most bits from current short that can be stored in previous short

short tmp = (short)(currentShort & longMasks[numberOfBitsLeftOver-1]);

System.out.println("tmp after mask: "+toBinaryString(tmp));

//shift over to rightmost position

tmp = (short)(tmp >> (16-numberOfBitsLeftOver));

tmp = (short)(tmp & masks[numberOfBitsLeftOver-1]);

System.out.println("tmp: "+toBinaryString(tmp));

lastShort = new Short((short)(lastShort.shortValue() + tmp));

System.out.println("lastshort fter spillage: "+toBinaryString(lastShort));

//v.setElementAt(lastShort,(index-1));

//

//v.remove(index-1);

//v.add(index-1,lastShort);

result.setElementAt(lastShort,(index-1)); //could be buggy (check with dict legnths of 2!)

currentShort = (short)(currentShort << numberOfBitsLeftOver);

}

}

currentProgress = toBinaryString(currentShort);

System.out.println("progress after spilling to previous: "+currentProgress);

//if(currentBitLength <= 8)

//{

//numberOfBitsLeftOver = 8 - currentBitLength;

//}

//else

{// if currentBitLength > 8

numberOfBitsLeftOver = 16 - currentBitLength;

if(numberOfBitsLeftOver < 0)

{

numberOfBitsToSpillOver = Math.abs(numberOfBitsLeftOver);

numberOfBitsLeftOver = 0;

}

else

{

numberOfBitsToSpillOver = 0;

}

short pluggedInValue = (short)(valueCarriedOver << (16-numberOfCarriedBits)); //shift value to appropriate left most position

if(numberOfCarriedBits > 0)

{

//isolate part of currentShort that will be 'pushed off' to the right

short temp = (short)(currentShort & masks[numberOfCarriedBits-1]);

valueCarriedOver = temp;

}

else

{

valueCarriedOver = 0;

}

currentShort = (short)(currentShort >> numberOfCarriedBits);

currentShort = (short)(pluggedInValue + currentShort);

lastShort = new Short(currentShort);

numberOfCarriedBits = numberOfBitsToSpillOver;

currentProgress = toBinaryString(currentShort);

System.out.println("number of carried bits: "+numberOfCarriedBits);

System.out.println("num bits left over: "+numberOfBitsLeftOver);

System.out.println("progress: "+currentProgress);

//System.out.print(toBinaryString(currentShort)+", ");

}

}

else

{// currentWordLength = 0

System.out.println("word length is zero");

}

if(numberOfBitsLeftOver >= 16)//hopefully should be ok, remove condition if not and remove redundant shorts afterwards

{

//reset this value and dont add current shot

numberOfBitsLeftOver = numberOfBitsLeftOver - 16;

//System.out.println("BOOLEAN");

//System.out.println(toBinaryString((Short)result.get(index-1)));

//result.add(new Boolean(false)); //just to act as a flag

System.out.println("position of redundantness? "+index);

//store position of redundant short

redundantShortPositions.add(new Integer(index));

}

//else

//{

//result.add(new Short(currentShort));

//}

result.add(new Short(currentShort));

}

//remove redundant shorts

int offset = 0;

for(int j=0;j<redundantShortPositions.size();j++)

{

int p = ((Integer)(redundantShortPositions.get(j))).intValue();

result.remove(p-offset);

offset++;

}

System.out.println("result:");

for(int i=0;i<result.size();i++)

{

System.out.print(toBinaryString(((Short)result.get(i)).shortValue())+", ");

}

return result;

}

here my crappy attempt at solving the first mentioned problem (converting from variable length to fixed legnth):

public Vector convertToFatterVector(Vector v)

{

System.out.println("initial:");

for(int i=0;i<v.size();i++)

{

System.out.print(toBinaryString(((Short)v.get(i)).shortValue())+", ");

}

short mask1 = 0x1;

short mask2 = 0x3;

short mask3 = 0x7;//000 0000000000111;

short mask4 = 0xF; //000 0000000001111;

short mask5 = 0x1F; //000 0000000011111;

short mask6 = 0x3F; //000 0000000111111;

short mask7 = 0x7F; //000 0000001111111;

short mask8 = 0xFF; //000 0000011111111;

short mask9 = 0x1FF; //000 0000111111111;

short mask10 = 0x3FF; //000 0001111111111;

short mask11 = 0x7FF; //000 0011111111111;

short mask12 = 0xFFF; //000 0111111111111;

short mask13 = 0x1FFF;//000 1111111111111;

short mask14 = 0x3FFF;//001 1111111111111;

short mask15 = 0x7FFF;//011 1111111111111;

//short mask16 = 0xFFFF;//111 1111111111111;

long longMask1 = 0x8000;//100 0000000000000;

long longMask2 = 0xC000;//110 0000000000000;

long longMask3 = 0xE000;//111 0000000000000;

long longMask4 = 0xF000;//111 1000000000000;

long longMask5 = 0xF800;

long longMask6 = 0xFC00;

long longMask7 = 0xFE00;

long longMask8 = 0xFF00;

long longMask9 = 0xFF80;

long longMask10 = 0xFFC0;

long longMask11 = 0xFFE0;

long longMask12 = 0xFFF0;

long longMask13 = 0xFFF8;

long longMask14 = 0xFFFC;

long longMask15 = 0xFFFE;

short[] masks = {mask1, mask2, mask3, mask4, mask5, mask6, mask7, mask8, mask9, mask10, mask11, mask12, mask13, mask14, mask15};

long[] longMasks = {longMask1, longMask2, longMask3, longMask4, longMask5, longMask6, longMask7, longMask8, longMask9, longMask10, longMask11, longMask12, longMask13, longMask14, longMask15};

Vector result = new Vector();

int[] indexLengths = {6, 6, 6, 6, 6, 6, 6, 6};

long lengthMask = (long)0xF000;

//long tmp = 0xE000;

//short lengthMask = (short)tmp;//111 0000000000000;

short indexMask = 0x1FFF; //000 1111111111111;

System.out.println();

System.out.println("length mask: "+toBinaryString((short)lengthMask));

System.out.println();

short currentShort;

long currentLong;

String currentProgress;

Short currentShortObject;

long currentWordLength;

int indexLength;

//int numberOfBitsSpillingOver=0;

short spillValue = 0;

Short lastShort;

int currentBitLength;

int numBitsCarriedOver=0;

short firstShort;

short secondShort;

long firstLong;

long secondLong;

short valueThatWasSpilledOver=0;

short valueThatWillSpillOver=0;

for(int index=0;index<v.size();index++)

{

currentShort = ((Short)v.get(index)).shortValue();

currentLong = ((Short)v.get(index)).longValue();

if(index == 0)

{

currentWordLength = (short)(currentShort >> 13);

//currentWordLength = (long)(currentShort & lengthMask);

//System.out.println("currentShort: ");

//

//

//currentWordLength = (long)(currentWordLength>>13);

System.out.println("current word legnth: " + toBinaryString((short)currentWordLength));

indexLength = indexLengths[toAbsoluteIntValue((short)currentWordLength)];//(int)currentWordLength];

//numBitsCarriedOver = (int)(13-indexLength);//////

}

//valueThatWillSpillOver = (short)(masks[numBitsCarriedOver-1]&currentShort);

currentWordLength = (valueThatWasSpilledOver<<(16-numBitsCarriedOver)&lengthMask);

currentWordLength = (short)(currentWordLength>>13);

//dont forget might need to be set somewhere!

currentShort = (short)((currentShort>>numBitsCarriedOver)+(valueThatWasSpilledOver<<(16-numBitsCarriedOver)));

System.out.println("progress after spilling from previous:"+toBinaryString(currentShort));

//currentShort should now have the length and index contained in it but the index is shifted to the left

indexLength = indexLengths[(int)currentWordLength];

numBitsCarriedOver = (int)(13-indexLength);

valueThatWillSpillOver = masks[13-indexLength];

valueThatWasSpilledOver = valueThatWillSpillOver; //?

//get index(shifted to the right) on its own

short temp = (short)((currentShort & indexMask)>>(13-indexLength));

System.out.println("index on its own shifted to the right: "+toBinaryString(temp));

//get length on its own

short temp2 = (short)(currentShort & lengthMask);

System.out.println("length on its own: "+toBinaryString(temp2));

currentShort = (short)(temp + temp2);

System.out.println("progress after shifting index to the right: "+toBinaryString(currentShort));

result.add(new Short(currentShort));

}

System.out.println("result");

for(int i=0;i<result.size();i++)

{

System.out.print(toBinaryString(((Short)result.get(i)).shortValue())+", ");

}

>

digiologya at 2007-7-9 22:45:27 > top of Java-index,Java Essentials,Java Programming...