Encryption improvement...

Okay, I've been making a simple little encryption program and I'm currently trying to improve it.

The reason i'm posting it here is that i've found a few problems with it and need some advice in how to improve it.

Problems i've found are:

1. The ciphertext is at least 3x larger then the plaintext :( probably solve with general compression but perhaps there is a obvious specialized method I can't see...

2. Small variations of the key ( swapping of numbers next to each other) can lead to similiar plaintexts' ( not identical.. )

My main concern is 2. and id also appreciate any other comments you have on it in general! :)

i'm not going to post it within the "code" tags so that it can be easily copied.

following is the key ( contained in a file called "keyfile.mkf" ) ( line breaks added so page doesn't wrap):

749285748269834562928973637928386436869586982968268926369459638896269298

292987263947865928346592873694876598276983649582734527929757246766428472

662469872523455627865928569238452345988796928356569298626394656892966928

356928365928365239859642569254978243969298725696972349562349758629354626

262698356982354965987689569827954698236968268962698266982398694596985689

268968926893946856567629234756298376266272349586595958726926269347566246

956925949823546293865923562934765928569595982765923856923456234959254984

325698256985429826629834659285923856923495959958726392634956236269868293

459234785629345629356266262637495692835692386592835692874592662692836592

876592845925959592873465928374659286459236262672893458269348759595959823

476592386529626266263749529834659238759595992837465923865926262782662966

928346582983659238569283465923846526626262634854964329824958762394856239

487659238542983645982395496823996829686892369869234582634656923495629346

526394985269529835629385692385234529384569293849569823492649596824398692

9562956892596843

And here is the code:

import java.io.*;

import java.math.*;

import java.util.*;

/*

Author: Michael Silk, (c) 2003.

*/

public class Moozy {

private static final String HEADER = "mef!";

public final static boolean TEXT_OUTPUT = true;

public final static boolean FILE_OUTPUT = false;

private final static String HELP = "\nexamples: " +

" \n\t-s -d -in file1\t\t\t prints the result of decrypting file1" +

" \n\t-f -d -in file1 -out file2\t puts the result of decrypting file1 into file2" +

" \n\t-f -e -in file1 -out file2\t encrypts the contents of file1 into file2" +

" \n==========================================================================================" +

" \nhence we arrive at the definitions of: " +

" \n\t-s\t\t specifies that output goes to the screen" +

" \n\t-d\t\t specifies that the input is for decryption" +

" \n\t-e\t\t specifies that the input is for encryption" +

" \n\t-in\t\t specifies a file for input" +

" \n\t-out\t\t specifies a file for output";

private final static String PARAM_ENDER = " \"Moozy \\?\" for a list of commands and examples.";

private final static String PARAM_NO_OUTPUT_METHOD = "\tNo output method was specified please try again or " +

" \n\tsee " + PARAM_ENDER;

private final static String PARAM_NO_HANDLING = "\tNo mode [ -e or -d ] specified, I cannot handle the file without " +

" \n\***! please see " + PARAM_ENDER;

private final static String PARAM_NO_OUTPUT_FILENAME = "\tPARAM_NO_OUTPUT_FILENAME";

private final static String PARAM_NO_INPUT_FILENAME = "\tPARAM_NO_INPUT_FILENAME";

private static final int MODE_SCREEN_OUTPUT = 1;

private static final int MODE_FILE_OUTPUT = 2;

private static final int MODE_DECRYPT_INPUT = 4;

private static final int MODE_ENCRYPT_INPUT = 8;

private static byte BLOCK_IDENTIFIER[] = { 0x1, 0x1, 0x1, 0x1, (byte) 0xff, (byte) 0xff, (byte) 0xff };

private static String randomOutputFile(){ return "tmp" + Integer.toHexString(new Random().nextInt(100000)+1) + ".txt"; }

public static int toInt( byte c ){ return Integer.parseInt( (char) c + "" ); }

public static int[] strip( int me[] ){

int firstZero = -1;

for( int k = 0; k < me.length; ++k )

if( me[k] == 0 ){

firstZero = k;

break;

}

if( firstZero == -1 )

return me;

int stripped[] = new int[firstZero++];

for(int k = 0; k < stripped.length; ++k)

stripped[k] = me[k];

return stripped;

}

private static int has(String me[], String word){

int l = me.length;

for( int p = 0; p < l; ++p )

if( me[p].equals(word) )

return p;

return -1;

}

private static void generateKey() throws IOException {

String outfile = "keyfile.mkf";

// coming soon

}

public static void main(String args[]) throws IOException {

// args = new String[]{ "-s", "-d", "-in", "out.java" };

args = new String[]{ "-f", "-e", "-in", "Moozy.java", "-out", "out.java" };

String infile;

String outfile = "";

int runmode = 0;

if(args.length < 3){

System.err.println(HELP);

return;

}

if(has(args, "-s") == -1 && has(args, "-f") == -1){

System.err.println(PARAM_NO_OUTPUT_METHOD);

return;

}

if(has(args, "-d") == -1 && has(args, "-e") == -1){

System.err.println(PARAM_NO_HANDLING);

return;

}

if(has(args, "-f") != -1){

int outi = has(args, "-out");

if(outi == -1 || (args.length == outi + 1)){

System.err.println(PARAM_NO_OUTPUT_FILENAME);

return;

}

outfile = args[outi + 1];

runmode += MODE_FILE_OUTPUT;

}else{

runmode += MODE_SCREEN_OUTPUT;

outfile = randomOutputFile();

}

if(has(args, "-in") != -1){

int ini = has(args, "-in");

if(ini == -1 || (args.length == ini + 1)){

System.err.println(PARAM_NO_INPUT_FILENAME);

return;

}

infile = args[ini + 1];

}else{

System.err.println(" -in not specified ");

return;

}

if(has(args, "-e") == -1)

runmode += MODE_DECRYPT_INPUT;

else

runmode += MODE_ENCRYPT_INPUT;

Moozy mo;

try{

mo = new Moozy(infile, outfile, runmode);

}catch(IllegalArgumentException ae){

System.err.println(ae);

return;

}

if(!mo.assignKey())

return;

mo.process();

}

private int mode;

private int key[];

private String strinf;

private String outf;

private FileOutputStream outs;

private FileInputStream ins;

private FileOutputStream otherOut;

public BigInteger bigkey;

public int outcount;

public StringBuffer buf;

Moozy(String inf, String outf, int mode){

this.mode = mode;

this.outf = outf;

openFile(inf);

createNewFile(outf);

outcount = 0;

buf = new StringBuffer( 1026 );

}

public boolean assignKey() throws IOException {

FileInputStream keyin = new FileInputStream("keyfile.mkf");

byte keybuf[] = new byte[keyin.available()];

if((keybuf.length & 1) == 1){

System.out.println("Invalid key.");

return false;

}

keyin.read(keybuf);

int keyvalue = 0;

int keyindex = 0;

int tempkey[] = new int[keybuf.length];

for( int k = 0; k < keybuf.length; ++k ){

keyvalue = toInt( keybuf[k] );

while( (keyvalue == 0 || keyvalue == 1) && k < keybuf.length ){

keyvalue += toInt( keybuf[++k] );

}

tempkey[keyindex++] = keyvalue;

}

keyin.close();

key = strip( tempkey );

StringBuffer tmp = new StringBuffer(key.length);

for(int z = 0; z < key.length; z++)

tmp.append(key[z]);

bigkey = new BigInteger(new String(tmp));

return true;

}

private void createNewFile(String filename){

File file = new File( filename );

try {

file.createNewFile();

outs = new FileOutputStream(file);

}catch(IOException ioe){

throw new IllegalArgumentException("Read error: " + ioe.toString());

}

}

private void openFile(String filename){

File file;

try {

file = new File(filename);

ins = new FileInputStream(file);

}catch(FileNotFoundException fnfe){

throw new IllegalArgumentException("Something odd: " + fnfe.toString());

}

}

public void process() throws IOException {

if((mode & MODE_DECRYPT_INPUT) == MODE_DECRYPT_INPUT)

mode_decryptInput();

else

mode_encryptInput();

if((mode & MODE_SCREEN_OUTPUT) == MODE_SCREEN_OUTPUT){

FileInputStream outs = new FileInputStream(outf);

byte data[] = new byte[outs.available()];

outs.read( data );

for( int k = 0; k < data.length; ++k )

System.out.print( (char) data[k] );

System.out.println("\n");

}

}

private void mode_decryptInput() throws IOException {

byte header[] = new byte[HEADER.length()];

ins.read(header);

String headers = new String(header);

if(!headers.equals(HEADER)){

System.out.println("Cannot process this file for decryption.");

return;

}

byte bad[] = new byte[ins.available()];

ins.read(bad);

decode(bad);

}

private void mode_encryptInput() throws IOException {

byte bad[] = new byte[ins.available()];

ins.read(bad);

outs.write(HEADER.getBytes());

encode(bad);

}

private void encode( byte me[] ) throws IOException {

StringBuffer waitingFor1k = new StringBuffer();;

int keyCharIndex = 0;

int keyLength = key.length;

for(int encodeIndex = 0; encodeIndex < me.length; encodeIndex++){

if(keyCharIndex == keyLength)

keyCharIndex = 0;

takeDown( me[encodeIndex], keyCharIndex );

++keyCharIndex;

}

outs.close();

}

public void takeDown( byte it, int keyindex ) throws IOException {

int times = key[keyindex] - 1;

final int keylen = key.length-1;

int finalResult = it & 0xff;

int currentValue = 0;

boolean zero_d = false;

String tmp;

Random x = new Random();

for(int k = 0; k < times; ++k){

currentValue = (zero_d) ? x.nextInt(30)+1 : x.nextInt(finalResult >> 1);

finalResult -= (zero_d) ? 0 : currentValue;

if( currentValue == 0 ){

currentValue = 0xff;

zero_d = true;

}

currentValue ^= key[keylen - keyindex];

outs.write( (byte) currentValue );

++outcount;

}

finalResult ^= key[keylen - keyindex];

outs.write( (byte) finalResult );

++outcount;

}

private String toString( byte bytes[] ){

StringBuffer buf = new StringBuffer( bytes.length * 3 );

for(int x = 0; x < bytes.length; ++x){

buf.append( (bytes[x] & 0xff) + "" );

}

return new String( buf );

}

public void decode( byte me[] ) throws IOException {

int chartotal = 0;

int keyIndex = 0;

int keyCount = 0;

int keylen = key.length-1;

while( true ){

if(keyCount == key.length)

keyCount = 0;

if(keyIndex >= me.length)

break;

int toget = key[keyCount];

int rtmp = 0;

for( int k = 0; k < toget; ++k ){

int tmp = me[keyIndex + k];

tmp ^= key[keylen - keyCount];

if(tmp == -1 && key[keyCount] != 1){

k = toget-1;

rtmp = me[keyIndex + k];

rtmp ^= key[keylen - keyCount];

chartotal += rtmp;

break;

}else{

rtmp = me[keyIndex + k];

rtmp ^= key[keylen - keyCount];

chartotal += rtmp;

}

}

keyIndex += key[keyCount++];

outs.write( (byte) chartotal );

chartotal = 0;

}

}

}

[13936 byte] By [silkma] at [2007-9-28 9:14:10]
# 1

I'm no Bruce Schneier, but your crypto doesn't look so good.

... probably solve with general compression

I hope not.

Compression algorithms work by finding patterns in non-random data.

A good test of a random data generator is to see how incompressable it's output is.

A good preliminary test of an encryption algorithm is to see how random it's output is.

Small variations of the key ( swapping of numbers next to each other) can lead to similiar plaintexts'

Not good.

Look into Feistel ciphers. They're always good for homebrew crypto. And straightforward to implement.

If you have some time, read this (free online):

http://www.cacr.math.uwaterloo.ca/hac/

mgbolusma at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 2

> Small variations of the key ( swapping of numbers next to each other)

> can lead to similiar plaintexts'

> Not good.

Yeah, hence the reason why I posted it.

My solution to this was to buffer 1kb of semi-encrypted data and xor that with the 1kb key ( both stored in big integer objects ) but this got a little complicated and I started to wonder why bother doing the first part of the encryption at all if I was going to do that. However perhaps that is the best way to go...?

silkma at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 3

Like I said, a Feistel cipher is very easy to understand and implement once you understand how it works. Basically it acts as a framework whos encryption works based on a one way transformation you implement (can be ANYTHING you want) and an initial key set (if desired).

It's very flexible.

Really worth looking into.

mgbolusma at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 4

> Okay, I've been making a simple little encryption

> program and I'm currently trying to improve it.

The question is: why?

I've taken a graduate course in cryptography/security and I can tell you that developing secure encryption algorithms is HARD. In fact, current encryption algorithms are only released after significant amounts of peer scrutiny. Why 'roll your own' when there are publicly available implementations that are proven secure and readily available (DES, etc)?

This may be interesting as an intellectual exercise but I wouldn't plan on trusting the security of any data encrpyted with your program unless you have a PhD in CS and have worked extensively in cryptography.

meadandalea at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 5

> > Okay, I've been making a simple little encryption

> > program and I'm currently trying to improve it.

>

> The question is: why?

It's fun and it allows you to learn. I don't think he plans on using it for any practical purpose. (=

> I've taken a graduate course in cryptography/security

> and I can tell you that developing secure encryption

> algorithms is HARD. In fact, current encryption

> algorithms are only released after significant amounts

> of peer scrutiny. Why 'roll your own' when there are

> publicly available implementations that are proven

> secure and readily available (DES, etc)?

This is slightly off topic but do you know of any good resources on Cryptography outside of Applied Cryptography and The Codebreaker? I'm planning on writing a research paper on crypto, and those seem to be the only two books that get mentioned...

Zemthematress42a at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 6

http://www-cse.ucsd.edu/users/mihir/cse207/index.html

Above is a link to course notes from a more recent course by the professor that I studied with. This guy is doing some great research and you could probably find some other great material on his webpage.

The course covers the mathematics behind cryptanalysis as well as cryptographic theory. It focuses on what he calls "provable security".

meadandalea at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 7
Wow, Thank you, that looks great!
Zemthematress42a at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 8

> > The question is: why?

> It's fun and it allows you to learn. I don't think he plans on using

> it for any practical purpose. (=

Yup, the matress is right, for real usage i've decided to try and implement Rijndael; problem is that the paper on it from the AES website is wrong! anyway ...

silkma at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 9
Possible source for Rijndael: http://www.esat.kuleuven.ac.be/~rijmen/rijndael/
GiddingsRa at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 10

> I've taken a graduate course in cryptography/security ...

> there are publicly available implementations that are proven secure and readily available (DES, etc)?

I took a double-take at that one. DES is proven insecure by the simple mechanism of someone having built a machine to crack DES keys in a couple of hours. I haven't heard that 3DES has been broken yet, but I wouldn't trust it myself for anything important.

YATArchivista at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 11

> I took a double-take at that one. DES is proven

> insecure by the simple mechanism of someone having

> built a machine to crack DES keys in a couple of

> hours. I haven't heard that 3DES has been broken yet,

> but I wouldn't trust it myself for anything important.

First of all. Nothing is secure. Every encryption can be broken. Yes, that's right, every encryption can be broken (this assumes your definition of 'broken' means recovering the plaintext). The variable is time. How long will it take and at what cost?

Even 'provably' secure algorithms assume reasonableness in the face of bruteforce attacks. As CPU speed increases, brute force attacks become more and more an issue which is why keys are getting longer.

Many encryption algorithms suffer from the fact that brute force attacks on the key space are parallizeable and assessments of the security they provide are weighed in the face of current technology and cost: what's the liklikhood that the person wanting to break my key has access to 2^64 or 2^128 cycles?

Yes, DES isn't the best out there right now. So use AES, blowfish or 3DES. I was simply giving AN example to the poster.

meadandalea at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 12

> Nothing is secure. Every encryption can be broken.

As a (the only) counterexample, I present the one-time pad.

> Even 'provably' secure algorithms assume reasonableness in the face of bruteforce attacks.

Sure, but it's possible to do better than brute force against DES. Admittedly only by a factor of 2^12, but the keyspace is only 2^56 so I wouldn't discount it.

By the way, I approve of the quotes round "provably" - not only do good algorithms assume that their keyspace is too large for brute force, but they also assume that there won't be any major theoretical breakthroughs tomorrow.

YATArchivista at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 13

> > Nothing is secure. Every encryption can be broken.

>

> As a (the only) counterexample, I present the one-time

> pad.

>

Maybe I should have said "nothing 'usable' is completely secure" .

Since key exchange is the primary insecurity in symmetric encryption, how does it help the matter by requiring that you send a key WITH EVERY MESSAGE YOU SEND or, pregenerate and exchange all of the keys for a communication session beforehand?

I think that we could agree that the overhead for using OTP makes it untenable as a useful encryption scheme for people to exchange messages?

meadandalea at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 14
In most cases, yes.
YATArchivista at 2007-7-11 22:10:25 > top of Java-index,Other Topics,Algorithms...
# 15
> I'm no Bruce Schneier, but...He is one cool guy - I can highly recommend his Crypto-Gram newsletter, if you're interrested in aspects on security: http://www.counterpane.com/crypto-gram.htmlKind regards from one of the fans ;)
kvolsa at 2007-7-18 19:53:24 > top of Java-index,Other Topics,Algorithms...
# 16

> First of all. Nothing is secure. Every encryption can

> be broken. Yes, that's right, every encryption can be

> broken (this assumes your definition of 'broken' means

> recovering the plaintext). The variable is time. How

> long will it take and at what cost?

What about xor with random data the length of the message? Maybe infeasible but impossible to recover the plaintext.

nrlza at 2007-7-18 19:53:24 > top of Java-index,Other Topics,Algorithms...
# 17

> ...xor with random...

Well, if it's random, how will you recover the encrypted message...?

If it's pseudo-random, with a password-based seed, I suggest that you use ROT13 instead. Then you at least don't have to bother anyone with a password... :))

Xor-encryption is trivial - check out http://forum.java.sun.com/thread.jsp?thread=332027&forum=9&message=1363641

kvolsa at 2007-7-18 19:53:24 > top of Java-index,Other Topics,Algorithms...
# 18
Quantum cryptography will be fairly secure, though there is always the fallible human element.
GiddingsRa at 2007-7-18 19:53:24 > top of Java-index,Other Topics,Algorithms...
# 19

> > ...xor with random...

>

> Well, if it's random, how will you recover the

> encrypted message...?

>

> If it's pseudo-random, with a password-based seed, I

> suggest that you use ROT13 instead. Then you at least

> don't have to bother anyone with a password... :))

>

> Xor-encryption is trivial - check out

> http://forum.java.sun.com/thread.jsp?thread=332027&foru

> =9&message=1363641

randomly generated != unknown to the user

do a google search for one time pad

amishslayera at 2007-7-18 19:53:24 > top of Java-index,Other Topics,Algorithms...
# 20

> > ...xor with random...

>

> Well, if it's random, how will you recover the

> encrypted message...?

>

> If it's pseudo-random, with a password-based seed, I

> suggest that you use ROT13 instead. Then you at least

> don't have to bother anyone with a password... :))

Well you have to create the random bits first and make sure both parties have the same set of random bits. Then the other party can use their set of random bits to recover the message.

And you can try using hardware to make purely random numbers.

nrlza at 2007-7-18 19:53:24 > top of Java-index,Other Topics,Algorithms...
# 21
> If it's pseudo-random, with a password-based seed, I> suggest that you use ROT13 instead. Then you at least> don't have to bother anyone with a password... :))Pseudo-random with a password-based seed is much stronger than ROT13.
nrlza at 2007-7-18 19:53:24 > top of Java-index,Other Topics,Algorithms...
# 22
> And you can try using hardware to make purely random numbers.I have a circuit diagram somewhere for a circuit which, if you carefully tune certain components, behaves chaotically. Leave that running for an hour, then sample it and you'll do pretty well.
YATArchivista at 2007-7-18 19:53:24 > top of Java-index,Other Topics,Algorithms...
# 23

> Xor-encryption is trivial - check out

> http://forum.java.sun.com/thread.jsp?thread=332027&foru

> =9&message=1363641

Only if you either reuse the key, or if the key is short enough. If the key were truly random and the length of the document it'd be theoretically unbreakable, just like any one time pad....

Zemthematress42a at 2007-7-18 19:53:24 > top of Java-index,Other Topics,Algorithms...