Let's assume you downloaded a Linux installation package like this:
jdk-1_2_2_012-linux-i386.tar.gz
It is a gzipped (that is, a gzip-compressed) tar (tape archive) file.
You decide where you want to install the package, let us assume it is /usr/lib in order other users can access it too.
Then you gain root privileges (so that you can write the installation directory).
Then you go to the installation directory /usr/lib:
# cd /usr/lib
Then you say
# tartzvf/tmp/jdk-1_2_2_012-linux-i386.tar.gz
(assuming this very path name).
This will uncompress and "untar" the package.
Then you can put it into use either by expicitly calling java, javac etc. or by including /usr/lib/jdk1.2.2/bin into your PATH:
PATH=$PATH:/usr/lib/jdk1.2.2/bin
Hope this helped.