Library name compatibility 32/64 bit and across platforms
Hi all,
I have a numerical C library which I wish to link to using JNI. The binary will be shipped in the jar file to keep everything "clean" for the client.
Unfortunately the SUN System.mapLibraryName method is completely useless when it comes to bundling binaries that are specific to an architecture. Some tests I have performed show:
Linux 64 bit Pentium M: os.name = "Linux", os.arch = "amd64", mapLibraryName = "libNAME.so"
Linux 32 bit: os.name = "Linux", os.arch = "i386", mapLibraryName = "libNAME.so"
iBook G4: os.name = "Mac OS X", os.arch = "ppc", mapLibraryName = "libNAME.jnilib"
Windows XP 32 bit: os.name = "Windows XP", os.arch = "x86", mapLibraryName = "NAME.dll"
Sun Blade 1500 (64 bit): os.name = "SunOS", os.arch = "sparc", mapLibraryName = "libNAME.so"
you will notice that Linux 32/64 and Solaris all use "libNAME.so"... despite the architecture they are running on! (Apple is not a problem because it supports Universal binaries).
I'm thinking that I will have to use some horrible library name and loading code to deal with these problems... creating library names along the lines of
Linux 64 bit Linux: "libNAME-linux-intel-64.so"
Linux 32 bit Linux: "libNAME-linux-intel-32.so"
Linux 32 bit PPC Linux: "libNAME-linux-ppc-32.so"
iBook G4: "libNAME.jnilib"
Windows XP 64 bit: "NAME-64.dll"
Windows XP 32 bit: "NAME-32.dll"
Sun Blade 1500 (64 bit): "libNAME-sunos-64.so"
My main concern is Linux and Windows 32/64 bit, rather than cross-CPU handling.
I am not looking forward to writing this... has this problem come up before? And if so, what have people done to deal with it? I guess an alternative would be to use a naming convention in the directory structure, but that's really the same solution.
Kind regards,
Sam
[1872 byte] By [
fommila] at [2007-11-27 4:04:56]
# 1
Maybe I'm missing something, but why can't you put them in separate directories?
# 2
> Unfortunately the SUN System.mapLibraryName method is
> completely useless when it comes to bundling binaries
> that are specific to an architecture.
That has nothing to do with Java however. The point of that method is to correctly resolve to a shared library name. And it does that successfully and correctly on all platforms.
>
> I'm thinking that I will have to use some horrible
> library name and loading code to deal with these
> problems... creating library names along the lines
> of
>
No doubt.If you want to load different shared libraries then you will have to be able to differentiate them.
Again that has nothing to do with java - that is how the OSes do it.
If you wish to change that then Linux is open source so you can modify the source yourself. For Windows you will have to file a bug with MS and see what they think.
# 3
bschauwejava wrote:
> Maybe I'm missing something, but why can't you put them in separate directories?
It is not possible to put the files in separate directories with System.loadLibrary as it looks in the common places... it would be possible to do this with System.load however as it takes an absolute filename, but then you lose the benefits of path checking and have to manage locations of files manually.
# 4
> Again that has nothing to do with java - that is how> the OSes do it.It has everything to do with the implementation of Java... specifically the native method "System.mapLibraryName", not the OS itself.
# 5
I have concluded that the best way to solve this problem is to create a pre-processing method for System.loadLibrary that takes the library name (as it would be passed to System.loadLibrary) and returns a more operating system and architecture-friendly String.
I have created a class that does OS/Architecture detection and then returns the following on various platforms for the input parameter "name":
Apple (G3, G4, G5, Intel): "name" (expects user to build Universal Binaries)
Linux (i686): "name-linux-x86"
Linux (Intel/AMD 64): "name-linux-x86_64"
Linux (sparc): "name-linux-sparc"
Linux (PPC 32 bit): "name-linux-ppc"
Linux (PPC 64 bit): "name-linux-ppc_64"
Windows XP/Vista (i686): "name-windows-x86"
Windows XP/Vista (Intel/AMD 64): "name-windows-x86_64"
Sun Solaris (Blade): "name-sun-sparc"
Sun Solaris (Intel 64 bit): "name-sun-x86_64"
This String can then be safely sent to System.loadLibrary, which will do the usual additions of "lib", ".so", ".jnilib" and ".dll" and search in all the right places. It is not possible to write a wrapper for System.loadLibrary itself as native library loading needs to know the name of the calling class.
I'd be happy to share the code with anyone who requests, but it's a bit too long to post here.
Incidentally... in my research into native library loading, I encountered the "Bundle-NativeCode" entry for Manifest files. It may be possible to construct an entry for each os.name/os.arch pair to assign native binaries accordingly. However it appeared that every pairing would need to be specified (whereas my solution is more forgiving), and I'm not entirely sure if that method allows one to define the (shortened form of the) library name... it may just be useful for bundling binaries inside the jar file.
# 6
> > Again that has nothing to do with java - that is
> how
> > the OSes do it.
>
> It has everything to do with the implementation of
> Java... specifically the native method
> "System.mapLibraryName", not the OS itself.
No.
As you pointed out, one of your target OSes has a dual binary.
Two of the others do not.
The naming scheme is intended to resolve shared library naming.
It is not intended to resolve mixed model single implementation for multiple OSes.
Thus when you install a JVM (not your application) which uses shared libraries it correctly resolves to a single name. But it doesn't have the problem that you are because the install targets a single memory model.
And yet VMs support more than one memory model.
# 7
Note that you can also submit a change to allow java to do this. Or even code a solution yourself since the code is now or shortly will be open sourced.
# 8
> Note that you can also submit a change to allow java
> to do this.
>
> Or even code a solution yourself since the code is
> now or shortly will be open sourced.
It is a shame that System.load* were not coded with multiple architectures in mind but the behaviour of those methods is now well documented and widely used... it would be catastrophic to change it at this stage!
I already coded a simple solution in Java land, please read a few replies up. I don't see why you feel the solution should be to roll my own patched JVM... that's not ever an option.
# 9
> > Note that you can also submit a change to allow
> java
> > to do this.
> >
> > Or even code a solution yourself since the code is
> > now or shortly will be open sourced.
>
> It is a shame that System.load* were not coded with
> multiple architectures in mind but the behaviour of
> those methods is now well documented and widely
> used... it would be catastrophic to change it at this
> stage!
>
> I already coded a simple solution in Java land,
> please read a few replies up. I don't see why you
> feel the solution should be to roll my own patched
> JVM... that's not ever an option.
I suggested that if you considered this is important that you should take the initiative to have it added to the java API.
Myself I don't consider it important because for products that I work on that are cross platform compatible the installation process is complex enough that there is no point in delivering differently named libraries within the same deliverable, so there can be no naming problem.
