Accessing VolatileImage via Java native interface
Hello,
I currently try to integrate our high performance 2D drawing engine, which is based on X11, into a lightweight Java SWING container.
My first approach was to make usage of shared memory X11 pixmap within our drawing engine
and to transfer the rendering result via a memcpy into a MemoryImageSource, which is drawn
via Java 2D API.
This solution works fine, hower, especiially when using high resolutions of 2k x 2k, the copying
operation are not fast enough.
Therefore I think of a solution, which should be based on a VolatileImage and a direct copy
operation from an X11 pixmap into that VolatileImage. However, up to now I have not found
any interface to retrieve the internal pixmap ID of the VolatileImage (even the AWT native interface does not seem to work in this case) ,
Any help on this topic is appreciated.
Thanks in advance,
Oliver
[935 byte] By [
ok1a] at [2007-10-3 7:33:43]
Unfortunately there's no way to access VolatileImage's underlying pixmap.
(In some cases it may not even be a pixmap)
Have you tried using BufferedImage instead of MemoryImageSource?
You can get the data buffer of the BI, and then get the data array,
pass it to the native code and put your pixels into the array using JNI.
Then you can either copy this buffered image to the Swing's backbuffer directly or first copy it to another VolatileImage , and then to the backbuffer.
The latter would be useful if you want to cache the rendered data (that is, if you don't re-render the data on each Swing's repaint).
I've seen folks using this strategy successfully for video, may work for
you as well.
Thanks,
Dmitri
Java2D Team
Another thought: rendering to a shared memory pixmap will most likely be
unaccelerated by the X server, so you'll be using X11's software renderer
with system memory destination, so I wonder if you could try using directly
Java2D instead of your native renderer in the first place? May be it will
be more performant than your current approach.
Thanks,
Dmitri
In principle we perform exactly the same operations like you suggested with the MemoryImageSource and we are also using the animation and refresh feature. What should be the advantage of using a BufferedImage ?Thanks,Oliver
ok1a at 2007-7-15 2:33:39 >
Sorry, using Java2D directly is not a feasible option for the moment, but
might be an option for the future.
However, as far as I know the X11 rendering within Java2D has JNI overhead for
each call to any X11 draw request. Furthermore there are no advanced optimization
strategies which can be used for drawing of highly dynamic al objects in order to reduce
the necessary redraws.
Therefore I am still searching for a high performance solution based on the Pixmap approach as the interface between the two worlds (Java + external rendering engine).
Meanwhile, I had a look to the Java sources and found a way to retrieve the drawable
from the VolatileImage without the need to modify the Java sources.
However, my C/C++ code relies on some internal data structures of the Java sources
and is therefore not very compatible regarding different versions of the virtual machine.
Therefore it would be nice if the Java Native Interface for AWT could be extended by
the missing functionality. This would make life much easier to integrate native
rendering engines into the Java AWT/SWING environment.
Thanks,
Oliver
ok1a at 2007-7-15 2:33:39 >
> However, my C/C++ code relies on some internal data structures of the Java sources
and is therefore not very compatible regarding different versions of the virtual machine.
Yes, your code will be highly incompatible (we may change the implementation any time) and I would advise you against doing it.
Also, there's a potential for race conditions (our access to x11 display is synchronized).
> This would make life much easier to integrate native rendering engines into the Java AWT/SWING environment.
While I agree that it would, I believe it would be better if we addressed the issue of having the need for native renderers - whatever performance problems you have with our current X11 renderer should be solved instead..
Also, have you tried the OpenGL pipeline? It has much lower JNI overhead.
Thanks,
Dmitri
> What should be the advantage of using a BufferedImage ?We go through less hoops for rendering BufferedImages, so the overhead should be lower.Thanks, Dmitri
>> What should be the advantage of using a BufferedImage ?> We go through less hoops for rendering BufferedImages, so the overhead should be lower.Sounds good, so I will try it out and I'm curious about it.Thanks,Oliver
ok1a at 2007-7-15 2:33:39 >
>> However, my C/C++ code relies on some internal data structures of the Java sources
and is therefore not very compatible regarding different versions of the virtual machine.
>Yes, your code will be highly incompatible (we may change the implementation any time) and I would advise you against doing it.
You are right, maybe we will use the optimized solution only in rare cases as an
acceleration option and fallback to the BufferedImage solution in general.
>> This would make life much easier to integrate native rendering engines into the Java AWT/SWING environment.
>While I agree that it would, I believe it would be better if we addressed the issue of having the need for native renderers - whatever performance problems you have with our current X11 renderer should be solved instead..
In general your are right, but in our market it is important to have a drawing engine which
is highly optimized for a large number of dynamic/moving objects and to fullfill
operational display requirements. Therefore I cannot see a possibility to move to Java
rendering at the moment.
Maybe, I can find an alternative solution based on the ideas I got from the
accelerated implementation of GLJPanel from the JOGL project.
In combination with the latest GLX extension "EXT_texture_from_pixmap" it would
be possible to access an X11 pixmap as a texture and then make usage
of the Java2D/JOGLE bridge.
Thanks,
Oliver
ok1a at 2007-7-15 2:33:39 >
Looks like you're on the right track.Dmitri
Hello Dmitri,
currently I try to implement your proposed BufferedImage solution and
I am having problems to get the data array from the BufferedImage
in an efficient way. Furthermore, it seems to be the case that I always get
a copy of the DataBuffer and therefore I do not know how to refresh the
BufferedImage based on the modified DataBuffer.
It would be fine, if you could give me some more hints.
Thanks,
Oliver
ok1a at 2007-7-15 2:33:39 >
Hmm. You should be getting the same databuffer (and not a copy) if you're using the same BufferedImage and not creating a new one on every
refresh.
You don't have to "refresh" the BufferedImage since it just uses the DataBuffer as the storage for pixels - so whatever changes you make to the DB are immediately "available" to the BufferedImage.
Also, take a look at this thread:
http://www.javagaming.org/forums/index.php?topic=12453.0
It may give you a more clear idea on the approach.
Specifically, see this post for some flags:
http://www.javagaming.org/forums/index.php?topic=12453.msg100335#msg100335
These two flags from this post may help you a bit:
// magic passes
System.setProperty("sun.java2d.accthreshold", "0");
System.setProperty("sun.java2d.allowrastersteal", "true");
Basically, they mean "attempt to cache BufferedImages in a
Pixmap on a first copy", and "do not disable acceleration for
images for which the application got direct access to the pixels".
This will only be helpful if you do not intend to update the
BufferedImage with your renderer's output on every screen update -
which means that we could use the cached copy for at least
some of the updates.
However, if you use the "allowrastersteal" flag, you'll need to let us
know that the image changed by rendering to it (like do a single pixel fillRect):
// somewhere..
BufferedImage bi = ... // say it's an IntRgb image - create the image once
System.setProperty("sun.java2d.allowrastersteal", "true");
System.setProperty("sun.java2d.accthreshold", "0");
void render(Graphics backBufferG) {
DataBufferInt dbi = ...;
int[] data = dbi.getData();
if (nativeUpdatePending) {
// now update the pixels from the native code if there's update pending
updateDataBufferWithDataFromNative(data);
// now let 2d know that the pixels changed by touching the image:
Graphics g = bi.getGraphics();
g.setColor(TranparentColor);
g.fillRect(0,0,1,1);
}
// now copy the image to the backbuffer:
backBufferG.drawImage(bi, ..);
}
Or, alternatively, forget about the flags and do what other folks
were doing in that thread (note that the d3d and ddraw-related
flags obviously won't work for you, but the opengl one might).
Hopefully I didn't confuse you completely.
Thanks,
Dmitri
Hi, Dmitri
i have seen the examples you suggested, but i still have a little confused, if i get the databuffer in my native function, yes it's not a copy, but if i have to fill the databuffer by setting every pixel, like:
for(int i = 0; i < length; i++)
{
buffer[i] = RGB(255, 0, 255);
}
it would be very inefficient, because the length of databuffer is very large, is there any way to copy the entire array of pixels directly into databuffer? usually, i use bitmap to draw graphics on dc.
thanks a lot.
