Multithreading ConcurrentHashMap and Lists

Hello all,

I apologise for re-posting.

I have a data structure as follows

ConcurrentHashMap<String, Object>

The Object corresponds to 3 lists

List<Long>, List<Long>, List<Long>

These long's correspond to file pointer positions

and the string is a file name.

i.e. each file has a series of long 'positions' from where

objects will be read/modified and written back.

This ConcurrentHashMap will be accessed by multiple threads,

and objects from the file will be read/modified and written back.

There is another set of threads which write data to those files.

(if a file is created for the first time, then a new entry will be

put in the ConcurrentHashMap)

Right now - my pseudo - code looks something like this

////////THIS GENERIC MAP CAN BE ACCESSED BY ALL THREADS AND STORES ALL DATA

////////

public ConcurrentHashMap<String, ListObjects> GenericMapStructure;

////////

////////

//Threads that writes data to file uses this function

////////

public synchronized boolean writeSMS(Object object, String fileName){

//

//Do the write process

//Update the list values

//Update the ConcurrentHashMap

GenericMapStructure.put(fileName, listObjects);

//return a boolean

//

}

////////

//Threads that read/modify the log values and update the concurrent hash map

////////

//This thread will be called multiple times by passing the file name via the //thread class constructor

private ConcurrentHashMap<String, ListObjects> concurrentMap;

public void run(){

for(;;){

//Get a local copy of the GenericMapStructure

concurrentMap = GenericMapStructure;

if(FILENAME IS AVAILABLE IN THE CONCURRENTHASHMAP ){

//GET THE LIST OBJECTS FROM THE CONCURRENTHASHMAP

//EXTRACT LISTS FROM THAT OBJECT

//ITERATE OVER THE LIST

Iterator activeIterator = smsList.iterator();

//MANIPULTAE THE OBJECTS FROM THE FILE;

//UPDATE THE LIST with LONGS & PUT IT BACK IN THE ACTUAL GENERIC QUEUE

GenericMapStructure.put(String.valueOf(keyIS), listObjects);

}

else{

Thread.sleep(100);

}

}

}

The problem is the API states that ConcurrentHashMap is thread safe, however

Retrievals reflect the results of the most recently completed update operations holding upon their onset.

So imagine my writeSMS method writes

1 long value into the 1st List<Long>

2 long values into the 2nd List<Long>

3 long values into the 3rd List<Long>

& updates it in the Map.

Then read/modify threads starts accessing the Map and

starts changing the Long values in my lists; i.e. I am going over the Iterator (smsList.iterator())

At this point is there a possiblity for my writeSMS method to write new values into the smsList

of the corresponding Key (filename)?

If that happens then my Map will gert corrupted, i.e. i dont want to write something into the

MAP when I am reading/modifying the list values.

I am having a local copy of the GenericMapStructure for each thread, since there will be multiple threads accessing different keys of ther MAP and I dont want the threads to wait for one-another!

In short - is my pseudo-code thread safe?

best regards.

[3488 byte] By [angeshwara] at [2007-11-27 8:45:04]
# 1

No.

In addition to the problem you've pointed out, you also have to consider the thread-safety of the lists themselves.

What you're doing is transactional in nature. I would use synchronized blocks myself. If you're recording offsets into files the concurrency requirements can't be much of a bottleneck.

ejpa at 2007-7-12 20:46:06 > top of Java-index,Core,Core APIs...
# 2

Thanks for the reply, if I use a synchronized block would the sudeo code look something like

public synchronized boolean writeSMS(Object object, String fileName){

//

//Do the write process

//Update the list values

//Update the ConcurrentHashMap

GenericMapStructure.put(fileName, listObjects);

//return a boolean

//

}

////////

//Threads that read/modify the log values and update the concurrent hash map

////////

//This thread will be called multiple times by passing the file name via the //thread class constructor

public void run(){

for(;;){

//Get a local copy of the GenericMapStructure

concurrentMap = GenericMapStructure;

if(FILENAME IS AVAILABLE IN THE CONCURRENTHASHMAP ){

synchronized(this){

//GET THE LIST OBJECTS FROM THE CONCURRENTHASHMAP

//EXTRACT LISTS FROM THAT OBJECT

//ITERATE OVER THE LIST

Iterator activeIterator = smsList.iterator();

//MANIPULTAE THE OBJECTS FROM THE FILE;

//UPDATE THE LIST with LONGS & PUT IT BACK IN THE ACTUAL GENERIC QUEUE

GenericMapStructure.put(String.valueOf(keyIS), listObjects);

}

}

else{

Thread.sleep(100);

}

}

}

But again I dont think this will make a huge difference, imagine if the Keys of the Map are filename1, filename2 making the read/update part synchronized will only make that section synchronized and prevent other threads from

reading/updating that local copy of the list/map held by the thread.

Nothing actually prevents the writeSMS method in writing data to list objects of the MAP! when reading/updating to/from the map is being done.

Also I need to make sure that thread that writes data for key filename2 should not wait until operations for key filename1 have been completed, which means I shold not lock the entire Map but rather lock each key and corresponding List objects seperately for each write and read/update threads.

any pseudo-code will be of great help!

angeshwara at 2007-7-12 20:46:06 > top of Java-index,Core,Core APIs...
# 3

Hello,

I was just looking into the API,

My current data structure is

ConcurrentHashMap<String, Object>

The Object corresponds to 3 lists

List<Long>, List<Long>, List<Long>

Will using a ConcurrentLinkedQueue<Long> instead of a List<Long>

help to solve the problem?

The API says that ConcurrentLinkedQueue is threadsafe, so will it make both the ConcurrentHashMap and the underlying object (ConcurrentLinkedQueue) thread-safe?

angeshwara at 2007-7-12 20:46:06 > top of Java-index,Core,Core APIs...
# 4
I think your problem is transactional and you have to surround the entire transaction with synchronized {} blocks.I could be wrong.You could always try it.
ejpa at 2007-7-12 20:46:06 > top of Java-index,Core,Core APIs...
# 5

If I use the synchronised block then that thread waits till the iteration

is done (which is OK, but not what I expect to see)

I dont need the other thread to wait until Iteration has been done, i.e. I need the List/Queue to be dynamically updated.

I checked the iterator method in ConcurrentLinkedQueue

But what is surprising is - the API states the following for Iterator -

Returns an iterator over the elements in this queue in proper sequence. The returned iterator is a "weakly consistent" iterator that will never throw ConcurrentModificationException, and guarantees to traverse elements as they existed upon construction of the iterator, and may (but is not guaranteed to) reflect any modifications subsequent to construction.

" may (but is not guaranteed to) reflect any modifications subsequent to construction. "

is quite misleading, so what happens if there is an update on the

ConcurrentLinkedQueue while iterating over it? will new values be reflected?

Could this be used at all?

angeshwara at 2007-7-12 20:46:06 > top of Java-index,Core,Core APIs...