RDT_assertion failed: 0, file ../src/rde/lockset.c, line 493

The following assertion occurs with the Feb 2007 thread analyzer

(Sun Solaris 10 11/06 SPARC with all the patches required by

Sun Studio C++ Feb 2007)

% collect -r all tntnet -c tntnet.conf

WARNING: Target `tntnet' is not instrumented for datarace detection;

reported datarace data may be misleading

Creating experiment database tha.6.er ...

2007-04-26 13:33:16.09216 [19660.1] INFO tntnet.tntnet - create 5 worker threads

RDT_assertion failed: 0, file ../src/rde/lockset.c, line 493

% CC -V

CC: Sun C++ 5.9 SunOS_sparc Build40_1 2007/02/08

%

For the "WARNING: Target `tntnet' is not instrumented for datarace detection",

I look through the large build output for cxxtools, tntnet, and the myfirstproject.so

shared library and as far as I can see, every .o, .so in cxxtools, tntnet, and

myfirstproject.so is compiled and linked with -mt -xinstrument=datarace.

But no need to take my word for it, the testcase is:

1. Need to fix SC12_02_07/SUNWspro/prod/include/CC/Cstd/stdcomp.h

(I will explain why in the Sun Studio C++ forum):

/% diff -wc /h/goanna/0/SC12_02_07/SUNWspro/prod/include/CC/Cstd/orig/stdcomp.h /h/goanna/0/SC12_02_07/SUNWspro/prod/include/CC/Cstd/stdcomp.h

*** /h/goanna/0/SC12_02_07/SUNWspro/prod/include/CC/Cstd/orig/stdcomp.h Thu Feb 8 23:18:51 2007

/h/goanna/0/SC12_02_07/SUNWspro/prod/include/CC/Cstd/stdcomp.hThu Apr 19 14:57:28 2007

***************

*** 1304,1311 ****

/* #define _RWSTD_NO_BAD_CAST 1 */

/* #define RWSTD_NO_BAD_CAST 1 */

! #define _RWSTD_NO_MEMBER_TEMPLATES 1

! #define RWSTD_NO_MEMBER_TEMPLATES 1

#define _RWSTD_NO_FRIEND_TEMPLATES 1

#define RWSTD_NO_FRIEND_TEMPLATES 1

1304,1311 -

/* #define _RWSTD_NO_BAD_CAST 1 */

/* #define RWSTD_NO_BAD_CAST 1 */

! /* #define _RWSTD_NO_MEMBER_TEMPLATES 1 */

! /* #define RWSTD_NO_MEMBER_TEMPLATES 1 */

#define _RWSTD_NO_FRIEND_TEMPLATES 1

#define RWSTD_NO_FRIEND_TEMPLATES 1

***************

*** 1406,1415 ****

#define _RWSTD_NO_INIT_CONST_TEMPLATE_REF_ARG 1

#define RWSTD_NO_INIT_CONST_TEMPLATE_REF_ARG 1

! #define _RWSTD_NO_CLASS_PARTIAL_SPEC 1

! #define RWSTD_NO_CLASS_PARTIAL_SPEC 1

! #define _RWSTD_NO_FUNC_PARTIAL_SPEC 1

! #define RWSTD_NO_FUNC_PARTIAL_SPEC 1

/* #define _RWSTD_NO_OVERLOAD_OF_TEMPLATE_FUNCTION 1 */

/* #define RWSTD_NO_OVERLOAD_OF_TEMPLATE_FUNCTION 1 */

1406,1415 -

#define _RWSTD_NO_INIT_CONST_TEMPLATE_REF_ARG 1

#define RWSTD_NO_INIT_CONST_TEMPLATE_REF_ARG 1

! /* #define _RWSTD_NO_CLASS_PARTIAL_SPEC 1 */

! /* #define RWSTD_NO_CLASS_PARTIAL_SPEC 1 */

! /* #define _RWSTD_NO_FUNC_PARTIAL_SPEC 1 */

! /* #define RWSTD_NO_FUNC_PARTIAL_SPEC 1 */

/* #define _RWSTD_NO_OVERLOAD_OF_TEMPLATE_FUNCTION 1 */

/* #define RWSTD_NO_OVERLOAD_OF_TEMPLATE_FUNCTION 1 */

/%

2. Download and install the latest autotools (automake, autoconf, libtool, gettext).

3. download and unpack cxxtools 1.4.3.5 and tntnet 1.5.3.4 from:

http://www.tntnet.org/download.hms

4. grep pthread **/Makefile.am, remove -lpthread and -pthread.

5. Make the following changes to Makefile.am files:

% grep "Wc," **/Makefile.am

cxxtools-1.4.3.5/Makefile.am:AM_CXXFLAGS=-Wc,-xinstrument=datarace

cxxtools-1.4.3.5/src/Makefile.am:libcxxtools_la_CXXFLAGS = -Wc,-xinstrument=datarace

tntnet-1.5.3.4/framework/cgi/Makefile.am:libtntnet_cgi_la_CXXFLAGS = -Wc,-xinstrument=datarace

tntnet-1.5.3.4/framework/common/Makefile.am:libtntnet_la_CXXFLAGS = -Wc,-xinstrument=datarace

tntnet-1.5.3.4/framework/defcomp/Makefile.am:tntnet_la_CXXFLAGS = -Wc,-xinstrument=datarace

tntnet-1.5.3.4/sdk/tools/common/Makefile.am:libtntnet_sdk_la_CXXFLAGS = -Wc,-xinstrument=datarace

%

This is because libtools removes all compiler options it does not know

about unless they are prefixed with -Wc.

6. Set env vars:

% env | egrep "(CC|FLAGS)"

CXXFLAGS=-g -mt -xinstrument=datarace

CXX=CC

CC=cc

CXXCPP=CC -E

CPPFLAGS=-I/usr/sfw/include -I/where_you_want_it/tntnet/cxxtools-1.4.3.5/include

LDFLAGS=-g -mt -xinstrument=datarace -L/usr/sfw/lib

%

7. In cxxtools-1.4.3.5:

autoreconf --install --force

./configure --prefix=/where_you_want_it/tntnet

make

8. As root:

make install

9. Set PATH to include /where_you_want_it/tntnet/bin

10. Install openssl (its on SunFreeware.com)

11. In tntnet-1.5.3.4:

autoreconf --install --force

./configure --prefix=/where_you_want_it/tntnet --with-ssl=openssl

make

12. As root:

make install

13. Follow the instructions to build and run the first example in:

http://www.tntnet.org/download/tntnet-1.5.3/doc/quick-start-guide.pdf

Of course you need to tweak the Makefile:

% diff -wc myfirstproject/orig/Makefile myfirstproject/Makefile

*** myfirstproject/orig/MakefileSat Apr 7 13:24:41 2007

myfirstproject/MakefileThu Apr 26 22:34:11 2007

***************

*** 3,10 ****

SUFFIXES=.ecpp .gif .jpg .css .js

ECPPC=ecppc

! CXXFLAGS+=-I/where_you_want_it/tntnet/include -fPIC -O2

! LDFLAGS+=-shared -L/where_you_want_it/tntnet/lib -ltntnet

%.cpp: %.ecpp

$(ECPPC) $(ECPPFLAGS) $(ECPPFLAGS_CPP) $<

3,10 -

SUFFIXES=.ecpp .gif .jpg .css .js

ECPPC=ecppc

! CXXFLAGS+=-I/where_you_want_it/tntnet/include -KPIC -O2 -mt -xinstrument=datarace

! LDFLAGS+=-G -mt -L/where_you_want_it/tntnet/lib -ltntnet

%.cpp: %.ecpp

$(ECPPC) $(ECPPFLAGS) $(ECPPFLAGS_CPP) $<

%

Thanks, Mark

[5862 byte] By [markw22a] at [2007-11-27 2:28:31]
# 1
Hi Mark, thanks for the detailed description. We are looking at the problem now.A quick note. In step 5, instead of using "-Wc,-xinstrument=datarace", you need to use"-W2,-xinstrument=datarace -O2". Thanks,-- Yuan
yuana at 2007-7-12 2:40:44 > top of Java-index,Development Tools,Solaris and Linux Development Tools...
# 2
Hi Mark, thanks for the detailed description. We are looking at the problem now.A quick note. In step 5, instead of using "-Wc,-xinstrument=datarace", you need to use"-W2,-xinstrument=datarace -O2". Thanks,-- Yuan
yuana at 2007-7-12 2:40:44 > top of Java-index,Development Tools,Solaris and Linux Development Tools...
# 3

Hello Yuan,

Here is one of the shared library link lines :

/bin/ksh ../libtool --tag=CXX--mode=link CC -Wc,-xinstrument=datarace -g -xinstrument=datarace -mt -version-info 3:1:0 -g -L/usr/sfw/lib -o libcxxtools.la -rpath /h/goanna/1/sw/tntnet/lib libcxxtools_la-base64stream.lo libcxxtools_la-cgi.lo libcxxtools_la-dlloader.lo libcxxtools_la-fdstream.lo libcxxtools_la-hdstream.lo libcxxtools_la-hirestime.lo libcxxtools_la-httpreply.lo libcxxtools_la-httprequest.lo libcxxtools_la-iniclass.lo libcxxtools_la-log.lo md5.lo libcxxtools_la-md5stream.lo libcxxtools_la-multifstream.lo libcxxtools_la-net.lo libcxxtools_la-pollclass.lo libcxxtools_la-pipe.lo libcxxtools_la-pipestream.lo libcxxtools_la-query_params.lo libcxxtools_la-syserror.lo libcxxtools_la-tcpstream.lo libcxxtools_la-tee.lo libcxxtools_la-thread.lo libcxxtools_la-udp.lo libcxxtools_la-udpstream.lo libcxxtools_la-xmltag.lo libcxxtools_la-iconvstream.lo ../libltdl/libltdlc.la -lrt -lsocket

CC -G -hlibcxxtools.so.3 -o .libs/libcxxtools.so.3.0.1.libs/libcxxtools_la-base64stream.o .libs/libcxxtools_la-cgi.o .libs/libcxxtools_la-dlloader.o .libs/libcxxtools_la-fdstream.o .libs/libcxxtools_la-hdstream.o .libs/libcxxtools_la-hirestime.o .libs/libcxxtools_la-httpreply.o .libs/libcxxtools_la-httprequest.o .libs/libcxxtools_la-iniclass.o .libs/libcxxtools_la-log.o .libs/md5.o .libs/libcxxtools_la-md5stream.o .libs/libcxxtools_la-multifstream.o .libs/libcxxtools_la-net.o .libs/libcxxtools_la-pollclass.o .libs/libcxxtools_la-pipe.o .libs/libcxxtools_la-pipestream.o .libs/libcxxtools_la-query_params.o .libs/libcxxtools_la-syserror.o .libs/libcxxtools_la-tcpstream.o .libs/libcxxtools_la-tee.o .libs/libcxxtools_la-thread.o .libs/libcxxtools_la-udp.o .libs/libcxxtools_la-udpstream.o .libs/libcxxtools_la-xmltag.o .libs/libcxxtools_la-iconvstream.o -z allextract ../libltdl/.libs/libltdlc.a -z defaultextract -L/usr/sfw/lib -ldl -lrt -lsocket -library=Cstd -library=Crun -lc-xinstrument=datarace -mt

Please note that -Wc,-xinstrument=datarace is passed to the libtool ksh script,

and then the libtool ksh script removes the -Wc, prefix and passes

-xinstrument=datarace to CC.

libtool discards any options it does not know about unless they are prefixed

by -Wc,

On another note, in step 5 the CPPFLAGS should be:

CPPFLAGS=-I/usr/sfw/include -I/where_you_want_it/tntnet/include

Thanks, Mark

markw22a at 2007-7-12 2:40:44 > top of Java-index,Development Tools,Solaris and Linux Development Tools...
# 4

HI, Mark

I'm trying to reproduce the bug. But there's problem while I'm building tntnet. I'm wondering about the step 4.

4. grep pthread **/Makefile.am, remove -lpthread and -pthread.

Why should we remove the -lpthread and -pthread? And I got following message whether I removed the "-lpthread" and "-pthread" or not.

make[1]: Entering directory `/export/home/tnt_src/tntnet-1.5.3.4/sdk/demos'

../../sdk/tools/ecppc/ecppc alldemos.ecpp

ld.so.1: ecppc: fatal: thread: open failed: No such file or directory

ld.so.1: ecppc: fatal: thread: audit initialization failure: disabled

make[1]: *** [alldemos.cpp] Killed

make[1]: Leaving directory `/export/home/tnt_src/tntnet-1.5.3.4/sdk/demos'

make: *** [all-recursive] Error 1

Is it related the reason of removing those options?

Would you like to try "collect -r races tntnet -c tntnet.conf" and "collect -r deadlock tntnet -c tntnet.conf" and post the result? That will be helpful.

You can send mail to me directly. (xi.qian@sun.com)

Thanks,

-Xi

Message was edited by:

Xi.Qian

Xi.Qiana at 2007-7-12 2:40:44 > top of Java-index,Development Tools,Solaris and Linux Development Tools...
# 5

Hello Xi,

The error:

ld.so.1: ecppc: fatal: thread: open failed: No such file or directory

ld.so.1: ecppc: fatal: thread: audit initialization failure: disabled

occurs because the gcc -pthread option is not recognized by Sun CC,

so it passes it to the linker, which interprets it as specifying thread

as an audit library (with the -p ld option). Which can be verified by

running dump -Lv on the executable, and on the shared libraries

that were built that the executable depends on.

The -lpthread option is probably harmless, but not necessary as

it should instead use the -mt option.

The changes to the Makefile.am files take effect after the commands

in steps 7 and 11.

The collect -races encountered the assertion:

% collect -r races tntnet -c tntnet.conf

WARNING: Target `tntnet' is not instrumented for datarace detection;

reported datarace data may be misleading

Creating experiment database tha.1.er ...

2007-05-01 00:17:08.45017 [796.1] INFO tntnet.tntnet - create 5 worker threads

RDT_assertion failed: 0, file ../src/rde/lockset.c, line 493

%

The collect -r deadlock seems to work. So I tried running a

test that I would like to run the collect -r races with, using a

script:

#!/bin/ksh

/usr/apache2/bin/ab -n 100000 -c 30 http://10.148.1.102:8000/myfirstproject

It looked like that would take a long time with that may repeats, I might

need to use a smaller experiment. The initial truncated output is:

% collect -r deadlock tntnet -c tntnet.conf

Creating experiment database tha.2.er ...

2007-05-01 00:17:45.57069 [939.1] INFO tntnet.tntnet - create 5 worker threads

2007-05-01 00:21:54.75722 [939.2] INFO tntnet.worker - process request: GET /myfirstproject from client 10.148.1.102 user-Agent "Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7) Gecko/20060629"

2007-05-01 00:21:54.84579 [939.2] INFO tntnet.comploader - load library "myfirstproject"

2007-05-01 00:21:54.99394 [939.2] INFO tntnet.worker - request ready, returncode 200 - ContentSize: 124

2007-05-01 00:23:10.26068 [939.6] INFO tntnet.worker - process request: GET /myfirstproject from client 10.148.1.102 user-Agent "ApacheBench/2.0.41-dev"

2007-05-01 00:23:10.36622 [939.2] INFO tntnet.worker - process request: GET /myfirstproject from client 10.148.1.102 user-Agent "ApacheBench/2.0.41-dev"

2007-05-01 00:23:10.41058 [939.4] INFO tntnet.worker - process request: GET /myfirstproject from client 10.148.1.102 user-Agent "ApacheBench/2.0.41-dev"

2007-05-01 00:23:10.46237 [939.2] INFO tntnet.worker - request ready, returncode 200 - ContentSize: 124

2007-05-01 00:23:10.52218 [939.4] INFO tntnet.worker - request ready, returncode 200 - ContentSize: 124

2007-05-01 00:23:10.58946 [939.6] INFO tntnet.worker - request ready, returncode 200 - ContentSize: 124

Thanks very much for looking at this.

Regards, Mark

markw22a at 2007-7-12 2:40:44 > top of Java-index,Development Tools,Solaris and Linux Development Tools...
# 6
We've found the problem and have a fix in our internal build. Unfortunately, there is no workaround right now. We will update the forum when the new build is released.Thanks a lot!-- Yuan
yuana at 2007-7-12 2:40:44 > top of Java-index,Development Tools,Solaris and Linux Development Tools...
# 7
Hi, MarkI installed tntnet and got some interesting experiment result with race detection. Would you like to contact me by e-mail so that we can have more discussion?My e-mail address is: Xi.Qian@sun.comThanks,-Xi
Xi.Qiana at 2007-7-12 2:40:44 > top of Java-index,Development Tools,Solaris and Linux Development Tools...