Weakness with SAML/Single-SignOn?

I have a question (please tell me if I'm understanding this all wrong too):

In SAML single-sign on, user "Bob" authenticates with some server. That server now says, "ok, yes this user is Bob." The server then generates an authentication assertion and sends it to server #2 (which also accepts SAML). That server then looks at the assertions and says, "OK, Bob can do X,Y,Z in this server...go ahead" (Yes, I'm greatly simplifying but those are the high-level steps as I understand them).

Now here's my problem/question:

Bob has essentially given server #1 the ability to pretend it's him. Yes, i understand he "trusts" that site (which is why he's logging in with them), but I think there are different levels of trust and separation that need to exist. Let's say server #1 is his doctor/medical info site and #2 is his bank account. Since both accept SAML assertions as Bob's identity, someone with access to the medical server can now access Bob's bank account.

He really should only have trusted them with his medical info and nothing else. But SAML makes it impossible to separate those trusts.

Am I understanding this incorrectly?

[1180 byte] By [6tr6tra] at [2007-10-2 5:02:13]
# 1
Actually, server #2, as the saml client,knows the assertion it gets from server#2 is about medical access only and and should be used for medical purpose.
markrlina at 2007-7-16 1:05:56 > top of Java-index,Security,Java Secure Socket Extension (JSSE)...