mount: /tmp is already mounted, swap is busy... during boot

Our Solaris 9 got unstable yesterday and it does not boot anymore. First symptom was when I tried to open ssh connection to the server and it replied "Could not allocate pty..." or something similar. When I logged in locally I couldn't run some commands like ls in /tmp dir, I just got an error. Next I tried to run fsck and it told me that there was problems and asked whether I wan't to salvage the root partition. I didn't want to do that since it was mounted and I tried to boot the system to runlevel 1 so I could do it more securely. The system didn't succeed to do that with init 1 command so I changed the inittab defaultrunlevel to 1 and after that the system goes to infinite loop during boot and just prints the following over and over again:

The system is coming up for administration. Please wait.

checking ufs filesystems

/dev/rdsk/c0t0d0s6: is stable.

mount: /tmp is already mounted, swap is busy,

or the allowable number of mount points has been exceeded

mount: /dev/dsk/c0t0d0s6 is already mounted, /globaldevices is busy,

or the allowable number of mount points has been exceeded

The system is ready for administration.

The system is coming up for administration. Please wait.

checking ufs filesystems

/dev/rdsk/c0t0d0s6: is stable.

mount: /tmp is already mounted, swap is busy,

or the allowable number of mount points has been exceeded

mount: /dev/dsk/c0t0d0s6 is already mounted, /globaldevices is busy,

or the allowable number of mount points has been exceeded

The system is ready for administration.

When I send break signal I can access the PROM but I haven't been able to figure out what I could do... for example test-all just gives me "Rejecting alloc-mem!" and booting with any parameters does always the same. Any ideas what I could do? Is there any way to modify the inittab from PROM?

[1935 byte] By [Serotonin] at [2007-11-25 23:38:46]
# 1

If the root filesystem really does have a problem, just doing an init 1 or init s won't put you in a state to fix it. It's still mounted read/write at that point.

You'd probably be best to boot from install media, then fsck the filesystems on the disk to check. If they're clean, you'll have to troubleshoot further to determine your issues.

Can you boot single user cleanly? (boot -s)

--

Darren

Darren_Dunham at 2007-7-5 18:22:34 > top of Java-index,General,Talk to the Sysop...
# 2

Nope, boot -s does the same thing. I somehow succeeded to configure the system to do the fsck during boot and it corrected the problem in the filesystem (printed "SALVAGED"). However this did not fix the problem of infinite loop during the boot. It still prints again and again:

The system is coming up for administration. Please wait.

checking ufs filesystems

/dev/rdsk/c0t0d0s6: is stable.

mount: /tmp is already mounted, swap is busy,

or the allowable number of mount points has been exceeded

mount: /dev/dsk/c0t0d0s6 is already mounted, /globaldevices is busy,

or the allowable number of mount points has been exceeded

The system is ready for administration.

The system is coming up for administration. Please wait.

...

I think the problem might be that init fails and retries again and again. Also the system prints "swap: getexecname() failed" during the boot and it did it for some commands when the system started acting weirdly. Maybe if I could modify the inittab default runlevel back to 3 might help? Is it possible to modify it from PROM or ALOM? I don't have the cd-rom in the server so only way to boot from install media is probably over network and it seems pretty complicated task to do...

Serotonin at 2007-7-5 18:22:34 > top of Java-index,General,Talk to the Sysop...