3

I was performing routine administrative tasks on an Oracle instance and I was not able to connect to it.

I proceeded to do as follows to reach a diagnosis:

  1. I checked for Oracle processes in memory and saw that the instance was down.
  2. I tried to startup the instance but it never responded to the startup command. It simply stalled. No output or feedback, no matter how long I wait. Only a kill -9 can get me out of it.
  3. I reviewed the alert log and the last message was 3 days ago:

    "DBW0: terminating instance due to error 472
    Instance terminated by DBW0, pid = 14952"
    

Questions

  • What can cause BDW to terminate an instance?
  • Why I don't get any feedback, whether in the sqlplus console or the alert log when I try to startup the instance.

When I ran strace, I got this:

ERROR: unable to open /dev/log.

I'm running Oracle on SunOS 5.8 Generic_117350-08 sun4u sparc SUNW,Sun-Fire. The RDBMS version is 9.2.0.8.0

EDIT:

I followed advice from both @balazs-papp and @jsapkota about running truss:

When I run truss on the sqlplus the I try to startup the instance, I get this output:

read(0, " s t a r t u p\n", 1024)               = 8
write(9, "\0 U\0\006\0\0\0\0\0038A".., 85)      = 85
read(10, 0x10029A536, 2064)     (sleeping...)
signotifywait()                 (sleeping...)
lwp_cond_wait(0xFFFFFFFF7D62B058, 0xFFFFFFFF7D62B068, 0xFFFFFFFF7D621C80) (sleeping...)
lwp_cond_wait(0xFFFFFFFF7D62B058, 0xFFFFFFFF7D62B068, 0xFFFFFFFF7D621C80) (sleeping...)
door_return(0x00000000, 0, 0x00000000, 0) (sleeping...)
  • How can I interpret this output? It's totally cryptic to me.
  • 2
    Check to see if you have any volumes with no free space. Specifically the volume that has adump. Also check to see if you have License high water mark set. Finally you can clear out your bdump dir, try to start the database and see what is in the alert log and if any trace files were created. You should keep what is in the bdump dir in case you need it later. Since 9i is desupported, you won't need those files for Oracle support. – Gandolf989 Apr 12 '16 at 17:31
  • @Gandolf989, I have check the space and privileges and all look ok, Thanks. – Oscar Caraballo Apr 12 '16 at 20:27
  • Kindly provide alerts logs to investigate further. Also contact My Oracle Support. – JSapkota Apr 13 '16 at 15:28
  • Before you do any truss/strace etc, have you ensured all your environment variables set correctly for the instance you are trying to start? like using '. oraenv' ? Can you do that one more time, set environment correctly, log in as sysdba and then issue startup. Investigate alert log first and associated trace files before running with truss/strace etc. – Raj Apr 14 '16 at 12:25
  • If you use NFS for database storage, this could be caused by file locking issues. More details on MOS: Network File System (NFS): Unable to lock a file on an NFSv3 client (Doc ID 1009026.1) – Balazs Papp Apr 15 '16 at 19:42
1

I exected startup nomount with explicit pfile paremeter and it works.

I have the init < SID >.ora in the $ORACLE_HOME/dbs/ directory, and always was working.

  • Kindly provide alerts logs to investigate further – JSapkota Apr 17 '16 at 6:14
0
  1. This is quite obvious from the error.

    00472, 00000, "PMON  process terminated with error"
    // *Cause:  The process cleanup process died
    // *Action: Warm start instance
    

PMON died, so DBWR terminated the instance. If any process dies from SMON, PMON, LGWR, DBWR, CKPT, the remaining processes will terminate the instance. So the real question is, why did PMON die?

  1. Could be anything. Typically environmental/configuration issue. Time to do an strace -f/truss -f, depending on the platform, which we don't know:

    strace -f -o startup.out sqlplus / as sysdba
    SQL> startup
    
  • When I ran strace, I got this: ERROR: unable to open /dev/log. I'm running oracle in SunOS 5.8 Generic_117350-08 sun4u sparc SUNW,Sun-Fire. @Balazs_Papp – Oscar Caraballo Apr 12 '16 at 20:45
  • Try restarting the server. It is possible that you have a memory leak in /dev/log. I googled this, getupdates.oracle.com/readme/117350-62 it could also be that the memory for the database changed and either your server doesn't have the memory, or the shared memory setting on the server are not large enough to run the database with the memory that you are trying to allocate. You might want to look through the before you install 9i for Solaris guide and verify that the memory settings are correct. akadia.com/services/… – Gandolf989 Apr 13 '16 at 13:46
  • Please see the edit to my question, I added the output of truss command and I cannot make sense of it. Please shed some light on it. – Oscar Caraballo Apr 13 '16 at 14:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.