Last login: Fri Jul 20 16:54:26 2018 from 9.1.173.100[root@ctcdb ~]# su - oracle[ctcdb:oracle]:/home/oracle>sqlplus "/ as sysdba"SQL*Plus: Release 11.2.0.4.0 Production on Fri Jul 20 16:55:38 2018Copyright (c) 1982, 2013, Oracle. All rights reserved.Connected to an idle instance.SQL> startupORACLE instance started.Total System Global Area 1.3683E+11 bytesFixed Size 2270360 bytesVariable Size 1.2616E+10 bytesDatabase Buffers 1.2402E+11 bytesRedo Buffers 189456384 bytesDatabase mounted.ORA-00742: Log read detects lost write in thread %d sequence %d block %dORA-00312: online log 1 thread 1: '+DATADG/orcl/redo01.log'SQL> shutdown abortORACLE instance shut down.SQL> startup ORACLE instance started.Total System Global Area 1.3683E+11 bytesFixed Size 2270360 bytesVariable Size 1.2616E+10 bytesDatabase Buffers 1.2402E+11 bytesRedo Buffers 189456384 bytesDatabase mounted.ORA-00742: Log read detects lost write in thread %d sequence %d block %dORA-00312: online log 1 thread 1: '+DATADG/orcl/redo01.log'SQL> RECOVER DATABASE USING BACKUP CONTROLFILE UNTIL CANCELORA-00279: change 361503907875 generated at 07/20/2018 14:00:39 needed forthread 1ORA-00289: suggestion : +ARCHDGORA-15173: entry 'ORCL' does not exist in directory '/'ORA-00280: change 361503907875 for thread 1 is in sequence #9922Specify log: { =suggested | filename | AUTO | CANCEL}ORA-00308: cannot open archived log '+ARCHDG'ORA-17503: ksfdopn:2 Failed to open file +ARCHDGORA-15045: ASM file name '+ARCHDG' is not in reference formORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error belowORA-01194: file 1 needs more recovery to be consistentORA-01110: data file 1: '+DATADG/orcl/datafile/system.259.978924803'SQL> ALTER DATABASE OPEN RESETLOGS;ALTER DATABASE OPEN RESETLOGS*ERROR at line 1:ORA-01194: file 1 needs more recovery to be consistentORA-01110: data file 1: '+DATADG/orcl/datafile/system.259.978924803'SQL> shutdown abort;ORACLE instance shut down.SQL> startup mount;ORACLE instance started.Total System Global Area 1.3683E+11 bytesFixed Size 2270360 bytesVariable Size 1.2616E+10 bytesDatabase Buffers 1.2402E+11 bytesRedo Buffers 189456384 bytesDatabase mounted.SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options[ctcdb:oracle]:/home/oracle>lsnctl status-bash: lsnctl: command not found[ctcdb:oracle]:/home/oracle>lsnrctl statusLSNRCTL for Linux: Version 11.2.0.4.0 - Production on 20-JUL-2018 17:06:20Copyright (c) 1991, 2013, Oracle. All rights reserved.Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))STATUS of the LISTENER------------------------Alias LISTENERVersion TNSLSNR for Linux: Version 11.2.0.4.0 - ProductionStart Date 20-JUL-2018 16:53:40Uptime 0 days 0 hr. 12 min. 39 secTrace Level offSecurity ON: Local OS AuthenticationSNMP OFFListener Parameter File /u01/app/11.2.0.4/grid/network/admin/listener.oraListener Log File /u01/app/grid/diag/tnslsnr/ctcdb/listener/alert/log.xmlListening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=ctcdb)(PORT=1521)))Services Summary...Service "+ASM" has 1 instance(s). Instance "+ASM", status READY, has 1 handler(s) for this service...Service "orcl" has 1 instance(s). Instance "orcl", status READY, has 1 handler(s) for this service...The command completed successfully[ctcdb:oracle]:/home/oracle>sqlplus "/ as sysdba"SQL*Plus: Release 11.2.0.4.0 Production on Fri Jul 20 17:07:00 2018Copyright (c) 1982, 2013, Oracle. All rights reserved.Connected to:Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing optionsSQL> SQL> SQL> alter database open;alter database open*ERROR at line 1:ORA-01589: must use RESETLOGS or NORESETLOGS option for database openSQL> recover database using backup controlfile; ORA-00279: change 361503907875 generated at 07/20/2018 14:00:39 needed forthread 1ORA-00289: suggestion : +ARCHDGORA-15173: entry 'ORCL' does not exist in directory '/'ORA-00280: change 361503907875 for thread 1 is in sequence #9922Specify log: { =suggested | filename | AUTO | CANCEL}AUTOORA-00308: cannot open archived log '+ARCHDG'ORA-17503: ksfdopn:2 Failed to open file +ARCHDGORA-15045: ASM file name '+ARCHDG' is not in reference formORA-00308: cannot open archived log '+ARCHDG'ORA-17503: ksfdopn:2 Failed to open file +ARCHDGORA-15045: ASM file name '+ARCHDG' is not in reference formSQL> alter database open RESETLOGS;alter database open RESETLOGS*ERROR at line 1:ORA-01113: file 1 needs media recoveryORA-01110: data file 1: '+DATADG/orcl/datafile/system.259.978924803'SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options[ctcdb:oracle]:/home/oracle>ps -ef | grep ckptgrid 11493 1 0 16:54 ? 00:00:00 asm_ckpt_+ASMoracle 36151 1 0 17:05 ? 00:00:00 ora_ckpt_orcloracle 102216 11677 0 17:21 pts/2 00:00:00 grep ckpt[ctcdb:oracle]:/home/oracle>cat /etc/hosts127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4::1 localhost localhost.localdomain localhost6 localhost6.localdomain610.1.71.73 ctcdb[ctcdb:oracle]:/home/oracle>sqlplus / as sysdbaSQL*Plus: Release 11.2.0.4.0 Production on Fri Jul 20 17:22:12 2018Copyright (c) 1982, 2013, Oracle. All rights reserved.Connected to:Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing optionsSQL> select open_mode from v$database;OPEN_MODE----------------------------------------MOUNTEDSQL> alter database open;alter database open*ERROR at line 1:ORA-01589: must use RESETLOGS or NORESETLOGS option for database openSQL> SQL> SQL> SQL> SQL> recover database;ORA-00283: recovery session canceled due to errorsORA-01610: recovery using the BACKUP CONTROLFILE option must be doneSQL> show parameter backgroundNAME TYPE------------------------------------ ----------------------VALUE------------------------------background_core_dump stringpartialbackground_dump_dest string/u01/app/oracle/diag/rdbms/orcl/orcl/traceSQL> set line 300SQL> /alter database open*ERROR at line 1:ORA-01589: must use RESETLOGS or NORESETLOGS option for database openSQL> recover database using backup controlfile;ORA-00279: change 361503907875 generated at 07/20/2018 14:00:39 needed for thread 1ORA-00289: suggestion : +ARCHDGORA-15173: entry 'ORCL' does not exist in directory '/'ORA-00280: change 361503907875 for thread 1 is in sequence #9922Specify log: { =suggested | filename | AUTO | CANCEL}cancel Media recovery cancelled.SQL> recover database;ORA-00283: recovery session canceled due to errorsORA-01610: recovery using the BACKUP CONTROLFILE option must be doneSQL> shutdown immediateORA-01109: database not openDatabase dismounted.ORACLE instance shut down.SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options[ctcdb:oracle]:/home/oracle>exitlogout[root@ctcdb ~]# su - grid[ctcdb:grid]:/home/grid>which crsctl/u01/app/11.2.0.4/grid/bin/crsctl[ctcdb:grid]:/home/grid>exitlogout[root@ctcdb ~]# /u01/app/11.2.0.4/grid/bin/crsctl stop crsCRS-4013: This command is not supported in a single-node configuration.CRS-4000: Command Stop failed, or completed with errors.[root@ctcdb ~]# /u01/app/11.2.0.4/grid/bin/crsctl stop crs -fCRS-4013: This command is not supported in a single-node configuration.CRS-4000: Command Stop failed, or completed with errors.[root@ctcdb ~]# /u01/app/11.2.0.4/grid/bin/crsctl start crsCRS-4013: This command is not supported in a single-node configuration.CRS-4000: Command Start failed, or completed with errors.[root@ctcdb ~]# /u01/app/11.2.0.4/grid/bin/crsctl stop has CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ctcdb'CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'ctcdb'CRS-2673: Attempting to stop 'ora.ARCHDG.dg' on 'ctcdb'CRS-2673: Attempting to stop 'ora.DATADG.dg' on 'ctcdb'CRS-2677: Stop of 'ora.ARCHDG.dg' on 'ctcdb' succeededCRS-2677: Stop of 'ora.LISTENER.lsnr' on 'ctcdb' succeededCRS-5017: The resource action "ora.DATADG.dg stop" encountered the following error: ORA-03113: end-of-file on communication channelProcess ID: 11558Session ID: 1099 Serial number: 3. For details refer to "(:CLSN00108:)" in "/u01/app/11.2.0.4/grid/log/ctcdb/agent/ohasd/oraagent_grid//oraagent_grid.log".CRS-2677: Stop of 'ora.DATADG.dg' on 'ctcdb' succeededCRS-2679: Attempting to clean 'ora.DATADG.dg' on 'ctcdb'CRS-2681: Clean of 'ora.DATADG.dg' on 'ctcdb' succeededCRS-2673: Attempting to stop 'ora.asm' on 'ctcdb'CRS-2677: Stop of 'ora.asm' on 'ctcdb' succeededCRS-2673: Attempting to stop 'ora.cssd' on 'ctcdb'CRS-2677: Stop of 'ora.cssd' on 'ctcdb' succeededCRS-2673: Attempting to stop 'ora.evmd' on 'ctcdb'CRS-2677: Stop of 'ora.evmd' on 'ctcdb' succeededCRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ctcdb' has completedCRS-4133: Oracle High Availability Services has been stopped.[root@ctcdb ~]# /u01/app/11.2.0.4/grid/bin/crsctl start crsCRS-4013: This command is not supported in a single-node configuration.CRS-4000: Command Start failed, or completed with errors.[root@ctcdb ~]# /u01/app/11.2.0.4/grid/bin/crsctl start hasCRS-4123: Oracle High Availability Services has been started.[root@ctcdb ~]# su - oracle[ctcdb:oracle]:/home/oracle>sqlplus / as sysdbaSQL*Plus: Release 11.2.0.4.0 Production on Fri Jul 20 17:44:04 2018Copyright (c) 1982, 2013, Oracle. All rights reserved.Connected to an idle instance.SQL> startup nomount;ORACLE instance started.Total System Global Area 1.3683E+11 bytesFixed Size 2270360 bytesVariable Size 1.2616E+10 bytesDatabase Buffers 1.2402E+11 bytesRedo Buffers 189456384 bytesSQL> alter database mount;Database altered.SQL> alter database open;alter database open*ERROR at line 1:ORA-01589: must use RESETLOGS or NORESETLOGS option for database openSQL> alter database recover;alter database recover*ERROR at line 1:ORA-00283: recovery session canceled due to errorsORA-01610: recovery using the BACKUP CONTROLFILE option must be doneSQL> alter database open resetlogs;alter database open resetlogs*ERROR at line 1:ORA-01113: file 1 needs media recoveryORA-01110: data file 1: '+DATADG/orcl/datafile/system.259.978924803'SQL> SQL> SQL> recover datafile 1;ORA-00283: recovery session canceled due to errorsORA-01610: recovery using the BACKUP CONTROLFILE option must be doneSQL> recover database using backup controlfile;ORA-00279: change 361503907875 generated at 07/20/2018 14:00:39 needed forthread 1ORA-00289: suggestion : +ARCHDGORA-15173: entry 'ORCL' does not exist in directory '/'ORA-00280: change 361503907875 for thread 1 is in sequence #9922Specify log: { =suggested | filename | AUTO | CANCEL}autoORA-00308: cannot open archived log '+ARCHDG'ORA-17503: ksfdopn:2 Failed to open file +ARCHDGORA-15045: ASM file name '+ARCHDG' is not in reference formORA-00308: cannot open archived log '+ARCHDG'ORA-17503: ksfdopn:2 Failed to open file +ARCHDGORA-15045: ASM file name '+ARCHDG' is not in reference formSQL> archive log list; Database log mode No Archive ModeAutomatic archival DisabledArchive destination +ARCHDGOldest online log sequence 9920Current log sequence 9922SQL> select file#,CHECKPOINT_CHANGE# from v$datafile_header; FILE# CHECKPOINT_CHANGE#---------- ------------------ 1 361503907875 2 361503907875 3 361503907875 4 361503907875 5 361503907875 6 361503907875 7 361503907875 8 361503907875 9 361503907875 10 361503907875 11 361503907875 FILE# CHECKPOINT_CHANGE#---------- ------------------ 12 361503907875 13 361503907875 14 361503907875 15 361503907875 16 361503907875 17 361503907875 18 361503907875 19 361503907875 20 361503907875 21 361503907875 22 361503907875 FILE# CHECKPOINT_CHANGE#---------- ------------------ 23 361503907875 24 361503907875 25 361503907875 26 361503907875 27 361503907875 28 361503907875SQL> RECOVER DATABASE USING BACKUP CONTROLFILE UNTIL CANCEL;ORA-00279: change 361503907875 generated at 07/20/2018 14:00:39 needed forthread 1ORA-00289: suggestion : +ARCHDGORA-00280: change 361503907875 for thread 1 is in sequence #9922Specify log: { =suggested | filename | AUTO | CANCEL}+DATADG/orcl/redo01.logLog applied.Media recovery complete.SQL> alter database open RESETLOGS;SQL> select file#,CHECKPOINT_CHANGE# from v$datafile_header; FILE# CHECKPOINT_CHANGE#---------- ------------------ 1 361503908937 2 361503908937 3 361503908937 4 361503908937 5 361503908937 6 361503908937 7 361503908937 8 361503908937 9 361503908937 10 361503908937 11 361503908937 FILE# CHECKPOINT_CHANGE#---------- ------------------ 12 361503908937 13 361503908937 14 361503908937 15 361503908937 16 361503908937 17 361503908937 18 361503908937 19 361503908937 20 361503908937 21 361503908937 22 361503908937 FILE# CHECKPOINT_CHANGE#---------- ------------------ 23 361503908937 24 361503908937 25 361503908937 26 361503908937 27 361503908937 28 361503908937