RMAN fails with kgefec: fatal error 0 | kgepop: no error frame to pop to for error 603

Noticed one RMAN archivelog backup job has been failing with following error.

kgefec: fatal error 0

kgepop: no error frame to pop to for error 603

I was using the backup code as below. 

RMAN> backup archivelog all not backed delete input;




After doing some research, it was found that this issue comes either we have less resource at our DB server or when there are excessive number of archive logs to be backed with DELETE INPUT clause. In my case, OS level limits were set to UNLIMITED which was fine but I could see that there were more than 5100 archivelogs in there.

Just deleted those archives which were already backed up 1 times to TAPE.

RMAN>delete force archivelog all backed up 1 times to device type 'SBT_TAPE';

Now initiated the archive backup again after records clean-up and it went fine this time.

RMAN> backup archivelog all not backed up delete input;

RMAN> backup archivelog all not backed up delete input;

Starting backup at 12-JUN-19
current log archived
released channel: ORA_DISK_1
released channel: ORA_DISK_2
released channel: ORA_DISK_3
released channel: ORA_DISK_4
released channel: ORA_DISK_5
released channel: ORA_DISK_6
released channel: ORA_DISK_7
released channel: ORA_DISK_8
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: SID=937 instance=testdb1 device type=SBT_TAPE
channel ORA_SBT_TAPE_1: Data Domain Boost API
allocated channel: ORA_SBT_TAPE_2
channel ORA_SBT_TAPE_2: SID=23 instance=testdb1 device type=SBT_TAPE
channel ORA_SBT_TAPE_2: Data Domain Boost API
allocated channel: ORA_SBT_TAPE_3
channel ORA_SBT_TAPE_3: SID=323 instance=testdb1 device type=SBT_TAPE
channel ORA_SBT_TAPE_3: Data Domain Boost API
allocated channel: ORA_SBT_TAPE_4
channel ORA_SBT_TAPE_4: SID=632 instance=testdb1 device type=SBT_TAPE
channel ORA_SBT_TAPE_4: Data Domain Boost API
allocated channel: ORA_SBT_TAPE_5
channel ORA_SBT_TAPE_5: SID=932 instance=testdb1 device type=SBT_TAPE
channel ORA_SBT_TAPE_5: Data Domain Boost API
allocated channel: ORA_SBT_TAPE_6
channel ORA_SBT_TAPE_6: SID=1228 instance=testdb1 device type=SBT_TAPE
channel ORA_SBT_TAPE_6: Data Domain Boost API
allocated channel: ORA_SBT_TAPE_7
channel ORA_SBT_TAPE_7: SID=1521 instance=testdb1 device type=SBT_TAPE
channel ORA_SBT_TAPE_7: Data Domain Boost API
allocated channel: ORA_SBT_TAPE_8
channel ORA_SBT_TAPE_8: SID=24 instance=testdb1 device type=SBT_TAPE
channel ORA_SBT_TAPE_8: Data Domain Boost API
channel ORA_SBT_TAPE_1: starting archived log backup set
channel ORA_SBT_TAPE_1: specifying archived log(s) in backup set
input archived log thread=2 sequence=49069 RECID=15597 STAMP=1010709576
input archived log thread=1 sequence=49981 RECID=15596 STAMP=1010709576
channel ORA_SBT_TAPE_1: starting piece 1 at 12-JUN-19
channel ORA_SBT_TAPE_2: starting archived log backup set
channel ORA_SBT_TAPE_2: specifying archived log(s) in backup set
input archived log thread=1 sequence=49982 RECID=15599 STAMP=1010709742
input archived log thread=2 sequence=49070 RECID=15598 STAMP=1010709741
channel ORA_SBT_TAPE_2: starting piece 1 at 12-JUN-19
channel ORA_SBT_TAPE_3: starting archived log backup set
channel ORA_SBT_TAPE_3: specifying archived log(s) in backup set
input archived log thread=2 sequence=49071 RECID=15601 STAMP=1010709861
input archived log thread=1 sequence=49983 RECID=15600 STAMP=1010709859
channel ORA_SBT_TAPE_3: starting piece 1 at 12-JUN-19
channel ORA_SBT_TAPE_4: starting archived log backup set
channel ORA_SBT_TAPE_4: specifying archived log(s) in backup set
input archived log thread=1 sequence=49984 RECID=15603 STAMP=1010709876
input archived log thread=2 sequence=49072 RECID=15602 STAMP=1010709876
channel ORA_SBT_TAPE_4: starting piece 1 at 12-JUN-19
channel ORA_SBT_TAPE_5: starting archived log backup set
channel ORA_SBT_TAPE_5: specifying archived log(s) in backup set
input archived log thread=2 sequence=49073 RECID=15605 STAMP=1010711116
channel ORA_SBT_TAPE_5: starting piece 1 at 12-JUN-19
channel ORA_SBT_TAPE_6: starting archived log backup set
channel ORA_SBT_TAPE_6: specifying archived log(s) in backup set
input archived log thread=1 sequence=49985 RECID=15604 STAMP=1010711113
channel ORA_SBT_TAPE_6: starting piece 1 at 12-JUN-19

channel ORA_SBT_TAPE_1: finished piece 1 at 12-JUN-19

Finished backup at 12-JUN-19

Starting Control File and SPFILE Autobackup at 12-JUN-19

piece handle=c-2679281492-20190612-00 comment=API Version 2.0,MMS Version 4.5.0.0

Finished Control File and SPFILE Autobackup at 12-JUN-19

RMAN>