SQL script to delete tape drives with "MISSING_DRIVE" status from the EMM database. There can be situations where there are drives in the EMM database with "MISSING_DRIVE" status, and it is not possible to delete them via "tpconfig -delete -drive " command.
Details of the issue:
----------------------------
A. "tpconfig -emm_dev_list -noverbose" command executed from a media server having problem will show entries similar to the following:
TPC_DEV60 DRIVE T10K_2_1_1_1 16 1 2 -1 - 2 1 1 1 media_server MISSING_DRIVE:531001002370 531001002370 - 128 4 -1 -1 -1 -1 0 11889 0 0 - 0 - - STK~~~~~T10000A~~~~~~~~~1.38 - 16
B. "tpconfig -l" executed on the media server may show the entry similar to the following:
drive - 16 hcart3 - DISABL - T10K_2_1_1_1 MISSING_DRIVE:531001002370 ACS=2, LSM=1, PANEL=1, DRIVE=1
C. To verify what are the entries in the Enterprise Media Manager (EMM) database & tables, run the following command:
/usr/openv/db/bin/nbdb_unload -t EMM_MAIN.EMM_Device, EMM_MAIN.EMM_MachineDeviceConnection, EMM_MAIN.EMM_DriveIndex
Entries like the following will appear in the nnn.dat files output received from the above command:
436.dat:
'2000185',0xECD8529A2C3213F88038865237F37639,'2','16','0','128','1','NetBackup HCART3','NetBackup HCART','523118080','16176','6','0','T10K_2_1_1_1',','2000176','2','1',','-1','STK','T10000A','1.38',',',','531001002370',',','STK T10000A 1.38',','0',','0','0','1970-01-01 00:00:00.000000','1970-01-01 00:00:00.000000','2009-04-23 19:33:18.
000000','0','42802796','0','0',0x00000000,0x00000000000000000000000000000000,'-1','-1','1970-01-01 00:00:00.000000','0','0','2','1','1','1',',',','0','0','0','8388608','2006-08-1502:18:16.184874','2009-05-18 23:22:46.007585'
444.dat:
'2002855',0x6F2D947E305911DE8000893E7E4907EB,'2','16','0','32907','1','NetBackup HCART3','NetBackup HCART','523118080','16176','6','0','T10K_2_1_1_1',','2000176','2','1',','-1','STK','T10000A','1.38',',',','531004007993',',','STK T10000A 1.38',','0',','1000036','1000036','1970-01-01 00:00:00.000000','1970-01-01 00:00:00.000000','2009-0
5-20 03:00:53.000000','0','248714','0','0',0x00000000,0x00000000000000000000000000000000,'-1','-1','2009-05-20 03:01:41.000000','1000036','1','2','1','1','1',','T17480','T17480','82,'0','0','8388608','2009-04-23 02:52:34.317443','2009-05-20 03:03:49.093255'
D. If "tpconfig -delete -drive" does not work, or if it works but entries like the above still appear from any commands executed (listed above), please engage Symantec Technical Support (who may require to engage Symantec Development, as otherwise a SQL script not approved by Symantec Development may further damage the EMM database) to get proper SQL script to delete the drives with MISSING_DRIVE status.
E. The above condition may lead to backup/duplication failures, and also may lead to unexpected slower performance from resource allocation and selection processes such as nbrb and mds (emm).
F. After the completion of successful removal of the drives with MISSING_DRIVE status, execute the following steps on NetBackup servers to start with a clean state in the environment:
----------------------------
A. "tpconfig -emm_dev_list -noverbose" command executed from a media server having problem will show entries similar to the following:
TPC_DEV60 DRIVE T10K_2_1_1_1 16 1 2 -1 - 2 1 1 1 media_server MISSING_DRIVE:531001002370 531001002370 - 128 4 -1 -1 -1 -1 0 11889 0 0 - 0 - - STK~~~~~T10000A~~~~~~~~~1.38 - 16
B. "tpconfig -l" executed on the media server may show the entry similar to the following:
drive - 16 hcart3 - DISABL - T10K_2_1_1_1 MISSING_DRIVE:531001002370 ACS=2, LSM=1, PANEL=1, DRIVE=1
C. To verify what are the entries in the Enterprise Media Manager (EMM) database & tables, run the following command:
/usr/openv/db/bin/nbdb_unload -t EMM_MAIN.EMM_Device, EMM_MAIN.EMM_MachineDeviceConnection, EMM_MAIN.EMM_DriveIndex
Entries like the following will appear in the nnn.dat files output received from the above command:
436.dat:
'2000185',0xECD8529A2C3213F88038865237F37639,'2','16','0','128','1','NetBackup HCART3','NetBackup HCART','523118080','16176','6','0','T10K_2_1_1_1',','2000176','2','1',','-1','STK','T10000A','1.38',',',','531001002370',',','STK T10000A 1.38',','0',','0','0','1970-01-01 00:00:00.000000','1970-01-01 00:00:00.000000','2009-04-23 19:33:18.
000000','0','42802796','0','0',0x00000000,0x00000000000000000000000000000000,'-1','-1','1970-01-01 00:00:00.000000','0','0','2','1','1','1',',',','0','0','0','8388608','2006-08-1502:18:16.184874','2009-05-18 23:22:46.007585'
444.dat:
'2002855',0x6F2D947E305911DE8000893E7E4907EB,'2','16','0','32907','1','NetBackup HCART3','NetBackup HCART','523118080','16176','6','0','T10K_2_1_1_1',','2000176','2','1',','-1','STK','T10000A','1.38',',',','531004007993',',','STK T10000A 1.38',','0',','1000036','1000036','1970-01-01 00:00:00.000000','1970-01-01 00:00:00.000000','2009-0
5-20 03:00:53.000000','0','248714','0','0',0x00000000,0x00000000000000000000000000000000,'-1','-1','2009-05-20 03:01:41.000000','1000036','1','2','1','1','1',','T17480','T17480','82,'0','0','8388608','2009-04-23 02:52:34.317443','2009-05-20 03:03:49.093255'
D. If "tpconfig -delete -drive
E. The above condition may lead to backup/duplication failures, and also may lead to unexpected slower performance from resource allocation and selection processes such as nbrb and mds (emm).
F. After the completion of successful removal of the drives with MISSING_DRIVE status, execute the following steps on NetBackup servers to start with a clean state in the environment:
1. Select a window with minimum or no jobs on the master server, if there are, cancel them. Make sure all jobs are stopped. To make sure no further jobs will be queued by nbpem during this activity, "nbpemreq -suspend_scheduling" can be run, and no manual backups, user backups or restores should be initiated during this time. If "nbpemreq -suspend_scheduling" is used, "nbpemreq -resume_scheduling" will have to be executed to resume the job scheduling when all the steps are done.
2. Run "nbrbutil -releaseAllocHolds" on the master server.
3. Run "nbrbutil -resetAll" on the master server.
4. Run nbrbutil -dump and take a copy for SYMC to review.
5. Stop all master server's daemons.
6. Recycle daemons on listed media servers those had missing drives before the execution of the SQL script. The media servers also can be found by running the following SQL command:
SELECT MachineKey, FQMachineName, MachinePrimaryName FROM "EMM_MAIN"."EMM_Machine" WHERE MachineKey IN (SELECT DISTINCT PrimaryMachineKey FROM "EMM_MAIN"."EMM_MachineDeviceConnection" WHERE PrimaryPath LIKE 'MISSING%'))
7. Recycle daemons on all remaining media servers.
8. Start NBU daemons on the Master server.
Comments:
Although only Solaris is mentioned here as the Operating System, the above condition may exist in any NetBackup environment with any supported Operating Systems.
3. Run "nbrbutil -resetAll" on the master server.
4. Run nbrbutil -dump and take a copy for SYMC to review.
5. Stop all master server's daemons.
6. Recycle daemons on listed media servers those had missing drives before the execution of the SQL script. The media servers also can be found by running the following SQL command:
SELECT MachineKey, FQMachineName, MachinePrimaryName FROM "EMM_MAIN"."EMM_Machine" WHERE MachineKey IN (SELECT DISTINCT PrimaryMachineKey FROM "EMM_MAIN"."EMM_MachineDeviceConnection" WHERE PrimaryPath LIKE 'MISSING%'))
7. Recycle daemons on all remaining media servers.
8. Start NBU daemons on the Master server.
Comments:
Although only Solaris is mentioned here as the Operating System, the above condition may exist in any NetBackup environment with any supported Operating Systems.
No comments:
Post a Comment