TSM Server

Sometimes there are specific errors, such as:
ANR0523W Transaction failed – error on output storage device
ANR8300E I/O error on library (OP=xx, CC=xx, KEY=xx, ASC=xx, ASCQ=xx, SENSE=xx)
ANR8301E I/O Error on library
ANR8355E I/O error reading label for volume NNNNNN on drive XXXXX
ANR8359E Media fault detected on volume NNNNNN in drive XXXXX
ANR8441E Initialization failed for SCSI library
ANR8779E Unable to open drive XXXXX, error number=ZZZ
ANR8944E Hardware or media error on drive
ANR8963E Unable to find path to match the serial number defined for drive

Frequently the TSM Server can automatically rediscover devices when using “SANDISCOVERY ON” or by using “UPDATE PATH” with “AUTODETECT=YES” to refresh the values.

However, there are times when that may not be successful. For example, if a tape drive, tape library, fibre/SCSI HBA, or SAN has experienced changes (such as hardware, firmware or device drivers) it may require rebuilding the TSM “special files” to re-establish connectivity to the library and drives. To rebuild the “special files,” we must delete and re-define the hardware devices to the TSM Server (UPDATE does not rebuild).

Perform these tasks in this sequence to totally re-define the tape devices to TSM. These steps should be taken only if attempts to update the devices/paths using the autodetect features have failed:
1) Before deleting anything, gather the output from these commands, so you can use the same naming conventions when re-defining the tape devices:
  QUERY STATUS (get SERVERNAME value for “<tsm_server>”)
  QUERY DEVCLASS
  QUERY LIBRARY FORMAT=DETAIL
  QUERY DRIVE FORMAT=DETAIL
  QUERY PATH FORMAT=DETAIL

2) Run the appropriate OS command to produce a list of the configured HW ‘special file’ device names.
      AIX   ==>   lsdev -Cc tape           (-or- 'cfgmgr')
                  lsdev -Cc adsmtape       (for TSM devices)
                  lsdev -Cc library
   Solaris  ==>   ls -l /dev/rmt/*st       (-or- 'sysdef')
                  ls -l /dev/rmt/*smc
     HP-UX  ==>   /usr/sbin/ioscan -funC tape
                  (-or  'ioscan -kfn')
     Linux  ==>   ls -l /dev/IBM*
                  ls -l /dev/tsmscsi/*
                  (-or- 'more /etc/sysconfig/hwconf')

If the tape devices are not defined to the OS, please work with your OS or SCSI/SAN hardware support to configure them. Until the OS can use the drives (can write to them, for example using ‘tar’ or ‘dd’) the tape devices cannot be defined to TSM.

3) From the ‘/dev’ directory, write down the OS-level device definitions for the library and drives:
                  AIX     Linux         Solaris    HP-UX
  TSM Drives      mt#     tsmscsi/mt#   rmt/#      rmt/tsmmt#
  IBM Drives      rmt#    IBMtape#      rmt/#st    rmt/#m
  TSM Library     lb#     tsmscsi/lb#   rmt/#lb    tsmchgr#
  358x Library    smc#    IBMchanger#   rmt/#smc   rmt/#chng
  3494 Library    lmcp#   3494lib       libmgrc#   libmgrc#

4a) First the drives and drive paths must be deleted. From a TSM Server admin commandline, for all the drives:
   DELETE PATH  <tsm_server>  <drive_name>  SRCTYPE=SERVER  DESTTYPE=DRIVE  LIBRARY=<library_name>

4b) Then delete all the TSM drive definitions:
   DELETE DRIVE  <library_name>  <drive_name>

5a) Next, delete the path for the tape library:
   DELETE PATH  <tsm_server>  <library_name>  SRCTYPE=SERVER  DESTTYPE=LIBR

5b) And finally delete the TSM library definition:
   DELETE LIBRARY  <library_name>

If the OS cannot access the tape drives at this point, stop. Check hardware, device drivers, update firmware, swap cables; consider power-cycling the tape library then deleting and re-defining to the OS. There is no point attempting to get TSM to write to the devices if they are not recognized by the OS; work with OS and/or hardware vendors to resolve HW issues before proceeding.

6a) Now the tape library and library path can be re-defined. Use the TSM QUERY outputs from “Step 1″ as a guide for the library name and LIBTYPE; no additional parameters are necessary in the syntax below. Redefine the library:
   DEFINE LIBRARY  <library_name>  LIBTYPE=<library_type>  SERIAL=AUTODETECT

Note: If this TSM Server is hosting a tape library for other systems, for example any “TSM Server Library Clients” or “TSM Storage Agents” then you also need “SHARED=YES” on the “DEFINE LIBRARY”.

6b) Redefine the path to the library. For SCSI libraries, confirm the DEVICE value matches the latest OS-level info gathered from “Step 2″. For 3494, ACSLS, and other types of libraries using software configuration files, use the previous values from “Step 1″ to redefine the DEVICE or ACSID, and so on:
   DEFINE PATH  <tsm_server>  <library_name>  SRCTYPE=SERVER  DESTTYPE=LIBRARY  DEVICE=</dev/lb#>

7a) Redefine the drives and drive paths. Redefine all the drives using names from “Step 1″ for example:
   DEFINE DRIVE  <library_name>  <drive_name>  SERIAL=AUTODETECT  ELEMENT=AUTODETECT

7b) Redefine paths to all drives, using the OS-level info gathered from “Step 2″ for the DEVICE values. Keep in mind the OS-level DEVICE values may have changed since the they were previously defined.
   DEFINE PATH  <tsm_server>  <drive_name>  SRCTYPE=SERVER  DESTTYPE=DRIVE  LIBRARY=<library_name>  DEVICE=</dev/mt#>

Note: If this TSM Server is hosting a tape library for other systems, for example any “TSM Server Library Clients” or “TSM Storage Agents” then in addition to the “TSM Server Library Manager” DRIVE PATH, you also need to define a new PATH for each drive for those systems, substituting the SERVERNAME (shown by “Q SERVER”) for the value of “<tsm_server>” and the local DEVICE value for the drive as seen by that other system.

8) Verify the library, drives, and paths are online:
  QUERY LIBRARY  <library_name>  FORMAT=DETAIL
  QUERY DRIVE   <library_name>  *  FORMAT=DETAIL
  QUERY PATH  *  *  FORMAT=DETAIL

9) Since the library is “new” to TSM, the volumes must be checked in again to re-create the inventory (AUDIT LIBRARY does not CHECKIN). Use *this* sequence, first SCRATCH, then PRIVATE:
  CHECKIN LIBVOL  <library>  SEARCH=Y STATUS=SCR CHECKL=BARC
  CHECKIN LIBVOL  <library>  SEARCH=Y STATUS=PRIV CHECKL=BARC

NOTE: For ACSLS libraries, use “CHECKLABEL=NO” on the CHECKIN commands, because “CHECKLABEL=BARCODE” is not supported for an ACSLS Library.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

If that doesn’t resolve the issue, the tape drive problem seems beyond the control of the TSM Server (software). Review the output from the OS-level logs for additional hardware error information:
                          Remove OS    Install
      OS   Diagnostics    Devices      OS Devices
 -------   ------------   ---------    ------------
     AIX   errpt –a       rmdev        cfgmgr
   Linux   dmesg                       /dev/MAKEDEV
 Solaris   mbin/prtdiag   rem_drv      drvconfig
   HP-UX   dmesg          rmsf         insf -e

If you cannot reach HW support immediately, you could take the additional action of power-cycling in this order:
1) Tape library.
2) SAN switch (if any).
3) Consider updating to latest device drivers and/or firmware.
4) Halt TSM and reboot system with TSM Server.
5) Re-define the tape device to the OS (see commands above).
6) If tape device definitions have changed, DELETE & re-DEFINE to TSM.

And that is all we can do from a software perspective, if errors persist it points to an issue at a layer which TSM cannot repair.