Archive for the ·

TSM

· Category...

TSM backup error: ANS1030E The operating system refused TSM request for memory

Comments Off

Problem:

A few AIX servers have TSM backup error:

 

ANS1030E The operating system refused a TSM request for memory allocation.

Solution:

 

  • Use the client option MEMORYEFFICIENTBACKUP YES. The maximum virtual memory used by the client process in this case becomes 300 bytes times the maximum number of files in any one directory.

Add the following line in dsm.sys and recycle dsmc scheduler:

               memoryefficientbackup yes
  •  Use the client option MEMORYEFFICIENTBACKUP DISKCACHEMETHOD. This option is only available on the Version 5.4 client, or later. The maximum virtual memory used by the client process in this case is usually less than 20 MB. A significant amount of client disk space might be required. See the Backup-Archive Clients Installation and User’s Guide for details.

For example:

Add the following line in dsm.sys and recycle dsmc scheduler:

INCLUDE.FS /your_filesystem_name MEMORYEFFICIENTBACKUP=DISKCACHEMETHOD DISKCACHELOCATION=/TSM_disk_cache

 

  • Use the Tivoli Storage Manager journal-based incremental backup function (Windows® client, and AIX® with a Version 5.3.2 client, or later). A full incremental backup must be completed before |a journal-based backup is possible.

 

  • Use the Tivoli Storage Manager image |backup function to back up the entire volume. This might require less time and resources than using incremental backups on some file systems with a very large number of very small files.

 

More explanation:

For incremental backups the client uses an average of about 300 bytes of memory for each file system object (a file or directory). These 300 bytes include the path and name of the object and attributes. The average memory requirement increases if longer file or directory names or more deeply nested directories are used. For a file system with 1 million files and directories, |for example, the client would require about 300 MB of virtual memory.

The size of a file system in gigabytes is not an important factor in determining the backup-archive client virtual memory requirements, except as it relates to the number of files and directories in that file system.

Virtual memory refers to the memory that is addressable by an application |process. The virtual memory size can be greater or smaller than the computer’s|real (physical) memory size. For example, a 32-bit application can address 4 GB of virtual memory. The application might be running on a workstation |that has only 512 MB of real memory, or it might be running on a server that has 8 GB of real memory.

|

Only part of an application’s virtual memory might be used for allocating |data structures. The maximum virtual memory available for allocation by a |32-bit process depends on the operating system. For Windows, |this is 2 GB for most processes. The Version 5.3.2 (and later) Windows client |can use up to 3 GB for data structures if the client system has been booted |using the /3gb boot.ini flag – . See Microsoft® Knowledge Base article 291988 |(http://support.microsoft.com/kb/291988). For AIX 5L™, this is nine memory |segments, or 2.25 GB. Other operating systems might have different limits. |Operating system quotas or limits might need to be set to allow a process |to use the maximum virtual memory. Refer to the ulimit command on UNIX systems.

|

The maximum number of files and directories that Tivoli Storage Manager can |back up on a 32-bit system (2 GB of process memory) using a single incremental |backup process and the default option MEMORYEFFICIENTBACKUP NO is about 7 |million: (2 * 1024 * 1024 * 1024) / (300 * 1000 * 1000). If the client process |tries to exceed the operating system process virtual memory limit, the backup |fails. If the client process memory requirements exceed the amount of real |memory on the system but do not exceed the virtual memory limit, then the |backup might succeed but is likely to exhibit poor performance due to paging.

Using a 64-bit client, if available for the platform in question, essentially |eliminates any concern about the virtual memory requirements, but might still |require significant real memory for optimal performance.

Comments Off

RC=0x0000006A=106 error during DB2 backup using Tivoli Storage Manager

Comments Off

Problem(Abstract)

Tivoli Storage Manager return code 106 seen in the db2diag.log during backup:

DATA #1 : TSM RC, PD_DB2_TYPE_TSM_RC, 4 bytes
TSM RC=0x0000006A=106 — see TSM API Reference for meaning.

Cause

The file permissions on the error log file (dsmierror.log/dsmerror.log)

Resolving the problem

The db2diag.log file can contain error messages during a failed Tivoli Storage Manager for Mail for SAP backup. For Tivoli Storage Manager API specific return codes, search for the text “TSM RC=” then reference the messages manual for the API return code meaning.

The following error was seen in the db2diag.log file during the archive logs backup:
2009-08-22-16.50.58.161181-300 I5302549A362       LEVEL: Warning
PID     : 53798                TID  : 4114        PROC : db2sysc 0
INSTANCE: db2pbw               NODE : 000
EDUID   : 4114                 EDUNAME: db2logmgr (PBW) 0
FUNCTION: DB2 UDB, data protection services, sqlpgArchiveLogFile,  probe:3108
MESSAGE : Started archive for log file S0082777.LOG.

2009-08-22-16.50.58.164580-300 E5302912A347       LEVEL: Error
PID     : 290858               TID  : 1           PROC : db2vend
INSTANCE: db2pbw               NODE : 000
EDUID   : 1
FUNCTION: DB2 UDB, database utilities, sqluvint, probe:374
DATA #1 : TSM RC, PD_DB2_TYPE_TSM_RC, 4 bytes
TSM RC=0x0000006A=106 -- see TSM API Reference for meaning.

2009-08-22-16.50.58.164968-300 E5303260A861       LEVEL: Error
PID     : 53798                TID  : 4114        PROC : db2sysc 0
INSTANCE: db2pbw               NODE : 000
EDUID   : 4114                 EDUNAME: db2logmgr (PBW) 0
FUNCTION: DB2 UDB, data protection services, sqlpInitVendorDevice, probe:1030

MESSAGE : ZRC=0x86100025=-2045771739=SQLP_MEDIA_VENDOR_DEV_ERR
          "A vendor device reported a media error."
DATA #1 : String, 29 bytes
Init failed!  Vendor rc info:
DATA #2 : Vendor RC, PD_DB2_TYPE_VENDOR_RC, 4 bytes
Vendor RC=0x0000000B=11 -- see DB2 API Guide for meaning.
DATA #3 : Hexdump, 48 bytes
0x0700000030698AD0 : 0000 006A 3337 3420 3130 3600 0000 0000
  ...j374 106.....
0x0700000030698AE0 : 0000 0000 0000 0000 0000 0000 0000 0000
  ................
0x0700000030698AF0 : 0000 0000 0000 0000 0000 0000 0000 0000
  ................

The API return code 106 means DSM_RC_ACCESS_DENIED and is referring to the error log file in the DSMI_LOG environment variable. To find out where this variable is pointing, use the following commands:

ps -elf | grep -i “db2sysc” | grep -i “<instance owner>”
ps eww <pid of db2sysc from the output above>

The output from the second command will return all the variables under which DB2 is running. Make sure that the error log has the proper R/W permissions for the user running the backup. If the variable is set incorrectly and needs to be changed, a restart of DB2 is required in order for the changed to take effect.

Please note that if there is an ERRORLOGNAME specified in the dsm.sys then this will take higher precedence and is the error log that needs to have write permissions.

If there is no ERRORLOGNAME, then the DSMI_LOG variable is used and should point to a directory that has write permissions. A dsierror.log file would be created in this DSMI_LOG directory location. If DB2 cannot be restarted to pickup a new setting for the DSMI_LOG variable, the ERRORLOGNAME option can be set to a writeable location/filename to work around the failure.

Comments Off

LANFREE timeouts and session processing

Comments Off

Question

Why are there multiple Storage Agent sessions for the LANFREE processing and what type of timeouts might be encountered during a LANFREE backup or restore.

Answer

For a LANFREE backup or restore operation within Tivoli Storage Manager, there are three timeouts that can be encountered:

  • Idletimeout
  • Commtimeout
  • Resourcetimeout

Three sessions are started when the Tivoli Storage Manager Storage Agent (STA) starts. The output from the STA will show these sessions initializing:

  • One sessions for the Paths
  • One session for the Event Logging
  • One session to initialize the Library sharing.

In many cases, these sessions terminate as there is no work to do. In some cases, they will transition into an IdleWait state.

When a backup or restore session begins, the following sessions are also initialized:

  • One session to the Tivoli Storage Manager Server from the client
  • One session to the Tivoli Storage Manager Server from the STA that is a proxy session for the client.
  • One session from the STA to the Tivoli Storage Manager Server that can be considered an administrative Library Client session. This session keeps statistics and tracks the data being backed up but does not perform the actual data transfer.
  • There are also Library Sharing sessions that are utilized to initiate the tape mounts/dismounts and are then terminated.

For every session on the STA, there is a corresponding session on the Tivoli Storage Manager Server. To verify the corresponding sessions, look at the tape volume being used and/or the “Proxy By Storage Agent” field on the Server side that shows the STA name.

If the proxy session from the STA for the client is not working (i.e, when it is waiting for data), the session that is seen on the Tivoli Storage Manager Server will be in a RecvWait (for a backup) or a SendWait (for restore).  The wait state is seen because it is waiting on something (a response) from the client.
For any delay in the transfer of data for the backup/restore, the RecvW and SendW sessions are seen on the Tivoli Storage Manager Server, the corresponding STA session goes into an IdleW. In order to prevent this from causing a failure, the IDLETIMEOUT setting on the Server and STA should be set to a high enough value to accommodate any expected delays in processing of the data. It is important to note that the RESOURCETIMEOUT value on the Tivoli Storage Manager server and the IDLETIMEOUT value on the storage agent will both come into play for the proxied storage agent sessions.

For example, during an incremental backup for some database applications, there are delays in the sending of the data such that the session will witness a RecvW. During a restore there may be times when a client application will first request a small amount of data to be restored. The client will then use this information to create the files/tablespaces that are needed to hold the restore data. While the application is preparing the files, the session for the Tivoli Storage Manager is in a SendWait. The restore will only start after the client application has completed the necessary processing to prepare for the restore. Both of these scenarios would see sessions in a wait state that can result in a timeout failure if the values are not set high enough to accommodate the delay.

All of these values will disconnect the session if they are reached:

  • The RESOURCETIMEOUT value is set on the Tivoli Storage Manager Server. Even if the RESOURCETIMEOUT is set for the STA, the Server value will override the STA setting.
  • The IDLETIMEOUT value needs to be set on both the Tivoli Storage Manager Server and the STA.
  • In the case of LAN backups, the COMMTIMEOUT value on the Tivoli Storage Manager Server will be utilized. Note that the COMMTIMEOUT should also be set on the STA.
Comments Off