NetWorker Client Push

Today we are going through, how to use Networker client push features

Use the software distribution feature, Client Push, to update the NetWorker and module software on NetWorker hosts in the datazone from the NetWorker server. By using this feature, you do not need to log in to each host and manually uninstall the old NetWorker software version and install the new NetWorker software version.

You cannot use Client Push to update a clustered host, an NMM 2.4.x or earlier client, aYou cannot use Client Push to update a clustered host, an NMM 2.4.x or earlier client, aNetWorker server, an EMC License Manager server, a Console server, a PowerSnap client,or an OS-X client.
There are two different scenarios to consider when preparing the software repository toupdate the NetWorker software by using Client Push:

1. The NetWorker server is the same operating system as the target hosts
2. The NetWorker server and target host operating systems differ

The NetWorker server is the same operating system as the target hosts:
When the target hosts are the same operating system as the server, extract each software
packages to the Media Kit Location. The Media Kit Location must reside on a file system
that is local to the NetWorker server.

The NetWorker server and target host operating systems differ:
When the operating system of the target hosts differs from the NetWorker server, for
example, when a Windows NetWorker server updates a UNIX client, you must configure a
proxy host to store the cross-platform packages.

Here I am only go with The NetWorker server is the same operating system as the target hosts.
Procedure:

1. Login to the NMC console with a user that is both an NMC administrator and a member of the Application administrators user group on the NetWorker server.

2. Connect to the NetWorker server in NMC.
3. In the Configuration menu >>>Select Configuration tab as below and, select Software Administration Wizard.

4. On the Welcome to the Software Administration Wizard window, click Next.

5. On the Select Operation window:
a. Select Add or remove products from my software repository.

b. Click Next.
6. On the Software Repository Operations window:
a. Select Add software products to the repository.

b. Click Next.
7. On the Create Software Repository window:

a. Click Yes.
b. Click Next.
8. If this is the first time you are creating the repository, specify the Repository location,
and click Next .
The default location differs on Windows and UNIX:
Windows: NetWorker_install_dir\repository
UNIX: /nsr/repository

9. On the Select platform type window:
a. Select the platform type of the software to add to the repository.

b. Click Next.
10. On the Add software products to repository window:
a. Click Yes .
b. Click Next.
11. On the Media Kit Location window:
a. Specify the path to the Media Kit location.
As per Unix admin I received path “/data/depot/networker/LINUX/x86_64lgto/linux_x86_64.v8113/”

b. Click Next.
12. The Select Products window contains a list of detected software packages in the
repository:
a. Select the products to add to the repository.

b. Click Next.

Inventorying the target hosts for installed products:
Before you can update the NetWorker software on a host, you must inventory the host to
determine the installed versions of NetWorker software.

After Configure all these things you can go ahead with upgrade:
Procedure:
1. Start the Software Administration Wizard and click Next.
2. Select Upgrade the software on my NetWorker clients, and click Next.

3. Select the option By Client, will upgrade on the clients that you select, and click Next.

4. Select the appropriate hosts, and click Next

5. In Product available for upgrade select the client again and click next

6. The Monitor Activity window appears and provides the status of upgrade job in the
top table and the status of each client upgrade operation in the bottom window, for
the selected upgrade job. When the upgrade job contains many clients, the job might
take awhile to complete.

Upgraded client successfully: - As per above process we are able to upgrade Linux client from server client push features.

RPC error: RPC program or version mismatch

We have looking today for RPC error: RPC program or version mismatch

There is a mismatch between the version of the Remote Procedure Call (RPC) service requested by the client and the service available on the server. 

This problem might be the result of an incorrect "Name to Address" resolution. 

There is a mismatch between the version of the RPC service used by the License Manager (lgtolmd) and the services available on the server. The default RPC version for lgtolmd is two.

Use rpcinfo to verify that the server and the clients are running the same version of RPC. For operating systems without the rpcinfo utility, it is provided on the EMC product CD.

Update the NetWorker software on the client or server.

Verify that the "Name to Address" resolution is correct. Test by running nslookup using the fully qualified domain name of the computer. On Windows systems, this verification also can be performed from the Network control panel.

Specify the same version of RPC for lgtolmd that is used by the other backup services on the backup server. This can be specified by either:

Entering the following command at the command line:

# lgtolic -s hostname:rpc_number:rpc_version


Setting the LMD_VERSION environment variable.

Networker backup group hung and unable to stop by GUI

Networker backup group hung and unable to stop by GUI

Error: "Group must be started either automatically or from the GUI and must be currently running in order to be stopped"

You can handle this scenario in two ways.

•         You can kill the job PID by command line (make sure not PPID, read differenbetween PID and PPID)

•                Another way is, use to savegrp command.

                  bash-$ savegrp -pvvv groupname

Make sure use first savegrp option if not resolved then you can use kill PID option

EMC NetWorker backup logs files paths and details for Windows

Hi,

Today here I would like to provide details of the log files available on a Windows Networker server and log

file management.

Windows Networker Server Log Files:-

Windows NMC Server Log files:

Windows Networker client Log Files:-

There is two types of logs files:

*.log >> It is plan text file and human readable
*.raw>> It is not human readable format


You can view manually raw files with help of nsr_render_log Networker command.
You can also set it in runtime with help of nsradmin and "update runtime rendered log" Networker configuration commands.

The nsr_render_log send the output result to stdout

You can also rollover the files by time and size with help of below commands:

nsradmin
update runtime rollover by time
update runtime rollover by size


Thanks

EMC NetWorker backup logs files paths and details for Unix/Linux

Hi,

Today here I would like to provide details of the log files available on a Unix Net-worker server and log

file management.

Networker Server Log Files:-

Networker Server Log Files:-

Networker Client Log files:-

There is two types of logs files:


*.log >> It is plan text file and human readable
*.raw>> It is not human readable format


You can view manually raw files with help of nsr_render_log Networker command.
You can also set it in runtime with help of nsradmin and "update runtime rendered log" Networker configuration commands.


The nsr_render_log send the output result to stdout


You can also rollover the files by time and size with help of below commands:


nsradmin
update runtime rollover by time
update runtime rollover by size


Thanks

How to Delete VBA Node in Networker

How to Delete VBA Node in Networker

Networker Ver: 8 to 9

If you want to remove VBA node entirely and delete it's backups from Networker server. The VBA node is online and responding to request from Networker server. Before Doing anything please consult with EMC Support for Assistance because VBA Node delete/remove is not recommended by EMC as a troubleshooting step.

Before starting any critical configuration you should take first backup of nsrdb and mm directories (Particular directory which related with that configuration)

Connect to Networker backup server and open the following directory

 <networker-install>\nsr

e.g   

·         Unix: /nsr/ 

·         Windows C:\Program Files\EMC NetWorker)

Make a copy of the following directories:

res\nsrdb

res\mm

After successfully copied mentioned directories you can start your actual configuration.

Remove all save sets for the client-id for this VBA node

List the Saveset:

mminfo  –c  <vba-name>  -kot

Remove Saveset:

nsrmm -d -S <SSID>

If the VBA is NOT online, then use the -X option to delete the Save set:

nsrmm -X -d -S <SSID>

Remove all workflows associated with the VBA and Delete VBA Mtree

Using GUI (NMC):

1. Login to NMC

2. Go to Configuration tab and click VMware Protection Policies

3. Sort the policies by VBA List

4. Open the first policy that has this VBA listed, go to VMware Backup Appliance Selection      tab and uncheck the VBA name.

5. Wait for the State column to change to Success

Using CLI (nsradmin):

nsradmin> p type: nsr hypervisor; name: <vcenter-name>

nsradmin> Delete

Remove VBA server resource using nsradmin

 d type: NSR VBA Server; name: <vba>

Note: This step may need to be done using offline nsradmin if VBA is NOT responding

Delete the VBA Appliance Delete the relevant VBA node VM from the vCenter server

On the NetWorker server, run nsradmin command utility to find the VBA DPN ID:

nsradmin> show name

nsradmin> show DPN ID

nsradmin> p type: nsr vba server; name: VBA-FQDN

To delete the storage unit on Data Domain:

Login to Data Domain as Admin and run command,

ddboost storage-unit delete <storage-unit>

Clean file system if required using command, filesys clean start

If the External proxy is no longer needed delete the external proxy virtual machine

How to check validcopies in NetWorker 9 with mminfo

How to check validcopies in NetWorker 9 with mminfo:

In your backup environment data clone going to a disk media, I suspect that we are rarely will see a save set with clflg mminfo attributes. The reason is that 'faulty/aborted' save sets will automatically be deleted.

If you want to know whether a save set has successfully been cloned you better check with 'validcopies' mminfo attribute:


mminfo -r "client,name,ssid,validcopies,pool"

or to verify all save sets with only one copy

mminfo -q "validcopies<2" -r "client,name,ssid,validcopies,pool"

Browse and retention time raised Script for multiple SSID

Today I would like to explain How Browse and retention time period extend by Script for multiple SSID

bash-$ mminfo -avot -q "savetime>=02/01/2016,savetime<=03/31/2016" -r client,ssid,group,ssbrowse,ssretent >ssid.txt 2>&1

Select the respective SSID which need to extend and put in to one text file ssid-re.txt. After that run the below script for extend the Browse and retention period;

Today Date : 04/01/2016

bash-$ for i in `cat ssid-re.txt`; do sudo nsrmm -S $i -e "6 months" -y; done

bash-$ for i in `cat ssid-re.txt`; do sudo nsrmm -S $i -w "6 months" -y; done

You will get the error browse time should be less than retention time.

You should mention commands like as 

for i in `cat ssid-re.txt`; do sudo nsrmm -S $i -w "Sun Feb 28 06:16:05 2016" -y; done

After extended the both policy verify it.

for i in `cat verifyssid.txt`; do mminfo -q ssid=$i -r 'client,ssid,sscreate,ssbrowse,ssretent' >>output.txt; done 

VSS System Writer is not found on this system

The VSS System Writer is not found on this system:

While backing up Windows 2008 and 2012 server on EMC Networker and you are getting below error, there will be many causes for this error. This is a very common permission issue.

98266:save: The VSS System Writer is not found on this system.  The backup is stopping because the System Writer is required for a successful system state backup.  Have the system administrator check the VSS infrastructure and fix the issue.

You should first try below solution:

Start or Restart below services:

1. COM+ Event System

2. COM+ System Application

3. Distributed Transactions Coordinator

4. Microsoft Software Shadow Copy Provider

5. Volume Shadow Copy service

6. Networker remote exec service

And try to take test backup, if it fails again then you should go with below Microsoft solution.

The system writer fails because permissions to files in the %windir%\winsxs\filemaps\ or %windir%\winsxs\temp\PendingRenames directories are incorrect.

Solution:

To resolve this issue, type the following commands from an elevated command prompt:

Takeown /f %windir%\winsxs\temp\PendingRenames /a
icacls %windir%\winsxs\temp\PendingRenames /grant "NT AUTHORITY\SYSTEM:(RX)"
icacls %windir%\winsxs\temp\PendingRenames /grant "NT Service\trustedinstaller:(F)"
icacls %windir%\winsxs\temp\PendingRenames /grant BUILTIN\Users:(RX)
Takeown /f %windir%\winsxs\filemaps\* /a
icacls %windir%\winsxs\filemaps\*.* /grant "NT AUTHORITY\SYSTEM:(RX)"
icacls %windir%\winsxs\filemaps\*.* /grant "NT Service\trustedinstaller:(F)"
icacls %windir%\winsxs\filemaps\*.* /grant BUILTIN\Users:(RX)

net stop cryptsvc
net start cryptsvc

Type the following command to verify that the system writer is now listed: vssadmin list writers


If the system writer is missing, check the Application event log for the following event

for more info check Microsoft Technical Tips :

https://support.microsoft.com/es-ar/kb/2009272#/es-ar/kb/2009272

How to BACK UP the NetWorker Management Console (NMC)

How to BACK UP the NetWorker Management Console (NMC):

We can backup NMC through command line as well as GUI method.

First we will see the command line method:

Open the command line interface and run the command “savepsm” which will backup the NMC database. The normal Networker “save” command cannot back up NMC database successfully if it is open during backup hence it is always better to use "Savepsm" command.

Now we will see using GUI method:

Login to NMC console

Go to SETUP

Click the SETUP button on the toolbar

Select “Set Database Backup Server”

 Mention the NetWorker server name that you want to back up the NMC database.

Check "Create client resource on this server".

Set Client name to the hostname of the machine that has the NMC database.

Click OK.

After successfully configuration connect to the NetWorker server. When you look at the list of clients, you will get there a new client.

You will also see that client configuration saveset name starting with "NMCASA:/gst_on_NMC-Server-name/lgto_gst" with Backup Command attribute: savepsm (for a Windows Console server) or savepsm.sh(for a UNIX Console server).

You should verify the backup which you configured after successful first backup with mminfo command.

On Windows: mminfo -avot -s (NetWorker server) | find /i "gst"

On UNIX: mminfo -avot -s (NetWorker server) | grep -i gst

This are the method you can use to backup your NMC database.

NSR peer information' provided did not match

NSR peer information' provided did not match

nsrexecd GSS critical An authentication request from backupclient.com  was denied

If you experience errors in the backup server daemon log such as this:

89879 07/04/14 04:04:13  nsrexecd GSS critical An authentication request from backupclient.com  was denied. The 'NSR peer information' provided did not match the one stored by backupserevr. To accept this request, delete the 'NSR peer information' resource with the following attributes from backupserevr's NSRLA database: name: backupclient.com; NW instance ID: XXXXXXXX-XXXXXXXX-XXXXXXXX-XXXXXXXX-XXXXXXXX-XXXXXXXX; peer hostname: backupclient.com

It is because a client has been re-installed, renamed or the NSRLADB has been rebuilt.

To resolve this, delete the NSR peer information for the client on the backup server.

For example:

# nsradmin -p nsrexecd

NetWorker administration program.

Use the "help" command for help, "visual" for full-screen mode.

nsradmin> . type:NSR peer information;name:backupclient.com

Current query set

nsradmin> delete

                        type: NSR peer information;

               administrator: "isroot,host= backupserevr ",

                              "isroot,host=backupserevr.com",

                              "user=root,host=localhost";

                        name: backupclient.com;

               peer hostname: backupclient.com;

          Change certificate: ;

    certificate file to load: ;

Delete? y

deleted resource id 0.0.XX.XXX.0.0.0.XXX.83.XXX.XXX.XXX.XXX.XX.XX.11(1)

nsradmin> quit

EMC Data Domain Advanced Command Troubleshoot

EMC Backup and Recovery with Data Domain

How to Delete a Networker License key

Open "Command Prompt" "Run As Administrator" or Unix "Root User"

To stop NetWorker services and dependent server services, "NetWorker Backup and 
Recover Server" and "EMC GSTService". on a Windows host:


Make sure you keep copy of res directory.

C:\>cd "C:\Program Files\EMC NetWorker\nsr\res"

C:\>nsradmin -d nsrdb

bash-$sudo nsradmin -d "/nsr/res/nsrdb"

NetWorker administration program.
Use the "help" command for help.
nsradmin>

Enter the following commands at the nsradmin prompt:

nsradmin> type: NSR license

nsradmin> print

nsradmin> delete

Delete? y

nsradmin> print

No resources found for query:

type: NSR license;

nsradmin> q

Start the NetWorker services and "EMC GSTService"

How to recover the Clone Save Set

How to test cloned save set recovery
How to test single file cloned recovery

We have two Networker Sites and Two Data Domain Sites

Networker1 : 

Data Domain 1 : Backup of Networker1 is stored on DD1 and Cloning to DD2

Data Domain 2 : Disaster Site DD 

We have data on both Sites.

But for testing we have to restore cloned save set (e.g from DD2 only single file daemon.raw file)

Backup Data Domain Volume: DD1Volume.AA
Clone Data Domain Volume:   DD2CloneVolume.BB

In this scenario  you should run first normal recovery like as below:

bash-4.1$ sudo recover -c backuptest.com -R backuptest.com -iR

Current working directory is /

recover> cd /nsr/logs  (Check your files path)

recover> add daemon.raw

1 file(s) marked for recovery

recover> volume

Volumes needed (all on-line):

        DD1Volume.AA at rd=storagenode01.backuptest.com:DD1DISK01

recover>

But above volume is belong to DD1 not Cloned DD2.

Go to following path:

Login NMC >>> Backup Server >>> Media  >>> Disk Voulumes >>> DD1Volume.AA

For single file you cannot find there SSID you can find SSID for whole save set.

Then run recover again with same procedure like above.

bash-4.1$ sudo recover -c backuptest.com -R backuptest.com -iR

Current working directory is /

recover> cd /nsr/logs  (Check your files path)

recover> add daemon.raw

1 file(s) marked for recovery

recover> volume

Volumes needed (all on-line):

        DD2CloneVolume.BB at rd=storagenode02.backuptest.com:DD2DISK01

And recovered on same server on E:\ drive

recover> relocate

6479:recover: <return> will recover files into their original location

New destination directory: E:\restored_11-11-2015

recover> recover

After Test recover is completed you can revert the changes:

 Login NMC >>> Backup Server >>> Media  >>> Disk Voulumes >>> DD1Volume.AA and mark save set from suspect to Normal.

If you want to recover whole save set like “/” you can find the SSID and Clone ID using mminfo command.

bash-4.1$ mminfo -q client=backuptest.com-ot -r "client,name,volume,sscreate,ssretent,ssid,cloneid"

client                   Save Set name         volume               created        ssid          clone id

backuptest.com      /                      DD1Volume.AA          12/02/2015 123456789  1111111111

backuptest.com     /                      DD2CloneVolume.BB   12/02/2015 123456789  2222222222

recover> recover -S 123456789/2222222222

For recovery more details check my below posts:

<http://learnbackupwithmoin.blogspot.com/2015/11/save-set-recover-from-command-prompt.html>

How to recover expired save set

• I am going to explain how to recover the expired save set or How to recover expired EMC Networker Data  or How to make Expired Save Sets to browsable for particular save set:

• Note:  If the browse and retention policy for the saveset has expired and the tape has been reused, you will be unable to recover the contents of the Saveset.

• Backup Software : EMC Networker

I will provide details as below scenario:

• Today Date : 12/05/2013

• Client want to restore data : 06/12/2013

• Browse/Retention Policy: Three month only

When I tried to browse the date 06/12/2013 with changetime command.

recover> changetime 06/12/2013

18360:recover: cannot browse /P/USER/ZZZZ55/docs/Outlook/ as of `06/12/2013 23:59:59' (mm/dd/yyyy)

recover>

Note: Need to follow below step to find the Index Tape related to that date,               ran mminfo to find the ssid of the index save set I'm looking for.

mminfo -r "name,savetime(17),ssid,cloneid,volume,location,level,ssretent" -q "client=Backup Server,name=index:client Name,savetime>06/01/13,savetime<06/30/13" -ot

name            savetime(17)      ssid     cloneid    level    volume      location    ssretent sumflags

index:BackupClient   6/12/2013   62433892   1371148564 full   11103L5      IBM3310      9/4/2013  cE

cE : E indicate Expired Save set

Make tape not recyclable as well as extend the expire time by Type -S and -o manully on command line:

nsrmm -S <ssid/CloneID> -e <new expiration date> ( DD/MM/YYYY )

nsrmm -S <ssid/CloneID> -o notrecyclable

nsrmm –S 62433892/1371148564  –e 12/12/2013

nsrmm –S 62433892/1371148564   –o notrecyclable

mminfo -r "name,savetime(17),ssid,cloneid,level,volume,location,ssretent,sumflags" -q "client=BackupServer,name=index:BackupClient,savetime>06/01/13,savetime<06/30/13" -ot

name   savetime(17) ssid cloneid    level  volume  location ssretent sumflags

index:BackupClient  6/12/2013  62433892 1371148564  full  11103L5  IBM3310  12/12/2013  cr

Cr : r indicate recoverable save set

State changed from cE to Cr Expired to recoverable and Expire date now changed from 09/04/2013 to 12/12/2013.

Then tried to rebuild index now.

nsrck -L7 -t MM/DD/YYYY <clientname> /  nsrck -v -L7 -t MM/DD/YYYY <clientname>

nsrck -t 06/12/13 -L7 BackupClient

recover> changetime 06/12/2013

18360:recover: cannot browse /P/USER/ZZZZ55/docs/Outlook/ as of `06/12/2013 23:59:59' (mm/dd/yyyy)

recover>

But we are getting same cannot browse error

Then search Data Tape and mark as recoverable and not recyclable and rebuild index.

mminfo -s BackupServer -r "client,name,savetime,ssid,cloneid,volume,location,ssretent,sumflags,level" -q "client=BackupClient,savetime>06/01/2013,savetime<06/30/2013" -ot | grep full

client             name    savetime      ssid           cloneid     volume         location   ssretent  sumflags level

BackupClient P:\user  06/12/13 414719219  1371108606 111023L5      IBM3310   09/04/13  hE        full

BackupClient P:\user  06/12/13 414719219  1371108606 111092L5      IBM3310   09/04/13  tE         full

hE : E indicate Expired Save set

tE : E indicate Expired Save set

nsrmm -S <ssid/CloneID> -e <new expiration date> ( DD/MM/YYYY )

nsrmm -S <ssid/CloneID> -o notrecyclable

nsrmm –S 414719219/1371108606  –e 12/12/2013

nsrmm –S 414719219/1371108606  –o notrecyclable

mminfo -s BackupServer -r "client,name,savetime,ssid,cloneid,volume,location,ssretent,sumflags,level" -q "client=BackupClient,savetime>06/01/2013,savetime<06/30/2013" -ot | grep full

client             name    savetime      ssid           cloneid        volume        location   ssretent    sumflags level

BackupClient P:\user  6/12/2013 414719219  1371108606 000023L5      IBM3310  12/12/2013   hr        full

BackupClient P:\user  6/12/2013 414719219  1371108606 000092L5      IBM3310  12/12/2013   tr         full

hr : r indicate recoverable save set

tr : r indicate recoverable save set

State changed from hE, tE to hr, tr Expired to recoverable and Expire date now changed from 09/04/2013 to 12/12/2013.

Then again try to rebuild indexes:

nsrck -L7 -t MM/DD/YYYY <clientname> /  nsrck -v -L7 -t MM/DD/YYYY <clientname>

nsrck -t 06/12/13 -L7 BackupClient

recover> changetime 06/12/2013

18361:recover: time changed to 06/12/2013 23:59:59 (mm/dd/yyyy)

Able to browse now 

Add all files and restored files as per my previous post which as below:

http://learnbackupwithmoin.blogspot.com/2015/11/directed-saveset-recovery-command-line.html

Also check Below Useful Commands:

• mminfo -avot (a for all, v for verbose, o for order and t for time)

• mminfo -q "volumename=<volname>" -r "name,ssid,savetime(26),ssbrowse(26),ssretent(26)" - this queries the media database for information relating to a specific volume

• mminfo -q"client=<clientname>" -r"name,ssid,ssbrowse(26),ssretent(26),ssflags,savetime(26)" - this queries the media database for information relating to a specific client

•  mminfo -avV -q"ssid=<ssid>" -r"ssbrowse(26),ssretent(26),savetime(26)"

• nsrinfo -t "US date and time for saveset" <clientname> - this is to list the files in the saveset

•  nsrmm -S <ssid/CloneID> -e <new expiration date> ( DD/MM/YYYY )

• nsrmm -S <ssid/CloneID> -o notrecyclable

•  nsrck -L7 -t MM/DD/YYYY <clientname> /  nsrck -v -L7 -t MM/DD/YYYY <clientname>

• mminfo -avV -q"ssid=<ssid>" -r"ssbrowse(26),ssretent(26)"

• scanner -i -S <ssid> <device path as configured in jukebox>

• scanner -S ssid -iv \\.\device.

• nsrjb -ln -S <slot number where tape is located> -f <device path as configured in jukebox>