Help with tar

Fri Feb 15 19:29:46 PST 2002

Aloha Yuser,

	I will include a copy of my backup script to this e-mail below so you 
can follow along.  The script is not perfect, and may likely generate a 
lot of suggestions for improvement on the list... but it pretty much does 
what I want it to do, which is perform full system backups excluding 
certain directories.  Essentially my script creates a file that contains 
the list of directories to be backed up.  The file containing the list of 
directory names is stored in the variable "$TARFILENAME."  The script 
command that does this magic is this one:

ls -1 | egrep -v '(proc|mnt|core)' > $TARFILENAME

BTW, that's "ls -one" not "ls -el."  So, list the filenames one per line, 
pipe that to egrep and don't list anything that is "proc, mnt, or core."  
Store these names in the $TARFILENAME.  From here, the command "tar -cb64 
-f /dev/st0 -T $TARFILENAME" does the backup on the scsi tape on /dev/st0 
with a 64x512 byte block size. (See tape drive documentation for the 
ideal size for your drive, or use the default.) The -T option tells tar 
to get the list of files to back up from the file stored in $TARFILENAME. 

	From here, it should be pretty easy to modify the "ls -1 | egrep -v 
'(proc|mnt|core)' > $TARFILENAME" to do whatever you want it to do.  You 
could for instance strip out anything in the "ie" directory, or you could 
be more specific and match certain file names explicitly.  

	Another solution would be to use the unix "find" command to construct a 
list of files.  You could for instance run    
 something like 

[root at VALinux /]# find / -type d ! \( -name "proc" -prune \)   

to see a list of all your directories except those beginning with "proc" 
and then feed that to a file listing that tar can use.   The -prune 
option tells find to not dive into any subdirectories if it finds one 
with "proc" in the  directory name.   You could easily change this to 
slice out any directory name you wish, even "Temporary Internet Files."

	I also have a Cache cleaner that I use to clean out my netscape caches 
before I backup the box.  If you do something like that before you back 
up, you can save the headache here and keep the backup script 
concentrating on just the backups, and the cache cleaner can clean up 
those files you don't want to backup before you run tar. That script is 
also included below after the backup script.

Hope this helps,

Ben  

########################################################################
###

#!/bin/bash

# MakeTOC:  a script to build a table of contents file and 
# create a tarfile for a full system backup. 

# 	By Ben Beeson
#	9 Feb 00

#	**********     TO DO     **********
#
#	Add some mag tape commands so that the 
#	script advances the tape to the end of
#	the valid data, and then does its thing.
#	This should make it a little easier to not
#	loose any archives accidentally.
#
#	Use the "no-rewind" tape drives
#	Have the script record the last block, and 
#	successful completion of the backup task and
#	send it in an email to root.....
#
#       Move the ps ax statement and find a way to fork
#	the tar command so you can get the process ID.  
#
#
#	***********************************

# First create the table of contents file.
# The table of contents file is called "tar_FULL_$today.TOC".
#
#	when generating the TOC file, skip the following 
#	directories:
#	/proc
#	/mnt

#	skip /proc so you don't waste time backing up process 
#		tags that are meaningless during a system restore.

#	skip /mnt so you don't back up other filesystems.  If you
#	want to do this, you can change behavior in the 
#	"ls -1 | egrep -v '(proc|mnt|core)' > $TARFILENAME"line below. 

# 	ignore any "core" files laying around

#	send the list of files down the pipe to the TOC file

# set the $today variable 

today=`date +%d%b%y%H%M%S`
echo "Today's date is: $today" 

# Place the table of contents file in the /tmp directory. 
# The table of contents file will be deleted later in the script.... 

TARFILENAME=/tmp/tar_FULL_$today.TOC

echo "creating TOC file: $TARFILENAME"
touch $TARFILENAME
# needs a test to see if file was actually created...

if (test -e $TARFILENAME)
	then
		echo "$TARFILENAME successfully created..."
	else
		# Should never get here...
		echo " Trouble in paradise, the Table of Contents file was not 
created."
  		echo " exiting backup script now...."
		exit 1
fi

# After a closer look, it appears that I should back up /tmp and /dev, so
# I won't strip them out in the `egrep` command below.... 

# create tarfile now
echo "creating tarfile on /dev/st0"

# Charlie Davis from Oak Ridge suggested I strip out the "/" 
directory.... Thanks Charlie!
# Charlie Davis at Oak Ridge National Labs suggested this one:

# Change to the root directory.  
cd /  

# Then get the tree that you want
ls -1 | egrep -v '(proc|mnt|core)' > $TARFILENAME

echo "Backing up the following directories:  "
cat $TARFILENAME

# Now back up the files to the tape drive.
# Turn verbose switch off...
# Use block size of 32KB.  HP C1599A DAT book page 33 recommends
# a transfer size of 32 KB either fixed or variable.
# Tar command man page sez blocksize is N*521byes..

tar -cb64 -f /dev/st0 -T $TARFILENAME 

echo "tar process ID is:  `ps ax | grep tar |  
			   egrep -v '(grep|startx)' | 
			   cut -d" " -f2` "

echo "done with backup ..."

echo "now cleaning up mess and deleting tempfiles"

# if TARFILENAME exists delete it so you don't fill up the /tmp 
directory....

if (test -e $TARFILENAME)
	then
		echo " deleting $TARFILENAME..."
		rm $TARFILENAME
	else
		echo "$TARFILENAME was not there to delete."		
fi

# done    

########################################################################
###########

Cache Cleaner script is next ...

########################################################################
###########

#! /bin/bash
#	CacheClean 
#
#
# This script expects to find the netscape cache in a directory 
# that looks like this: 
#
#	 $HOME/.netscape/cache/??  
#
#	Of course, you can edit this to suit your taste. :-)
#
#
# Echo the command entered. 
# See the bottom of the script for: echo "CacheClean: $@"
#
#######################################################################
usage="A script to clean out the netscape cache directories.
	This script looks for directories of the form: ~/.netscape/cache/??/*

	By Ben Beeson
	Version 1.1
	13 Feb 2000

	Usage:  CacheClean " 
#
# echo $usage to display the usage comments on the console.
#
#######################################################################

gethomedirectories()
{
# Get a list of potential $HOME directories from the /etc/passwd file
# and then sequentially check them for the presence of a netscape cache.
# If the cache is present, clean it...

touch /tmp/HOMEDIRECTORIES
if (test -e /tmp/HOMEDIRECTORIES) 	# if /tmp/HOMEDIRECTORIES exists
	then 
		echo "created /tmp/HOMEDIRECTORIES file."
		# Load the potential home directories in the /tmp/HOMEDIRECTORIES file
		cat /etc/passwd | gawk -F: '{print $6}' >> /tmp/HOMEDIRECTORIES
		# send the list of potential directories to the terminal
		echo "consider these directories"
		cat /tmp/HOMEDIRECTORIES
		echo " "
	else
		echo "could not create the /tmp/HOMEDIRECTORIES file."
		echo "exiting now..."
		exit 1
fi
}

#######################################################################

testhomedirectory()
{
# Test the potential directory for the presence of a netscape/cache 
directory. 

for testdir in `cat /tmp/HOMEDIRECTORIES`
    do
	# set a variable to hold the directory tree name
	CACHEDIR=$testdir/.netscape/cache
	echo "Testing $CACHEDIR"
	if (test -e $CACHEDIR)  	# if $CACHEDIR exists
		then                    # clean it....
		    cd $CACHEDIR    # Change to the selected directory
				    # cd $CACHEDIR ***before*** calling cleanit 
		    cleanit         # `find` in cleanit will fail if you don't
				    # because the PWD will be incorrect. 

		else
		    echo "Directory $CACHEDIR does not exist. Skip it..."
	fi
   done
}
#######################################################################
cleanit()
{
	#
	# Before we do anything else, delete any previous versions of the 
	# ./cachetemp file that may be laying around.

	if (test -e ./cachetemp) 	# if ./cachetemp exists
		then 
		rm ./cachetemp
	fi

	# Create a tmpfile containing the file names to be deleted.
	# Place this file in the current directory.

	echo "creating tempfile $CACHEDIR/cachetemp"
	touch ./cachetemp

	# Insert the cached file names into the tempfile.
	# Make sure you are in the correct directoy. 
	# -- See gethomedirectories() above.
	echo "PWD is: $PWD"
	find ./??/* -name '*' -print | cat >> ./cachetemp

	# The above uses the unix find command to search that path and list 
	# every file that is cached.  Don't forget to append the trailing / 
	# to the directory pathname. 

	# Do the deed....

	rm `cat ./cachetemp`	# This is what actually deletes the cache files.

	# Clean up the mess. Delete the tempfile "./cachetemp"
	echo "Deleting the tempfile:  ./cachetemp"

	rm ./cachetemp
}

#######################################################################

#######################################################################
#
# The main part of script is here....
#
#######################################################################
echo "CacheClean: $@"

gethomedirectories
testhomedirectory

#######################################################################
#
# Clean up mess
#

if (test -e /tmp/HOMEDIRECTORIES) 	# if tempfile exists
	then
		echo "deleting /tmp/HOMEDIRECTORIES."
		rm /tmp/HOMEDIRECTORIES
	else 
		echo "could not find the tempfile /tmp/HOMEDIRECTORIES"
		exit 1 # should not get to here.... 
fi

echo "Done CacheClean script"

#done

########################################################################
###########

>>>>>>>>>>>>>>>>>> Original Message <<<<<<<<<<<<<<<<<<

On 2/15/02, 4:16:12 PM, yuser at hi.net (yuser) wrote regarding [luau] Help 
with tar:

> I need help with some tar options..

> I am trying to use tar to backup my /home directory into a file.
> I have the contents of some directories that I do not want backed up.
> Example..

> Almost every user has a directory called
> some_user/ie/Temporary Internet Files and various others that I really
> do not to backup

> I have tried the --exclude option but it seems to only take exact
> names, meaning that to exclude "yuser/ie/Temporary Internet Files"
> I could add this to the --exclude, but what about the other users
> Temporary Internet Files folders?  Is there a way to prevent achiving
> any directory that is named Temporary Internet Files without adding a
> specific exclude statement for every user that has that directory?
> What would really be nice is for tar to create the directory called
> Temporary Internet Files in the backup but not backup any of the files
> that are in that directory, I have not got that far yet.

> Maybe I should be using one of the backup script packages and not
> trying to use tar directly.

> ---
> You are currently subscribed to luau as: beesond001 at hawaii.rr.com
> To unsubscribe send a blank email to $subst('Email.Unsub')
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freesoftwarehawaii.org/pipermail/luau-freesoftwarehawaii.org/attachments/20020216/6736242f/attachment-0001.htm>