The scrubber program deletes (scrubs) data container files from the cleartext storage pools and derived object (DO) storage pools of one or more VOBs. It also deletes corresponding (DOs) from a VOB database. Only cleartext pools and DO pools are affected; scrubbing is not defined for source pools.
Note: DOs are associated with dynamic views only; they are not applicable to snapshot views.
scrubber implements the following scrubbing algorithms:
By default or with the –o option, scrubber uses a free-space-analysis heuristic: it compares the current free-space level of a disk partition with a lower limit computed during its previous execution. This lower limit is stored in file /var/adm/rational/clearcase/cache/scrubber_fs_info (UNIX) or ccase-home-dir\var\cache\scrubber_fs_info (Windows).
- If the free-space level is still above the computed limit, scrubber does no scrubbing in that partition, regardless of the state of the storage pools within it. This performance optimization allows a quick check to take place frequently (for example, once an hour), without much system overhead.
- If the free-space level has fallen below the limit, scrubber performs parameter-driven scrubbing of each storage pool in the partition.
With the –f option, scrubber removes data container files from a storage pool according to the pool's scrubbing parameter settings. (The heuristic scrubbing algorithm can also fall through to this algorithm.)
When a derived object pool or cleartext pool is created with mkvob or mkpool, its scrubbing parameters are set to user-specified or default values:
Parameter-driven scrubbing proceeds as follows:
- Files are removed from a pool only if its current size exceeds its maximum size setting. In this case, scrubber begins deleting data containers that have not been referenced within age hours, proceeding on a least-recently-referenced basis.
- The data container for a derived object is deleted only if the DO's reference count is zero. In this case, the derived object in the VOB database is deleted, too. The associated configuration record is also deleted if no other derived object is associated with it.
- Cleartext data containers do not have reference counts; they are deleted solely on the basis of recent use.
- Scrubbing stops when the pool's size falls below its reclaim size setting. But in no case does scrubber delete any object that has been referenced within the last age hours.
A maximum size of zero is a special case: it instructs scrubber to delete all data containers that have not been referenced within age hours, regardless of the reclaim size setting.
Note: The scrubber considers access time rather than modification time. If your backup utility changes the access time on objects, scrubber does not delete the object if the backup utility ran within the period of time specified by age.
With the –e option, scrubber ignores a pool's scrubbing parameters and deletes these files:
By default, the scheduler runs scrubber periodically with the –f option, so that each pool is examined individually. For information about describing and changing scheduled jobs, see the schedule reference page.
You can scrub one or more pools manually at any time.
Scrubber Log File
scrubber documents its work in the host's scrubber log file:
For example, the following partial report describes the results of scrubbing a derived object pool.
04/27/99 08:03:00 Stats for VOB betelgeuse:/usr1/vobstorage/orange.vbs
04/27/99 08:03:00 Get cntr tm 918.928979
04/27/99 08:03:00 Setup tm 10631.121127
04/27/99 08:03:00 Scrub tm 1207.099240
04/27/99 08:03:00 Total tm 12757.149346
04/27/99 08:03:00 Start size 404789 Deleted 3921 Limit size 0
04/27/99 08:03:00 Start files 20349 Deleted 121 Subdir dels 0
04/27/99 08:03:00 Statistics for scrub of DO Pool ddft:
04/27/99 08:03:00 DO's 3671 Scrubs 121 Strands 1760
04/27/99 08:03:00 Lost refs 1790 No DO's 20228
04/27/99 08:03:00 No fscntrs 2
The first six lines, which contain elapsed times and file statistics, are included in the report for every pool. The last four lines are specific to DO pools.
|Get cntr tm||Elapsed time for first scrubbing phase: walk the file system tree to get pathname, size, and referenced-time information for each container in the pool.|
|Setup tm||Elapsed time for second scrubbing phase: perform setup processing specific to the kind of storage pool. For a cleartext pool, no setup is required. For a DO pool, setup is complicated; see “Processing of Derived Object Pools”.|
|Scrub tm||Elapsed time for third scrubbing phase: determine which containers to delete and then delete them.|
|Start size||Total size (KB) of all the container files in the storage pool directory before this scrubbing.|
|Deleted||Amount of storage (KB) reclaimed by this scrubbing.|
|Limit size||Desired size of the pool (KB), as specified by the pool's maximum size parameter.|
|Start files||Total number of container files in the storage pool directory before this scrubbing.|
|Deleted||Number of container files deleted by this scrubbing.|
|Subdir dels||Number of empty subdirectories of the storage pool directory deleted by this scrubbing.|
|DO's||Total number of zero-reference-count DOs in the VOB database before scrubbing.|
|Scrubs||Total number of shared zero-reference-count DOs deleted by this scrubbing. (This number equals the Deleted count, unless the scrubber removed shared zero-reference-count DOs that were missing their file system containers.)|
|Strands||Total number of stranded DOs deleted by this scrubbing. (These are described below.)|
|Lost refs||Total number of lost DO reference counts deleted by this scrubbing. (These are described below.)|
|No DO's||Total number of containers in the DO pool before scrubbing that are not associated with a zero-reference-count shared DO. (Each is presumably associated with a DO that is still referenced by some view, and hence cannot be scrubbed).|
number of shared zero-reference-count DOs that were missing their file system
This statistic is printed only when this condition occurs; also, the scrubber_log displays warning messages like this one:04/21/99 10:11:17 scrubber: Warning: Unable to remove “d/do_pool2/21/5/73f1f66679f611cea15c080009935288”: No such file or directory.
Processing of Derived Object Pools
For a DO pool, scrubber does more than delete old, unreferenced data containers.
- It finds and deletes all stranded DOs from the VOB database: DOs that were never shared and whose data containers have been deleted from view-private storage. (The VOB database is not updated when the DO's data file is removed or overwritten in the view, because of implementation restrictions.) There are no data containers in the DO storage pool for such DOs, because they were never shared. This occurs during the second phase of scrubbing.
- It finds and deletes all lost DO reference counts from the VOB database. Such entries are an implementation artifact; they correspond to files that were created during a build, but deleted before the build completed. This occurs during the second phase of scrubbing.
- It deletes the derived object in the VOB database corresponding to the data container, and possibly its associated configuration record as well. This occurs during the third phase of scrubbing.
- It finds and deletes all stranded configuration records: CRs that do not correspond to any existing derived object.
Some interesting results can be derived from these statistics:
OPTIONS AND ARGUMENTS
Specifying the Scrubbing Algorithm
- Invokes the free-space-analysis heuristic
described above, instead of examining pools individually.
- Examines all specified pools individually,
using the parameter-driven algorithm. This does not guarantee that any objects
will be removed from the pools.
- Examines all specified pools individually
(as with –f), using the everything-goes scrubbing
- Same as default.
Specifying the Pools
- All of a VOB's cleartext and derived object
pools are scrubbed.
- –p pool[,...]
- Restricts scrubbing to pools with the specified
names, which may occur in multiple VOBs. The list of pool names must be comma-separated
with no white space.
- –k kind[,...]
- Restricts scrubbing to pools of the specified
kinds. Valid kinds are do and cltxt.
The list of kinds must be comma-separated with no white space.
Specifying the VOBs
- Scrubs all VOBs listed in the storage registry
whose storage directories reside on the local host. An error occurs if a
VOB is listed in the registry, but cannot be found on the local host.
- vob-storage-dir-pname ...
- One or more pathnames of VOB storage directories,
indicating the particular VOBs to be scrubbed.
- Force scrubbing of all mounted VOBs with a storage directory on the local host.
- Scrub cleartext pools in the VOB whose storage directory is /usr/vobstore/project.vobs, using the free-space analysis heuristic.
- Force scrubbing of the default derived object pool (ddft) and the pool named do_staged in all mounted VOBs with a storage directory on the local host.