One of my customers ( Oracle Database 11gR2 on LINUX ) never noticed the Oracle software LUN filling up until a “df -h” reported 97% used, wondered why, and of course wanted it to be solved asap. Well, after some investigation I found the adump directory to be the problem.
At this site, database auditing is active, and although a proper cleanup and archiving of the audit log files is in place, this process stalls from time to time for reasons unknown. In this case the result was an adump directory with just under 700.000 files!
Removing the audit files proved to be a problem with LINUX command “rm”, because of the Error: “Argument list too long”. This can be solved with a combination of find and exec, for instance, with adump as current directory, “[ find ./ -type f -exec rm {} \;” or “[find ./ -type f –delete ]”. But I wasn’t very impressed by the performance, and looked for better and faster… and this is what I came to use:
[oracle@PRD ~]$ [oracle@PRD ~]$ mkdir /tmp/empty [oracle@PRD ~]$ ls /u01/app/oracle/admin/PRODDB/adump | wc -l 694088 [oracle@PRD ~]$ time rsync -aq --include=*.aud --include=*.xml --exclude=* --delete /tmp/empty/ /u01/app/oracle/admin/PRODDB/adump/ real 0m39.604s user 0m2.597s sys 0m19.086s [oracle@PRD ~]$ ls /u01/app/oracle/admin/PRODDB/adump | wc -l 15 [oracle@PRD ~]$ rmdir /tmp/empty [oracle@PRD ~]$
As you can see, this “rsync” statement is really fast, it takes a mere 20 seconds to remove all files, and can be configured to delete files with certain extensions, leaving any other file in the directory. The trick is that you synchronize an empty directory ( /tmp/empty ) with another directory ( /u01/…./adump ), and telling rsync with “–delete” to remove certain files from adump ( see the include list ), not present in the empty directory. Result is the delete of all files, configured in the include list. Rather surprising is the fact that you can do the same trick with robocopy under Windows, as demonstrated:
-1- Create empty directory and test directory with 6 test files and 3 different extensions
C:\Users\harry_d> C:\Users\harry_d>mkdir test empty C:\Users\harry_d\test>type nul> test01.xml C:\Users\harry_d\test>type nul> test02.xml C:\Users\harry_d\test>type nul> test01.aud C:\Users\harry_d\test>type nul> test02.aud C:\Users\harry_d\test>type nul> test01.txt C:\Users\harry_d\test>type nul> test02.txt C:\Users\harry_d\test>dir Volume in drive C has no label. Volume Serial Number is 90E1-85B4 Directory of C:\Users\harry_d\test 30-03-2014 14:56 . 30-03-2014 14:56 .. 30-03-2014 14:55 0 test01.aud 30-03-2014 14:55 0 test01.txt 30-03-2014 14:55 0 test01.xml 30-03-2014 14:55 0 test02.aud 30-03-2014 14:56 0 test02.txt 30-03-2014 14:55 0 test02.xml 6 File(s) 0 bytes 2 Dir(s) 26.941.235.200 bytes free C:\Users\harry_d\test>
-2- Remove the 2 files with .aud extensie
C:\Users\harry_d\test> C:\Users\harry_d\test>robocopy C:\Users\harry_d\empty\ C:\Users\harry_d\test\ *.aud /purge /e /r:0 /w:0 C:\Users\harry_d\test> ----------------------------------------------------------------------------- ROBOCOPY :: Robust File Copy for Windows ------------------------------------------------------------------------------- Started : Sun Mar 30 15:03:29 2014 Source : C:\Users\harry_d\empty\ Dest : C:\Users\harry_d\test\ Files : *.aud Options : /S /E /COPY:DAT /PURGE /R:0 /W:0 ------------------------------------------------------------------------------ 0 C:\Users\harry_d\empty\ *EXTRA File 0 test01.aud *EXTRA File 0 test02.aud ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 1 0 1 0 0 0 Files : 0 0 0 0 0 2 Bytes : 0 0 0 0 0 0 Times : 0:00:00 0:00:00 0:00:00 0:00:00 Ended : Sun Mar 30 15:03:29 2014 C:\Users\harry_d\test>
… The 2 .aud files are deleted:
C:\Users\harry_d\test> C:\Users\harry_d\test>dir Volume in drive C has no label. Volume Serial Number is 90E1-85B4 Directory of C:\Users\harry_d\test 30-03-2014 15:03 . 30-03-2014 15:03 .. 30-03-2014 14:55 0 test01.txt 30-03-2014 14:55 0 test01.xml 30-03-2014 14:56 0 test02.txt 30-03-2014 14:55 0 test02.xml 4 File(s) 0 bytes 2 Dir(s) 26.941.235.200 bytes free C:\Users\harry_d\test>
-3- Remove files with .aud en .xml extension… notice the fact that all files with .aud extension are already deleted
C:\Users\harry_d\test> C:\Users\harry_d\test>robocopy C:\Users\harry_d\empty\ C:\Users\harry_d\test\ *.aud *.xml /purge /e /r:0 /w:0 ------------------------------------------------------------------------------- ROBOCOPY :: Robust File Copy for Windows ------------------------------------------------------------------------------- Started : Sun Mar 30 15:04:50 2014 Source : C:\Users\harry_d\empty\ Dest : C:\Users\harry_d\test\ Files : *.aud *.xml Options : /S /E /COPY:DAT /PURGE /R:0 /W:0 ------------------------------------------------------------------------------ 0 C:\Users\harry_d\empty\ *EXTRA File 0 test01.xml *EXTRA File 0 test02.xml ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 1 0 1 0 0 0 Files : 0 0 0 0 0 2 Bytes : 0 0 0 0 0 0 Times : 0:00:00 0:00:00 0:00:00 0:00:00 Ended : Sun Mar 30 15:04:50 2014 C:\Users\harry_d\test>
… Now the 2 .xml files are also deleted… only 2 .txt files remaining:
C:\Users\harry_d\test> C:\Users\harry_d\test>dir Volume in drive C has no label. Volume Serial Number is 90E1-85B4 Directory of C:\Users\harry_d\test 30-03-2014 15:04 . 30-03-2014 15:04 .. 30-03-2014 14:55 0 test01.txt 30-03-2014 14:56 0 test02.txt 2 File(s) 0 bytes 2 Dir(s) 26.941.235.200 bytes free C:\Users\harry_d\test>
-4- Remove all files with .aud and .xml and .txt extension … but no output this time!
C:\Users\harry_d\test> C:\Users\harry_d\test>robocopy C:\Users\harry_d\empty\ C:\Users\harry_d\test\ *.aud *.xml *.txt /purge /e /r:0 /w:0 > nul C:\Users\harry_d\test>
… The remaining 2 .txt files are deleted… no files left!…:
C:\Users\harry_d\test>dir Volume in drive C has no label. Volume Serial Number is 90E1-85B4 Directory of C:\Users\harry_d\test 30-03-2014 15:06 . 30-03-2014 15:06 .. 0 File(s) 0 bytes 2 Dir(s) 26.941.194.240 bytes free C:\Users\harry_d\test>cd .. C:\Users\harry_d>rmdir empty test C:\Users\harry_d>
Use the provided code at your own discretion, and be careful. Rsync and robocopy are both very powerful commands, and should be tested properly before applied in production environments. For both rsync and robocopy this delete|purge trick is a nice but not well-known feature, because it is more of an option in synchronizing or backing up directories or even complete file systems. But there’s no harm in picking up such an option, and putting it to good use where it’s clearly very useful!
That’s an awesome technique Harry.
I am about to revisit a script that deal with deleting or moving large number of files on WIndows (which is a notoriously slow filesystem), xcopy and rmdir are too slow, so your suggestions will be put to good use. Thanks.