Program Tip

특정 파일 / 폴더를 제외한 tar 디렉토리에 대한 쉘 명령

programtip 2020. 9. 29. 18:23
반응형

특정 파일 / 폴더를 제외한 tar 디렉토리에 대한 쉘 명령


특정 파일 / 폴더를 보관에서 제외하는 것을 지원하는 간단한 셸 명령 / 스크립트가 있습니까?

백업 할 필요가없는 매우 큰 파일이 많이있는 하위 디렉터리로 보관해야하는 디렉터리가 있습니다.

해결책이 아닙니다.

tar --exclude=PATTERN명령은 주어진 패턴과 일치하고 해당 파일을 제외하지만 특정 파일 및 폴더를 무시해야합니다 (전체 파일 경로). 그렇지 않으면 유효한 파일이 제외 될 수 있습니다.

또한 find 명령을 사용하여 파일 목록을 만들고 보관하지 않으려는 파일을 제외하고 목록을 tar로 전달할 수 있지만 이는 적은 양의 파일에서만 작동합니다. 나는 수만 개가 있습니다.

유일한 해결책은 제외 할 파일 / 폴더 목록이있는 파일을 만든 다음 rsync with --exclude-from=file사용 하여 모든 파일을 tmp 디렉터리에 복사 한 다음 tar를 사용하여 해당 디렉터리를 보관하는 것이라고 생각하기 시작했습니다.

누구든지 더 나은 / 더 효율적인 솔루션을 생각할 수 있습니까?

편집 : Charles Ma 의 솔루션이 잘 작동합니다. 큰 문제는 tar 명령의 시작 부분에 --exclude='./folder' 있어야 한다는 것 입니다. 전체 명령 (cd 먼저, 백업은 해당 디렉토리에 상대적 임) :

cd /folder_to_backup
tar --exclude='./folder' --exclude='./upload/folder2' -zcvf /backup/filename.tgz .

tar에 대한 여러 제외 옵션을 가질 수 있으므로

$ tar --exclude='./folder' --exclude='./upload/folder2' -zcvf /backup/filename.tgz .

등이 작동합니다. 확인 확인 넣어 --exclude 전에 소스 및 대상 항목.


--excludetar를 사용 하여 디렉토리를 제외 할 수 있습니다 .

당신은 제외한 모든 보관할 경우 /usr사용할 수 :

tar -zcvf /all.tgz / --exclude=/usr

귀하의 경우 아마도

tar -zcvf archive.tgz arc_dir --exclude=dir/ignore_this_dir

tar를 사용하여 백업에서 파일 / 디렉토리를 제외하는 가능한 옵션 :

여러 패턴을 사용하여 파일 제외

tar -czf backup.tar.gz --exclude=PATTERN1 --exclude=PATTERN2 ... /path/to/backup

패턴 목록으로 채워진 제외 파일을 사용하여 파일 제외

tar -czf backup.tar.gz -X /path/to/exclude.txt /path/to/backup

건너 뛸 디렉토리에 태그 파일을 배치하여 태그를 사용하여 파일 제외

tar -czf backup.tar.gz --exclude-tag-all=exclude.tag /path/to/backup

많은 답변을 가진 오래된 질문이지만 나에게 충분히 명확한 질문이 없다는 것을 알았으므로 시도를 추가하고 싶습니다.

다음 구조가있는 경우

/home/ftp/mysite/

다음 파일 / 폴더 포함

/home/ftp/mysite/file1
/home/ftp/mysite/file2
/home/ftp/mysite/file3
/home/ftp/mysite/folder1
/home/ftp/mysite/folder2
/home/ftp/mysite/folder3

따라서 / home / ftp / mysite (사이트를 새 서버로 이동하기 위해) 내부의 모든 내용을 포함하는 tar 파일을 만들고 싶지만 file3쓰레기 일 뿐이며 모든 내용 folder3도 필요하지 않으므로이 두 가지를 건너 뛸 것입니다.

우리는 형식을 사용합니다

tar -czvf <name of tar file> <what to tar> <any excludes>

where the c = create, z = zip, and v = verbose (you can see the files as they are entered, usefull to make sure none of the files you exclude are being added). and f= file.

so, my command would look like this

cd /home/ftp/
tar -czvf mysite.tar.gz mysite --exclude='file3' --exclude='folder3'

note the files/folders excluded are relatively to the root of your tar (I have tried full path here relative to / but I can not make that work).

hope this will help someone (and me next time I google it)


You can use standard "ant notation" to exclude directories relative.
This works for me and excludes any .git or node_module directories.

tar -cvf myFile.tar --exclude=**/.git/* --exclude=**/node_modules/*  -T /data/txt/myInputFile.txt 2> /data/txt/myTarLogFile.txt

myInputFile.txt Contains:

/dev2/java
/dev2/javascript


I've experienced that, at least with the Cygwin version of tar I'm using ("CYGWIN_NT-5.1 1.7.17(0.262/5/3) 2012-10-19 14:39 i686 Cygwin" on a Windows XP Home Edition SP3 machine), the order of options is important.

While this construction worked for me:

tar cfvz target.tgz --exclude='<dir1>' --exclude='<dir2>' target_dir

that one didn't work:

tar cfvz --exclude='<dir1>' --exclude='<dir2>' target.tgz target_dir

This, while tar --help reveals the following:

tar [OPTION...] [FILE]

So, the second command should also work, but apparently it doesn't seem to be the case...

Best rgds,


This exclude pattern handles filename suffix like png or mp3 as well as directory names like .git and node_modules

tar --exclude={*.png,*.mp3,*.wav,.git,node_modules} -Jcf ${target_tarball}  ${source_dirname}

I found this somewhere else so I won't take credit, but it worked better than any of the solutions above for my mac specific issues (even though this is closed):

tar zc --exclude __MACOSX --exclude .DS_Store -f <archive> <source(s)>

For those who have issues with it, some versions of tar would only work properly without the './' in the exclude value.

Tar --version

tar (GNU tar) 1.27.1

Command syntax that work:

tar -czvf ../allfiles-butsome.tar.gz * --exclude=acme/foo

These will not work:

$ tar -czvf ../allfiles-butsome.tar.gz * --exclude=./acme/foo
$ tar -czvf ../allfiles-butsome.tar.gz * --exclude='./acme/foo'
$ tar --exclude=./acme/foo -czvf ../allfiles-butsome.tar.gz *
$ tar --exclude='./acme/foo' -czvf ../allfiles-butsome.tar.gz *
$ tar -czvf ../allfiles-butsome.tar.gz * --exclude=/full/path/acme/foo
$ tar -czvf ../allfiles-butsome.tar.gz * --exclude='/full/path/acme/foo'
$ tar --exclude=/full/path/acme/foo -czvf ../allfiles-butsome.tar.gz *
$ tar --exclude='/full/path/acme/foo' -czvf ../allfiles-butsome.tar.gz *

For Mac OSX I had to do

tar -zcv --exclude='folder' -f theOutputTarFile.tar folderToTar

Note the -f after the --exclude=


I agree the --exclude flag is the right approach.

$ tar --exclude='./folder_or_file' --exclude='file_pattern' --exclude='fileA'

A word of warning for a side effect that I did not find immediately obvious: The exclusion of 'fileA' in this example will search for 'fileA' RECURSIVELY!

Example:A directory with a single subdirectory containing a file of the same name (data.txt)

data.txt
config.txt
--+dirA
  |  data.txt
  |  config.docx
  • If using --exclude='data.txt' the archive will not contain EITHER data.txt file. This can cause unexpected results if archiving third party libraries, such as a node_modules directory.

  • To avoid this issue make sure to give the entire path, like --exclude='./dirA/data.txt'


To avoid possible 'xargs: Argument list too long' errors due to the use of find ... | xargs ... when processing tens of thousands of files, you can pipe the output of find directly to tar using find ... -print0 | tar --null ....

# archive a given directory, but exclude various files & directories 
# specified by their full file paths
find "$(pwd -P)" -type d \( -path '/path/to/dir1' -or -path '/path/to/dir2' \) -prune \
   -or -not \( -path '/path/to/file1' -or -path '/path/to/file2' \) -print0 | 
   gnutar --null --no-recursion -czf archive.tar.gz --files-from -
   #bsdtar --null -n -czf archive.tar.gz -T -

After reading this thread, I did a little testing on RHEL 5 and here are my results for tarring up the abc directory:

This will exclude the directories error and logs and all files under the directories:

tar cvpzf abc.tgz abc/ --exclude='abc/error' --exclude='abc/logs'

Adding a wildcard after the excluded directory will exclude the files but preserve the directories:

tar cvpzf abc.tgz abc/ --exclude='abc/error/*' --exclude='abc/logs/*'

Use the find command in conjunction with the tar append (-r) option. This way you can add files to an existing tar in a single step, instead of a two pass solution (create list of files, create tar).

find /dir/dir -prune ... -o etc etc.... -exec tar rvf ~/tarfile.tar {} \;

You can also use one of the "--exclude-tag" options depending on your needs:

  • --exclude-tag=FILE
  • --exclude-tag-all=FILE
  • --exclude-tag-under=FILE

The folder hosting the specified FILE will be excluded.


You can use cpio(1) to create tar files. cpio takes the files to archive on stdin, so if you've already figured out the find command you want to use to select the files the archive, pipe it into cpio to create the tar file:

find ... | cpio -o -H ustar | gzip -c > archive.tar.gz

gnu tar v 1.26 the --exclude needs to come after archive file and backup directory arguments, should have no leading or trailing slashes, and prefers no quotes (single or double). So relative to the PARENT directory to be backed up, it's:

tar cvfz /path_to/mytar.tgz ./dir_to_backup --exclude=some_path/to_exclude


After reading all this good answers for different versions and having solved the problem for myself, I think there are very small details that are very important, and rare to GNU/Linux general use, that aren't stressed enough and deserves more than comments.

So I'm not going to try to answer the question for every case, but instead, try to register where to look when things doesn't work.

IT IS VERY IMPORTANT TO NOTICE:

  1. THE ORDER OF THE OPTIONS MATTER: it is not the same put the --exclude before than after the file option and directories to backup. This is unexpected at least to me, because in my experience, in GNU/Linux commands, usually the order of the options doesn't matter.
  2. Different tar versions expects this options in different order: for instance, @Andrew's answer indicates that in GNU tar v 1.26 and 1.28 the excludes comes last, whereas in my case, with GNU tar 1.29, it's the other way.
  3. THE TRAILING SLASHES MATTER: at least in GNU tar 1.29, it shouldn't be any.

In my case, for GNU tar 1.29 on Debian stretch, the command that worked was

tar --exclude="/home/user/.config/chromium" --exclude="/home/user/.cache" -cf file.tar  /dir1/ /home/ /dir3/

The quotes didn't matter, it worked with or without them.

I hope this will be useful to someone.


Your best bet is to use find with tar, via xargs (to handle the large number of arguments). For example:

find / -print0 | xargs -0 tar cjf tarfile.tar.bz2

tar -cvzf destination_folder source_folder -X /home/folder/excludes.txt

-X indicates a file which contains a list of filenames which must be excluded from the backup. For Instance, you can specify *~ in this file to not include any filenames ending with ~ in the backup.


Possible redundant answer but since I found it useful, here it is:

While a FreeBSD root (i.e. using csh) I wanted to copy my whole root filesystem to /mnt but without /usr and (obviously) /mnt. This is what worked (I am at /):

tar --exclude ./usr --exclude ./mnt --create --file - . (cd /mnt && tar xvd -)

My whole point is that it was necessary (by putting the ./) to specify to tar that the excluded directories where part of the greater directory being copied.

My €0.02


I had no luck getting tar to exclude a 5 Gigabyte subdirectory a few levels deep. In the end, I just used the unix Zip command. It worked a lot easier for me.

So for this particular example from the original post
(tar --exclude='./folder' --exclude='./upload/folder2' -zcvf /backup/filename.tgz . )

The equivalent would be:

zip -r /backup/filename.zip . -x upload/folder/**\* upload/folder2/**\*

(NOTE: Here is the post I originally used that helped me https://superuser.com/questions/312301/unix-zip-directory-but-excluded-specific-subdirectories-and-everything-within-t)


Check it out

tar cvpzf zip_folder.tgz . --exclude=./public --exclude=./tmp --exclude=./log --exclude=fileName

The following bash script should do the trick. It uses the answer given here by Marcus Sundman.

#!/bin/bash

echo -n "Please enter the name of the tar file you wish to create with out extension "
read nam

echo -n "Please enter the path to the directories to tar "
read pathin

echo tar -czvf $nam.tar.gz
excludes=`find $pathin -iname "*.CC" -exec echo "--exclude \'{}\'" \;|xargs`
echo $pathin

echo tar -czvf $nam.tar.gz $excludes $pathin

This will print out the command you need and you can just copy and paste it back in. There is probably a more elegant way to provide it directly to the command line.

Just change *.CC for any other common extension, file name or regex you want to exclude and this should still work.

EDIT

Just to add a little explanation; find generates a list of files matching the chosen regex (in this case *.CC). This list is passed via xargs to the echo command. This prints --exclude 'one entry from the list'. The slashes () are escape characters for the ' marks.

참고URL : https://stackoverflow.com/questions/984204/shell-command-to-tar-directory-excluding-certain-files-folders

반응형