Copy files in Linux, avoid the copy if files do exist in destination

Question

I need to copy a /home/user folder from one hard disk to another one. It has 100,000 files and around 10G size.

I use

cp -r /origin /destination

sometimes I get some errors due to broken links, permissions and so on. So I fix the error, and need to start again the copy.

I wonder how could I tell the command "cp", once it tries to copy again, not to copy files again if they exist in the destination folder.

Don't us cp. Use rsync instead. – William Pursell Mar 11 '10 at 16:08 — William Pursell, Mar 11 '10 at 16:08

score 196 · Answer 1 · edited Jun 12 '20 at 13:48

196

Just use cp -n <source> <dest>.

From man page:

-n, --no-clobber

do NOT overwrite an existing file (overrides a previous -i option)

edited Jun 12 '20 at 13:48

Community

1

answered Jan 25 '17 at 13:45

Balmipour

2,149
2
10
8

20

This is the only true answer to the question. – sebix Mar 21 '17 at 18:19
1

@sebix OP says "sometimes I get some errors [...] and need to start again the copy" Unless I'm mistaken, this solution wouldn't overwrite corrupt files at the destination by re-copying, so you could end up with a bunch of corrupt files at the destination, while thinking that the copy process has been successful. – joe Jul 28 '20 at 17:37
1

@joe interesting remark, but the op explicitely asks for a way to skip copying files which already exist in the destination. I guess **rsync** would be a better choice, if one wants to validate the files integrity. (*note that re-copying everything wouldn't help much, and could as well replace some valid files with corrupted ones.*) – Balmipour Oct 14 '20 at 14:20
@Balmipour Yep, `rsync` is a better approach here. "but the op explicitly asks [...]" It's quite clear from OP's post that they wouldn't want corrupt/half-copied files at the destination. – joe Oct 15 '20 at 06:04

score 57 · Accepted Answer · edited Nov 23 '18 at 00:39

57

cp -R -u -p /source /destination

The -u (or --update) flag does just this:

From the man page for cp:

-u, --update

copy only when the SOURCE file is newer than the destination file or when the destination file is missing

edited Nov 23 '18 at 00:39

ifconfig

622
3
10
23

answered Mar 11 '10 at 16:02

user31894

2,789
18
9

13

To literally only copy files that don't exist and not update existing ones, `yes n | cp -i /source/* /destination 2>/dev/null` – sventechie Nov 21 '13 at 20:14
5

-u, --update copy only when the SOURCE file is newer than the destination file or when the destination file is missing -u, --update copy only when the SOURCE file is newer than the destination file or when the destination file is missing -p same as --preserve=mode,ownership,timestamps --preserve[=ATTR_LIST] preserve the specified attributes (default: mode,ownership,time‐ stamps), if possible additional attributes: context, links, xattr, all – Covich Feb 16 '16 at 15:29
2

There is no `-u` option on my Mac. I think `-n` option can be used which means "Do not overwrite an existing file" according to the man page. I see that is in another answer. Troubling that different systems don't agree on something as fundamental as `cp`. – wcochran Feb 09 '21 at 17:53

Martin Beckett · Answer 3 · 2016-04-24T21:17:08.233

30

rsync -aq /src /dest

Apart from only copying newer files, it will even only copy the newer parts of files if the file has changed. It's intended for copying over network links where you want to minimise the amount of data - but it also works great locally.

edited Apr 24 '16 at 21:17

answered Mar 11 '10 at 16:12

Martin Beckett

9,753
1
27
28

Kamil Maciorowski · Answer 4 · 2023-06-24T16:38:20.363

6

POSIX solution

Other answers use -u or -n options of cp. Neither of these is required by POSIX; nor is rsync from yet another answer; nor is yes used in one of the comments.

Still, we can reproduce yes n with a while loop. This leads to the following POSIX solution:

while true; do echo n; done | cp -Ri /origin /destination 2>/dev/null

edited Jun 24 '23 at 16:38

answered Nov 23 '18 at 06:33

Kamil Maciorowski

69,815
22
136
202

score 5 · Answer 5 · answered Mar 11 '10 at 16:01

5

Look up the "-u" option for the cp command.

answered Mar 11 '10 at 16:01

Pointy

881
1
8
20

3

Disagree that using -u is a good idea. -u copies only when source is newer or destination is missing. Original poster's issue was caused by file copy breaking for whatever reason. OP would therefore have half-written file that wasn't updated when re-running the copy. That file might be important to someone! **rsync solves this problem.** – hazymat Dec 21 '13 at 21:47

score 5 · Answer 6 · answered Feb 21 '19 at 08:39

5

Use cp -rn <sourcedirname>/. <destdirname>

The r switch makes the copy recursive over the directories.

The n switch (long version no-clobber) ensures existing files are never over-written.

The '/.' after the sourcedirname ensures that it does not become a subdirectory under the destdirname instead of all contents of the former being copied to the latter.

answered Feb 21 '19 at 08:39

taatparya

61
1
2

Thank you, your solution works for me. I used rsync and rsync ate my full cpu, my computer was too hot in high temperature, so I prefer using cp. – Bảo Nam Jan 21 '20 at 03:08

score 2 · Answer 7 · answered Jul 08 '18 at 04:58

2

All above answers are correct but if you are doing this recursively then

you should do:

 cp -rn SOURCE_PATH DESTINATION_PATH

answered Jul 08 '18 at 04:58

grepit

213
1
6

score 1 · Answer 8 · answered Mar 11 '10 at 16:09

1

You should be copying as root to maintain permissions/ownership

# cp -au

Also look at rsync

answered Mar 11 '10 at 16:09

pixelbeat

1,220
10
6

score 0 · Answer 9 · edited Jun 24 '23 at 15:11

0

If your source location is a remote file system, and you don't have shell access to it (and thus can't use rsync), then this might help:

cd /local/destination/path
echo "reget /remote/source/path" | sftp -r sftp://username@domainOrIpAddress

From this answer.

Note that reget is the same as get -a. It checks for the presence of half-copied files and finishes copying them.

edited Jun 24 '23 at 15:11

AsukaMinato

163
1
4

answered Jul 28 '20 at 18:06

joe

101
2

Copy files in Linux, avoid the copy if files do exist in destination

9 Answers9

POSIX solution

Linked