Subject: Problem selecting options for cp command

From: Richard Owlett <[email-masked]>
Date: Mon, 5 Sep 2016 07:29:52 -0500

I attempted to copy contents of one partition to another using
    cp -R /media/richard/myrepo /media/richard/test

/media/richard/myrepo is a hard disk partition
/media/richard/test   is a USB flash drive

I received "illegal operation" error messages as symbolic links 
were encountered.
"man cp" was too terse to be illuminating.

https://www.gnu.org/software/coreutils/manual/html_node/cp-invocation.html#cp-invocation
[apparently relevant excerpts below] was not enlightening.

"...
By default, cp does not copy directories. However, the -R, -a, 
and -r options cause cp to copy recursively by descending into 
source directories and copying files to corresponding destination 
directories.

When copying from a symbolic link, cp normally follows the link 
only when not copying recursively or when --link (-l) is used. 
This default can be overridden with the --archive (-a), -d, 
--dereference (-L), --no-dereference (-P), and -H options. If 
more than one of these options is specified, the last one 
silently overrides the others.

When copying to a symbolic link, cp follows the link only when it 
refers to an existing regular file. However, when copying to a 
dangling symbolic link, cp refuses by default, and fails with a 
diagnostic, since the operation is inherently dangerous. This 
behavior is contrary to historical practice and to POSIX. Set 
POSIXLY_CORRECT to make cp attempt to create the target of a 
dangling destination symlink, in spite of the possible risk. 
Also, when an option like --backup or --link acts to rename or 
remove the destination before copying, cp renames or removes the 
symbolic link rather than the file it points to.
..."

"...
‘-d’    Copy symbolic links as symbolic links rather than copying 
the files that they
         point to, and preserve hard links between source files 
in the copies. Equivalent
         to --no-dereference --preserve=links.
..."

"...
‘-H’    If a command line argument specifies a symbolic link, 
then copy the file it
         points to rather than the symbolic link itself. However, 
copy (preserving its
         nature) any symbolic link that is encountered via 
recursive traversal.
..."

"...
‘-R’
‘-r’
‘--recursive’
     Copy directories recursively. By default, do not follow 
symbolic links in the source unless used together with the --link 
(-l) option; see the --archive (-a), -d, --dereference (-L), 
--no-dereference (-P), and -H options. Special files are copied 
by creating a destination file of the same type as the source; 
see the --copy-contents option. It is not portable to use -r to 
copy symbolic links or special files. On some non-GNU systems, -r 
implies the equivalent of -L and --copy-contents for historical 
reasons. Also, it is not portable to use -R to copy symbolic 
links unless you also specify -P, as POSIX allows implementations 
that dereference symbolic links by default.
..."

What option(s) should I be using?
Should I be looking at "dd" rather than "cp"?
"man dd" and 
https://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html#dd-invocation 
not illuminating.

TIA

---

From: "Thomas Schmitt" <[email-masked]>
Date: Mon, 05 Sep 2016 15:03:03 +0200

Hi,

Richard Owlett wrote:
> I received "illegal operation" error messages as symbolic links were
> encountered.

Maybe the filesystem is of a type which does not support symbolic links ?

  $ cp -R ./test/softlinks /mnt/fat
  cp: cannot create symbolic link ‘/mnt/fat/softlinks/irrelang’: Operation not permitted

Have a nice day :)

Thomas

---

From: Henrique de Moraes Holschuh <[email-masked]>
Date: Mon, 5 Sep 2016 09:55:46 -0300

> I received "illegal operation" error messages as symbolic links were

That means the target filesystem does not support symlinks.

-- 
  Henrique Holschuh

---

From: Richard Owlett <[email-masked]>
Date: Mon, 5 Sep 2016 08:25:37 -0500

> That means the target filesystem does not support symlinks.

That may have been the problem. It's possible the flash drive HAD 
BEEN formatted FAT-32. No way to tell now as I brute forced wiped 
drive by having gparted format it as ext2.

---

From: Jonathan Dowland <[email-masked]>
Date: Mon, 5 Sep 2016 14:37:42 +0100

On Mon, Sep 05, 2016 at 08:25:37AM -0500, Richard Owlett wrote:
> That may have been the problem. It's possible the flash drive HAD BEEN
> formatted FAT-32. No way to tell now as I brute forced wiped drive by having
> gparted format it as ext2.

If you are going to put an extX family filesystem on it, I'd recommend at
least ext3, and ideally ext4, which have journals and therefore are more
robust in the face of things like power cuts.

-- 
Jonathan Dowland
Please do not CC me, I am subscribed to the list.

---

From: Mark Fletcher <[email-masked]>
Date: Mon, 05 Sep 2016 12:46:43 +0000

> "man cp" was too terse to be illuminating.
I think cp -a is your best bet. It will copy everything, shouldn't choke on
links, and will also set permissions etc correctly on the target device,
assuming you have permissions to do so (you should be doing the copy as
root, always assuming you want to keep permissions and access modes intact).

I recently used it to copy hard disk partitions to a replacement SSD (some
discussions about that are in the archives of this list) and it worked just
dandy.

Mark

---

From: Richard Owlett <[email-masked]>
Date: Mon, 5 Sep 2016 09:21:04 -0500

On 9/5/2016 7:46 AM, Mark Fletcher wrote:
>
>
> On Mon, Sep 5, 2016 at 9:30 PM Richard Owlett
> <[email-masked] <mailto:[email-masked]>> wrote:
>
> I think cp -a is your best bet. It will copy everything,
> shouldn't choke on links, and will also set permissions etc
> correctly on the target device, assuming you have permissions to
> do so (you should be doing the copy as root, always assuming you
> want to keep permissions and access modes intact).
>
> I recently used it to copy hard disk partitions to a replacement
> SSD (some discussions about that are in the archives of this
> list) and it worked just dandy.
>
> Mark

"archive" had brought up mental image of dusty back rooms where 
things were being preserved for posterity. Not making a copy to 
actually be used.

Someone had suggested rsync but that has too many options for me 
to bungle. "cp" may be slower but appears more comfortable.

Thanks to all.

---

From: Lars Noodén <[email-masked]>
Date: Mon, 5 Sep 2016 17:28:16 +0300

On 09/05/2016 05:21 PM, Richard Owlett wrote:
> "archive" had brought up mental image of dusty back rooms where things
> were being preserved for posterity. Not making a copy to actually be used.
> 
> Someone had suggested rsync but that has too many options for me to
> bungle. "cp" may be slower but appears more comfortable.
> 
> Thanks to all.

For rsync, the only options that you need to worry about are -a and -v.
Even then the -v is optional.

	rsync -av /media/richard/myrepo/ /media/richard/test/

Just remember the trailing slash on the directories and it will go fine.

Regards,
Lars

---

From: Dan Purgert <[email-masked]>
Date: Mon, 5 Sep 2016 12:46:18 -0000 (UTC)

Richard Owlett wrote:
> [snip]
> [snip]

Using 'dd' may work, though it seems you need to simply add the -l
option to the 'cp' command you were using.

-- 
|_|O|_| Registered Linux user #585947
|_|_|O| Github: https://github.com/dpurgert
|O|O|O|

---

From: Patrick Wiseman <[email-masked]>
Date: Mon, 5 Sep 2016 08:43:44 -0400

> /media/richard/test   is a USB flash drive

I think what you want is the magic tar command:

$tar cf - [files] | (cd /[dir]; tar xf - )

It's wise, just to make sure it's going to do what you expect, to do

$tar cf - [files] | (cd /[dir]; tar tvf - )

first!

Substitute 'u' for 'x' to update files; otherwise, old files will
overwrite newer files with the same name.  (I know; I did it!)

To copy a whole directory; create the parent dir if it doesn't already
exist, then:

$tar cf - [dir] | (cd /[parent_dir]; tar xf - )

I haven't had occasion to use it in a while, but I'm pretty sure it makes a
"perfect" copy.

HTH

Patrick