When are bs and count settings for dd wrong?

Question

(By wrong I mean "will break things".)

Suppose we're about to use dd. We've chosen a valid set of if, of, and possibly seek and skip. We've carefully ensured that the command will not write past our intended output region.

How can we now subtly break things by choosing a bad combination of bs and count? How would we know?

The reason I ask is that there seem to be magic preferred values that show up. For example, in this question about generating a random 1G file, the first two answers use if=/dev/urandom, of=sample.txt, bs=64M, and count=16.

How do I create a 1GB random file in Linux?

Of course these aren't the only valid settings but both answers used these settings, suggesting that this choice is particularly good and reasonable. Especially in a case like this where there's no filesystem or even physical disk, it's not clear to me if a choice of settings could be wrong -- not just inefficient, but wrong. My guess is that dd must write an integral number of blocks at a time so as bs increases so does memory usage and these values only affect performance.

The example is only an example and not something I'm specifically concerned with so go ahead and address cases where what's being copied has a filesystem.

Using dd is always a frightening leap of faith for me.

Answer 1

Help! dd is scary!

You are not alone. From tips for Linux:

The ' dd ' command is one of the original Unix utilities and should be in everyone's tool box... Some people believe dd means "Destroy Disk" or "Delete Data" because if it is misused, a partition or output file can be trashed very quickly. Since dd is the tool used to write disk headers, boot records, and similar system data areas, misuse of dd has probably trashed many hard disks and file systems.

But worry ye not. Once you learn of its intricacies you'll be invoking dd with the calm grace of a rodeo cowboy.

Will it break? What's this counting bs?

Let's deal with the first question first.

How can we now subtly break things by choosing a bad combination of bs and count? How would we know?

If you mean 'unintended consequences' by 'break', then just by specifying them, in theory. What do I mean by that? Well, lets say you want to copy image.img onto an SD card located at /dev/sdc. So you match the parameters in your example:

# dd if=/home/someone/image.img of=/dev/sdc bs=64M count=16

Should be fine to reuse those settings right? Let's just wait for dd to finish... now, what's it saying?

16+0 records in
16+0 records out
1073741824 bytes (1.1 GB) copied, 34.1303 s, 31.5 MB/s

Only 1.1 GB copied! But image.img is 2GB or so, I want the whole thing copied! By telling dd to copy 16 lots of 64M (where M=1024*1024, incidentally) you've specified a size. You would know by the status output. If you want to copy the whole file, either match the size with bs and count; or just omit those entirely:

# dd if=/home/someone/image.img of=/dev/sdc
4364864+0 records in
4364864+0 records out
2234810368 bytes (2.2 GB) copied, 45.9502 s, 48.6 MB/s

What happens if we get really naughty and tell dd to copy more data than is available?

# dd if=/home/someone/image.img of=/dev/sdc bs=1M count=4096
2131+1 records in
2131+1 records out
2234810368 bytes (2.2 GB) copied, 77.9768 s, 28.7 MB/s

Ah, dd only copies as many bytes as it receives. That's handy.

Monitoring dd

Speaking of status output, dd (GNU variant) respects and responds to the INFO signal SIGUSR1:

# echo "In another terminal or TTY"
# pkill -USR1 -n -x dd
# echo "Printed to stdout on the terminal/tty running dd:"
534+0 records in
534+0 records out
559939584 bytes (560 MB) copied, 1.68414 s, 332 MB/s

Handy for keeping an eye on a transfer which is slow or you expect to hang for some reason. Pairs nicely with watch, but be sure to give watch a reasonably long interval time.

Does dd have any other options I should know about?

dd does indeed have other options, but in most cases if you need to use them you will know what they do. Still, a few examples might give you an idea:

• conv=CONVS: convert the input as per one or more conversion options
• seek=N and skip=N: skip N [obs|ibs]-sized blocks at start of [output|input]
• status=X: (suppress output) either noxfer or none

I'm feeling much better about dd now!

Great! In no time you will be sending a bootable image across the internet to be written straight to a microSD card using a combination of dd and ssh.

But what does the dd acronym actually stand for?

A good last question. For that question and any others like it, I'll refer you to the jargon file entry for dd.

---

Feel encouraged to ask for clarification if anything is more opaque than it should be!

Answer 2

Your question is self-contradictory:

We've carefully ensured that the command will not write past our intended output region.

How can we now subtly break things by choosing a bad combination of bs and count? How would we know?

If you chose a bad combination of bs and count, the only thing you could break is writing past your intended output region (or not reaching it, which I consider the same issue)

If you have carefully ensured that the command will not write past your intended output region, you cannot break things because you've already ensured your combination of bs and count isn't bad. The two cannot occur simultaneously. Similarly, if you've not understood the bs and count parameters, you can't possibly have carefully ensured your output region is correct.

That said, the bs parameter is largely a performance tuning parameter. If you specify a bs smaller than the physical sector size and you are reading/writing in raw mode, or another completely out-of-whack value, dd will simply, and not in a subtle manner ('Abort: I/O error!'). Otherwise, things will just be slow.