DD Command

No, DD does not means Diamond Dog; it is a UNIX command for data transfer. DD command is one the powerful and deadly tool that gives Linux users the power of time control and data travel through spaces. It's so powerful that it can clone a physical device from one to another without needing any tools; yet it is so deadly that an overconfident Linux user can destroy his/her own operating system over a small mistake.

Basic DD Command

DD can be used in various deployment. Here, we learn the basic DD command follows this pattern:

$ dd if=<input device/file/data> of=<output device/file/data> bs=<block size per read> count=<by rounds>

So, a typical example for imaging a hard disk, sat the device is /dev/sdb, to a file ./disk.img, with block size of 4096, count of 500, that would be:

$ dd if=/dev/sdb of="./disk.img" bs=4096 count=500

Show Progress

Modern DD now has the status parameter to indicate progress. Although the presentation is verbose, it is kind of useless if you're seeking for wait time insight. To enable progress, simply provide status=progress argument.

$ dd status=progress if=<input device/file/data> of=<output device/file/data> bs=<block size per read> count=<by rounds>

So, a typical example for progress bar would be:

$ dd status=progress if=/dev/urandom of="./sample.img" bs=8255280 count=5120

37853708800 bytes (37 GB, 36 GiB) copied, 334.01 s, 120 MB/s

Yeah, it's pretty much useless if you're expecting how long more do you need to wait. However, it does reports in its progress though, with rate of transfer as well.

Finding Block Size and Block Count

If you notice, how did I get those block size and count. If you specify the total bytes more than the actual one, you're wasting storage and time; if otherwise, you have data loss. Fortunately, There are a few good practices to source these information.

Step #1 - Learn Your Disk Capability for Block Size

You need to learn your disk write capability. To do that, we use 2 commands and one of them requires root access.

First, use lsblk to determine the drive you plan to analyze with., Here's an example:

$ lsblk

NAME                  MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT

loop0                   7:0    0     8K  1 loop  /snap/bashell/11

loop1                   7:1    0  89.5M  0 loop  /snap/core/6130

sda                     8:0    0 931.5G  0 disk  

└─sda1                  8:1    0 931.5G  0 part  

  └─sdb1_crypt        254:0    0 931.5G  0 crypt 

    ├─localstore-swap 254:1    0  11.4G  0 lvm   [SWAP]

    ├─localstore-home 254:2    0 465.7G  0 lvm   /home

    └─localstore-root 254:3    0 454.5G  0 lvm   /

sdb                     8:16   0 465.8G  0 disk  

├─sdb1                  8:17   0   100M  0 part  

└─sdb2                  8:18   0  39.9G  0 part  

sr0                    11:0    1  1024M  0 rom  

From the example, we know that /dev/sda is our main disk, with localstore-root is our logical volume that we intend to write with , /dev/sdb is our extended storage device. In this case, our objective is to find out localstore-root maximum write limit.

Next, use fdisk -l to find out the disk data with that intended name. Since our example uses LUKS with LVM, the actual partition is mapped to /dev/mapper/localstore-root. Otherwise, you can just seek the hard disk write speed using /dev/sda.

$ sudo fdisk -l -u=cylinders /dev/mapper/localstore-root 

Disk /dev/mapper/localstore-root: 454.5 GiB, 488003076096 bytes, 953131008 sectors

Units: cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

$ sudo fdisk -l -u=cylinders /dev/sda

Disk /dev/sda: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors

Geometry: 255 heads, 63 sectors/track, 121601 cylinders

Units: cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disklabel type: gpt

Disk identifier: <DISK UUID>

Device     Start    End   Size Type

/dev/sda1      1 121602 931.5G Linux filesystem

$

Now take note of the Units. Both indicate the maximum amount of block bytes that hard disk can take. You should always choose the lowest available value as your maximum limit. Then, you can choose any size below that value to avoid over-read condition. Hence, we use the following equation:

1024 ⩽ block_size ⩽ lowest_disk_or_partition_units

TIP:

• We usually use 1 iB (1024) or 4 iB (4096) bytes at the minimum side. Anything below is a test of patience.

Step #2 - Learn Your Block Count

For count, it is basically the number of data you want to dd with. It follows the following equations:

total_bytes = block_size * block_count 

block_count = total_bytes / block_size

For normal data transfer, you can roughly determine the size using the above equation.

For disk measurement however, your fdisk report against the disk tells you the information. Let's look back the report above again:

$ sudo fdisk -l -u=cylinders /dev/sda

Disk /dev/sda: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors

Geometry: 255 heads, 63 sectors/track, 121601 cylinders

Units: cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disklabel type: gpt

Disk identifier: <DISK UUID>

Device     Start    End   Size Type

/dev/sda1      1 121602 931.5G Linux filesystem

$

Here, we know that the disk has 1 partition, /dev/sda1, starting at 1, ends at 121602 units. The total storage size follows this equation:

total_size = (end_unit - (start_unit - 1) ) * cylinder size

Following the equation above, you get:

total_size = (121602 - (1 - 1) ) * (16065 * 512)

           = 121602 * 16065 * 512

           = 1000210498560

           = 1000210 MB

           = 1000.21 GB

           = 1.21 TB

Hence, we can safely determine that the last partition's "END" unit is your count, which is 121602 in the example. To capture the last block, we increment it by 1, yielding 121603 as the correct block_count. Here's the count equation:

block_count = last_block_end_units + 1

Common Usages

DD is commonly used in the various file and disk deployments.

Backing Up Disk

This is also known as disk cloning. You can clone a disk using DD command and restore it back. Keep in mind that you need a disk larger than your subject in order to store its image (2TB to hold 1TB image, 1TB to hold 500GB image etc.). Here's an example based on the example above,

To backup the image:

$ sudo dd status=progress if=/dev/sda of=./backup.img bs=8225280 count=121603

To restore the image (interchange the input and output and does not need to specify block size and count):

$ sudo dd status=progress if=./backup.img of=./dev/sdb

NOTE:

• Always use lsblk to determine the correct storage device.
• Always TRIPLE check your DD commands before you press enter. DD has the ability to overwrite your existing operating system disk if you falsely provided to it as output.
  • DO NOT assume /dev/sdb is your external storage. Kernel always dynamically determines hardware so use lsblk to determine it at time.

Create Crypto Random File / Disk

In situation where we want to create a crypto random file or disk to prevent cryptanalysis attack, we commonly use DD for it. You usually see folks use DD for disk mainly because for file, many people will use pipe redirect. Some examples are:

$ dd status=progress if=/dev/urandom of=/dev/sda

$ dd status=progress if=/dev/urandom of=./random.file bs=1M count=10

NOTE:

• This is not safe for solid-state disk since we are writing bit by bit into the storage itself. It will definitely shorten the lifetime of your SSD life expectancy.

Wipe Disk / File

In situation where we want completely delete the file, we can also use DD. Some examples are:

$ dd status=progress if=/dev/urandom of=/dev/sda

$ dd status=progress if=/dev/urandom of=./random.file bs=1M count=10

$ dd status=progress if=/dev/zero of=/dev/sda

$ dd status=progress if=/dev/zero of=./random.file bs=1M count=10

NOTE:

• This is not safe for solid-state disk since we are writing bit by bit into the storage itself. It will definitely shorten the lifetime of your SSD life expectancy.
• If you want to wipe the target with random data, use /dev/urandom (crypto safe) or just /dev/random.
• If you want to wipe the target with all zero, use /dev/zero instead.

That's all about disk or file imaging with DD.