How to switch ddrescue from `/dev/sdq1` to `/dev/sdq` if I've already backed up data from `/dev/sdq1`?

Question

I started to recover my /dev/sdq1 when I was not familiar with HDD recovery. Well, I was careful enough and recover 85% in about 500 runs.

Afterwards, I've found out that I need /dev/sdq (without 1) contents actually to convert them after to .vhd with VBoxManage convertdd z:\fixed.vdi d:\disc.vhd --format VHD and after I can attach and read that .vhd in Windows, because VBoxManage does not support partition conversion and requires full harddrive raw data as an input. That means I should have started ddrescue with /dev/sdq and not /dev/sdq1.

I've found out that contents of /dev/sdq1 start at 32256 byte of /dev/sdq with fdisk. I dded these bytes at the beginning to separate file begin.raw and now running cat ./rescue/begin.raw ./rescue/fixed.vdi > ./sdq.raw to prepend them to recovered data from /dev/sdq1.

Is there an easier way to continue runs of ddrescue to that new ./sdq.raw with old rescuelog? I'm not sure that --output-position will fit here. (and how does it work exactly? -- This is not clear from docs how output-position will influence on rescuelog interpretation). I don't want to copy 500GB each time I want to get my ./sdq.raw.

Below is my script I use to recover data. Sometimes I had to detach HDD physically because my usb hub does not support complete power off on its ports.

#!/bin/bash
function run {
    ddrescue -T 5s --max-error-rate=10MB --skip-size=512KiB,10MiB \
        --direct -vv -P /dev/sdq1 ./fixed.vdi ./rescue.log
}

while true; do
    run &
    JOBPID=$!
    while true; do
        dmesg | tail -1 | grep 'reset high-speed USB device number' > /dev/null
        CODE=$?
        if [ $CODE -eq 0 ]; then
            break
        fi
        kill -0 "$JOBPID" > /dev/null
        CODE=$?
        if [ $CODE -ne 0 ]; then
            break
        fi
        sleep 3
    done
    ./uhubctl -i -a 2 -p 1 -l 1
    wait $JOBPID
    echo "MSG: waiting for re-attach"
    until dmesg | tail -1 | grep 'Attached SCSI disk'; do sleep 1; done
    sleep 5
done

Thanks!

P. S. I have Logitech USB 2.0 hub but it does not support full poweroff on ports either with uhubctl. Maybe there's other tool that can work with it?

Answer 1

Regardless of whether --output-position is the right ddrescue option or not, I present the general Linux way to concatenate two files without writing the result to disk.

The tool is dmsetup. It is in the dmsetup package in my Kubuntu, the same in Debian.

We have begin.raw of the size 32256 B and fixed.vdi corresponding to the partition of the size <X> (about 500 GB, the exact value unknown to me). We need to express the sizes in 512 B unit; the results are 63 and <Y> respectively. In the code below you have to replace <Y> with the computed value, so don't copy-paste it mindlessly.

# Creating loop devices and storing their names.
BEGIN_LOOP=$(sudo losetup -f --show begin.raw)
FIXED_LOOP=$(sudo losetup -f --show fixed.vdi)

# Creating a map to feed to `dmsetup` later.
# !!! REPLACE <Y> WITH THE PROPER VALUE !!!
echo "0 63 linear $BEGIN_LOOP 0" > whole.map
echo "63 <Y> linear $FIXED_LOOP 0" >> whole.map

# Creating a concatenated "file".
sudo dmsetup create whole.raw < whole.map
ln -s /dev/mapper/whole.raw

You can now use whole.raw as an image of the entire disk while ddrescue may still work with fixed.vdi. Every I/O operation on whole.raw will be mapped to begin.raw and fixed.vdi.

Note that this mapping is not persistent. After a reboot you need to repeat the whole procedure except ln -s.

To clean this setup without rebooting, invoke:

rm whole.raw                    # removes the symlink
sudo dmsetup remove whole.raw   # removes /dev/mapper/whole.raw
sudo losetup -d $FIXED_LOOP     # removes /dev/loop1 (or so)
sudo losetup -d $BEGIN_LOOP     # removes /dev/loop0 (or so)

If you lose the variables somehow (e.g. by closing the terminal emulator) then examine the output of losetup -a and do sudo losetup -d /dev/loopX accordingly.