2

I have three binary files (memory dumps). Call them file1, file2, file3. I'm trying to debug some software, and I'm toggling a switch.

  • file1 = switch off

  • file2 = switch on

  • file3 = switch off

I need to know which bytes changed between file1 and file2, which then also changed back to the same values (as file1) in file3.

There are a lot of unrelated changes between file1, file2 so that diff alone is not enough for me to determine what is changing when I toggle this switch, I'm trying to identify the unique bytes of entropy that change from file1,2,3,…

I know there are tools such as xxd, diff, vimdiff, colordiff. I'm just not sure how best to use them for this problem.

Scott - Слава Україні
  • 21,399
  • 46
  • 64
  • 121
authur1234
  • 23
  • 1
  • 3
  • 1
    "`diff` alone is not enough" – General note: tools that work with text (lines) cannot handle binary (i.e. non-text) input nicely. Even if they can in some cases, their output may not be what you expect. – Kamil Maciorowski Jun 25 '19 at 00:33
  • Possible duplicate of [How do I compare binary files in Linux?](https://superuser.com/q/125376/150988) – Scott - Слава Україні Jun 25 '19 at 00:33
  • 1
    Possible duplicate of [Three way binary comparison for savegame hacking](https://superuser.com/q/708461/150988).   Unfortunately, the only answer is a program that’s not free ($30 through $60, depending). – Scott - Слава Україні Jun 25 '19 at 00:33
  • Possible duplicate of [diff for multiple files](https://superuser.com/q/508608/150988). – Scott - Слава Україні Jun 25 '19 at 00:33
  • I don’t fully understand what you want. Can you give an example? It should be long enough to illustrate all the aspects that are important, but short enough to be comprehensible.  See [this](https://stackoverflow.com/help/minimal-reproducible-example) for some guidance. – Scott - Слава Україні Jun 25 '19 at 00:34
  • I can't think of a tool for this; the quickest way would be to write a real small program that does it (couple of dozen lines at most). – dirkt Jun 25 '19 at 07:28

3 Answers3

0

I'm a Windows user and I've used Beyond Compare for many years for file compares, including comparing 3 files (2 at a time) to each other.

It appears that Beyond Compare also has a Linux distro, so you might want to check it out.

https://www.scootersoftware.com/download.php?zz=kb_linux_install

I know the Windows version has options to show just differences, as well as a lot of other features that might help you out. I can't remember if they have a 3 way compare or not.

I'm not dev or sales for them, I just like the software a lot.

computercarguy
  • 837
  • 4
  • 15
0

To know bytes that changed between file1 and file2 and their respective values, use cmp -l:

cmp -l file1 file2 > changes12

Similarly for file3 and file2. The trick is to always investigate in the same direction (here: switch off to on), that's why I put file2 at the end:

cmp -l file3 file2 > changes32

Now you can find identical changes:

comm -12 changes12 changes32

and the output will be like (example):

     1629 152 112

which means the byte 1629 (decimal, numbering starts with 1) changed from 152 (octal) to 112 (octal).

Notes:

  • cmp -l in my Ubuntu "columnizes" its output. This means it may print lines with leading spaces, the width of the first column depends on the input size. I guess some implementations may not do this. There are few concerns:
    • If the original files are of different size (probably not your case), one cmp may generate wider first column than the other. In the context of later comm this is unacceptable. You can "decolumnize" the output by piping to awk '{print $1" "$2" "$3}'.
    • If the output is not "columnized" (or has been "decolumnized"), comm may complain the files are not sorted. You need sort (not sort -n) before you save to changesAB. This may generate somewhat unexpected order (e.g. differing byte 23 will appear after differing byte 100453) which may be fixed by piping the output of comm to sort -n.

  • changes* files may be huge. They are intermediate and temporary, so process substitution may be a good approach. This is non-POSIX though:

    # Korn shell syntax example
comm -12 <(cmp -l file1 file2) <(cmp -l file3 file2)

  • The output from comm can be used with yet another and another output of cmp to filter out unrelated changes better:

    comm -12 changes12 changes32 > result1
cmp -l file4 file5 | comm -12 - result1 > result2
cmp -l file6 file5 | comm -12 - result2 > result3

    But remember:

Documentation:

Kamil Maciorowski
  • 69,815
  • 22
  • 136
  • 202
0

Based on the response from Kamil, I used this to get what I needed:

cmp -l file1 file2 | awk '{print $1" "$2" "$3}' | sort > changes_12

cmp -l file3 file2 | awk '{print $1" "$2" "$3}' | sort > changes_32

comm -12 changes_12 changes_32 > common_changes

authur1234
  • 23
  • 1
  • 3