Why does xxd -r change the first byte of my binary file

Question

I'm confused by the behavior of xxd, here's what I'm doing.

I have a xxd dump :

$ head dump
00000000: 7f45 4c46 0201 0103 0000 0000 0000 0000  .ELF............
00000010: 0200 3e00 0100 0000 107f 4400 0000 0000  ..>.......D.....
00000020: 4000 0000 0000 0000 0000 0000 0000 0000  @...............
00000030: 0000 0000 4000 3800 0200 4000 0000 0000  [email protected]...@.....
00000040: 0100 0000 0500 0000 0000 0000 0000 0000  ................
00000050: 0000 4000 0000 0000 0000 4000 0000 0000  ..@.......@.....
00000060: 2487 0400 0000 0000 2487 0400 0000 0000  $.......$.......
00000070: 0000 2000 0000 0000 0100 0000 0600 0000  .. .............
00000080: 2854 0b00 0000 0000 2854 6b00 0000 0000  (T......(Tk.....
00000090: 2854 6b00 0000 0000 0000 0000 0000 0000  (Tk.............

But when I xxd -r it, the first byte is changed to another value:

$ xxd -r dump output
$ xxd output | head
00000000: ce45 4c46 0201 0103 0000 0000 0000 0000  .ELF............
00000010: 0200 3e00 0100 0000 107f 4400 0000 0000  ..>.......D.....
00000020: 4000 0000 0000 0000 0000 0000 0000 0000  @...............
00000030: 0000 0000 4000 3800 0200 4000 0000 0000  [email protected]...@.....
00000040: 0100 0000 0500 0000 0000 0000 0000 0000  ................
00000050: 0000 4000 0000 0000 0000 4000 0000 0000  ..@.......@.....
00000060: 2487 0400 0000 0000 2487 0400 0000 0000  $.......$.......
00000070: 0000 2000 0000 0000 0100 0000 0600 0000  .. .............
00000080: 2854 0b00 0000 0000 2854 6b00 0000 0000  (T......(Tk.....
00000090: 2854 6b00 0000 0000 0000 0000 0000 0000  (Tk.............

As you can see everything stays the same but the first byte is ce instead of 7f.

What you're describing shouldn't be possible. Are you sure that you are reading the same dump? I noticed in your first code block, you're reading from `dump`, but in the second one, you're reading from `input`. — Deltik, Jun 07 '19 at 18:45
Yes I'm positive that it's the same dump, sorry about the confusion in the names, I edited the filenames to make it less distracting but I messed up the names. I'll edit it. — aflamant, Jun 07 '19 at 19:01
Just tried it with a cut-and-paste of your `dump` on my system, with the exact sequence of commands you are using, and for me `7e` stays `7e`. — dirkt, Jun 08 '19 at 18:13

bmaupin · Answer 1 · 2022-08-15T16:16:08.840

Do you happen to have any lines in your dump file that don't contain data (e.g. lines containing only comments)?

For every line of input, xxd will try to read hex values for an address and data. It doesn't distinguish between lines that are properly formatted. Here's a contrived example:

$ cat dump 
00000000: 7f45 4c46 0201 0103 0000 0000 0000 0000  .ELF............
00000010: 0200 3e00 0100 0000 107f 4400 0000 0000  ..>.......D.....
00000020: 4000 0000 0000 0000 0000 0000 0000 0000  @...............
00000030: 0000 0000 4000 3800 0200 4000 0000 0000  [email protected]...@.....
00000040: 0100 0000 0500 0000 0000 0000 0000 0000  ................
00000050: 0000 4000 0000 0000 0000 4000 0000 0000  ..@.......@.....
00000060: 2487 0400 0000 0000 2487 0400 0000 0000  $.......$.......
00000070: 0000 2000 0000 0000 0100 0000 0600 0000  .. .............
00000080: 2854 0b00 0000 0000 2854 6b00 0000 0000  (T......(Tk.....
00000090: 2854 6b00 0000 0000 0000 0000 0000 0000  (Tk.............
# 0 ocelot something or other

$ xxd -r dump output

$ xxd output 
00000000: ce45 4c46 0201 0103 0000 0000 0000 0000  .ELF............
00000010: 0200 3e00 0100 0000 107f 4400 0000 0000  ..>.......D.....
00000020: 4000 0000 0000 0000 0000 0000 0000 0000  @...............
00000030: 0000 0000 4000 3800 0200 4000 0000 0000  [email protected]...@.....
00000040: 0100 0000 0500 0000 0000 0000 0000 0000  ................
00000050: 0000 4000 0000 0000 0000 4000 0000 0000  ..@.......@.....
00000060: 2487 0400 0000 0000 2487 0400 0000 0000  $.......$.......
00000070: 0000 2000 0000 0000 0100 0000 0600 0000  .. .............
00000080: 2854 0b00 0000 0000 2854 6b00 0000 0000  (T......(Tk.....
00000090: 2854 6b00 0000 0000 0000 0000 0000 0000  (Tk.............

In that example, it interpreted 0 as the address and ce from ocelot as the data. From the man page:

CAVEATS
       xxd -r has some builtin magic while evaluating line number information.  If the output file is
       seekable,  then  the  linenumbers at the start of each hexdump line may be out of order, lines
       may be missing, or overlapping. In these cases xxd will lseek(2) to the next position. If  the
       output file is not seekable, only gaps are allowed, which will be filled by null-bytes.

       xxd -r never generates parse errors. Garbage is silently skipped.

       When  editing hexdumps, please note that xxd -r skips everything on the input line after read‐
       ing enough columns of hexadecimal data (see option -c). This also means, that changes  to  the
       printable  ascii  (or  ebcdic)  columns  are always ignored. Reverting a plain (or postscript)
       style hexdump with xxd -r -p does not depend on the correct number of columns.  Here  anything
       that looks like a pair of hex-digits is interpreted.

The solution in that case is to remove any non-data lines. If you do have comments, they can be added after the data since xxd will ignore them, e.g.

00000000: 7f45 4c46 0201 0103 0000 0000 0000 0000  .ELF............ # 0 ocelot something or other

Why does xxd -r change the first byte of my binary file

1 Answers1