0

On Unix systems, files can be joined together using the cat utility (Windows systems have type or copy /B).

cat file1 file2 file3 > joined-file

This takes the content of the old files and copies it into a single new file. The process takes the time required to read the old files and write the new one. The new file is stored separately to the old files, so the command will fail if there is not enough free disk space left to write the new file.

Is there a quicker way to combine the files, or at least to "trick" programs into thinking that the files are combined, that does not involve moving or copying data on the disk?

One way to do this might be to modify the filesystem TOC so that the files are treated as fragments of a larger file. Is there a "safe" way to do this (i.e. safer than using dd to modify the TOC manually)?

I am interested in any method of joining the files, on any platform, that is quicker and requires less disk space than physically concatenating the files.

HullCityFan852
  • 1,191
  • 1
  • 10
  • 13
  • [This answer](https://superuser.com/a/1167639/432690). The restriction is that `dmsetup` tool is designed to work with block devices and/or regular files that are images of block devices, so it uses common blocks of 512 B. Maybe one could fork the project and modify the source to allow 1 B resolution, I don't know. – Kamil Maciorowski Apr 15 '18 at 06:37
  • Also [this answer](https://unix.stackexchange.com/a/94061/108618), it's from few years ago; apparently there were some changes to `nbd`, I couldn't make it work so easily. Investigate [concatfs](https://github.com/schlaile/concatfs) and/or [concat-fuse](https://github.com/concat-fuse/concat-fuse) (this one is already in davidgo's answer below). I haven't tested them so this is just a comment. – Kamil Maciorowski Apr 15 '18 at 07:08

1 Answers1

0

I postulated that this might be possible using some kind of overlay filesystem on top of an existing one (modifying an existing one is a recipe for disaster), then worked out you could probably write something for Linux using FUSE (filesystem in Userspace)

On a lark I googled and discover that someone has already implemented this - there is a GIT project called concat-fuse - https://github.com/concat-fuse/concat-fuse/blob/master/README.md - which looks to do what you are describing.

davidgo
  • 68,623
  • 13
  • 106
  • 163