3

I am trying to use named pipes as a convenient way to preprocess input on the fly for pipelines that sadly do not accept data from stdin directly.

everywhere I look for info I get basically the same gist: Named pipes should be dead simple to use.

the gist is mostly that the following should work:

mkfifo mynamedpipe echo "is this working?" > mynamedpipe cat mynamedpipe

when i run mkfifo mynamedpipe, the pipe is successfully created and visible with ls *.

But even after i grant myself write permission to that pipe, when i try to run echo "whatever" > mynamedpipe nothing happens and the terminal just hangs until I kill the process with ctrl+c.

I have this problem on my local linux machine (Ubuntu 14.04.5 LTS) as well as on a public server (Red Hat Enterprise Linux 7), and in zsh as well as in bash.

What am I doing wrong here?

jov14
  • 55
  • 1
  • 2
  • 9
  • 1
    try `cat < mynamedpipe`. also note this quote from https://www.linuxjournal.com/article/2156 : "If you watch closely, you'll notice that the first command you run appears to hang. This happens because the other end of the pipe is not yet connected, and so the kernel suspends the first process until the second process opens the pipe. In Unix jargon, the process is said to be “blocked”, since it is waiting for something to happen." – Frank Thomas Sep 06 '18 at 16:57
  • Thanks that clears things up for me. But it is annoying that virtually every tutorial on this matter (or at least all of the most prominent google hits) describes exactly the basic steps I posted above, stating that they should work "as-is", without ever addressing that problem. Is this something that changed in recent times? – jov14 Sep 06 '18 at 17:28
  • You should use the reader (`cat mynamedpipe`) **first**. Which means you need two terminals. Also, this smells like an [XY question](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem). Please explain the actual problem you are trying to solve: What are those "pipelines that sadly do not accept data from stdin directly" exactly? – dirkt Sep 07 '18 at 06:10

1 Answers1

1

This post seems to relate to your problem : Cat to named pipe causes hang.

The relevant remarks are :

  • You need to have something reading from the FIFO
  • Ensure that the pipe is created with a large enough buffer or readers are fast enough to avoid blocking

  • You need to assign the pipe to a file descriptor, as in :

    exec 3<>/tmp/stream_pipe
harrymc
  • 455,459
  • 31
  • 526
  • 924
  • Thanks for the info. Sadly that seems to mean that named pipes are not an alternative for me, as that poster also states that trying to change the buffer size changed nothing for him. I was looking for an alternative to plug in with downstream tools that do not read from stdin, so the pipe is not necessarily being consumed as it is being written. Makes sense for pipes to behave that way though. So back to writing to and reading from intermediate physical temp-files for me... – jov14 Sep 06 '18 at 17:34
  • Named pipes are nice but unfortunately not a solution for potentially very large data. – harrymc Sep 06 '18 at 17:42