Bash file redirection bug?

Question

From what I understand about program IO, there are stdin, stdout, and stderr data streams (and a return code). stdout and stderr are the two data output streams. So if I use bash redirection to close one of the output streams, I can narrow down which stream that text is being sent to. Right?

I'm running Ubuntu 18.04.1 LTS and I'm running into this bizarre issue with bash redirection.

Let me explain the example. Here is my command:

# apt-cache show php5
N: Can't select versions from package 'php5' as it is purely virtual
N: No packages found
# |

The php5 package does not exist on Ubuntu 18.04 so apt-cache displays an error. I would assume this text is sent to the stderr stream, so I attempted to close that stream:

# apt-cache show php5 2>&-
# |

It appears this verifies that the text was sent through stderr. All good so far! But now as a sanity check, I tried closing stdout (I should see the error text now):

# apt-cache show php5 1>&-
# |

What!? I redirected stdout this time but stderr is also not showing up?

According to the internet, I have my file descriptors correct: https://www.gnu.org/software/bash/manual/html_node/Redirections.html

I can't figure out what could be going on here.

Screenshot proof:

Answer 1

TL;DR: It's not bash, it's apt-cache that's messing around with file descriptors.

apt-cache is doing something very interesting - it tends to not write out lines starting with N: characters intended for stdout.

Consider this:

$ apt-cache show nonexistent
N: Unable to locate package nonexistent
E: No packages found

We see two lines, one starting with N: one starting with E:. N: lines go to stdout. In your example, you have two N: lines.

# apt-cache show php5
N: Can't select versions from package 'php5' as it is purely virtual
N: No packages found

If you trace the system calls via strace -e write -f bash -c 'apt-cache show randomtext >&-' you'll see that writing E: lines happens, but N lines aren't there:

[pid 12450] write(2, "E", 1E)            = 1
[pid 12450] write(2, ": ", 2: )           = 2
[pid 12450] write(2, "No packages found", 17No packages found) = 17
[pid 12450] write(2, "\n", 1
)           = 1
[pid 12450] +++ exited with 100 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=12450, si_uid=1000, si_status=100, si_utime=5, si_stime=2} ---
+++ exited with 100 +++

So apt-cache is smart enough to check for redirected stdout. But what about stderr ? Apparently writes are still there: if you do strace -e write,openat,dup2 -f bash -c 'apt-cache show randomtext 2>&- you'll see that apt-cache opens /dev/null to still have something present for stderr:

[pid 12543] openat(AT_FDCWD, "/dev/null", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 2
....
[pid 12543] write(2, "N", 1)            = 1
[pid 12543] write(2, ": ", 2)           = 2
[pid 12543] write(2, "Unable to locate package randomt"..., 35) = 35
[pid 12543] write(2, "\n", 1)           = 1
[pid 12543] write(2, "E", 1)            = 1
[pid 12543] write(2, ": ", 2)           = 2
[pid 12543] write(2, "No packages found", 17) = 17
[pid 12543] write(2, "\n", 1)           = 1
[pid 12543] +++ exited with 100 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=12543, si_uid=1000, si_status=100, si_utime=5, si_stime=3} ---
+++ exited with 100 +++

---

If you perform the same with other programs in bash, it works as expected:

# stdout closed, stderr not
$ ls -l /proc/self/fd >&-
ls: write error: Bad file descriptor
# stdout open , stderr closed, and it's number is assigned to whatever command is trying to open - in this case /proc/self/fd directory
$ ls -l /proc/self/fd 2>&-
total 0
lrwx------ 1 xie xie 64 Oct  6 11:32 0 -> /dev/pts/1
lrwx------ 1 xie xie 64 Oct  6 11:32 1 -> /dev/pts/1
lr-x------ 1 xie xie 64 Oct  6 11:32 2 -> /proc/12723/fd