48

I'm looking for a way to programmatically watch the output of a command until a particular string is observed and then exit. This is very similar to this question, but instead of tailing a file, I want to 'tail' a command.

Something like:

watch -n1 my_cmd | grep -m 1 "String Im Looking For"

(But this doesn't work for me.)

UPDATE: I need to clarify that my_cmd does not continuously output text but needs to be repeatedly called until the string is found (which is why I thought of the watch command). In this respect, my_cmd is like many other Unix commands such as: ps, ls, lsof, last, etc.

phuclv
  • 26,555
  • 15
  • 113
  • 235
gdw2
  • 1,295
  • 1
  • 11
  • 11
  • I would have thought it was possible to `tail -f` a program output just as well as a file... Am I wrong? – Joanis Jan 06 '12 at 00:52
  • @Joanis. You're right, but in my case 'my_cmd' doesn't continuously produce output and must be repeatedly called (much like most commands: ps, ls, lsof, etc) – gdw2 Jan 06 '12 at 04:01

6 Answers6

54

Use a loop:

until my_cmd | grep -m 1 "String Im Looking For"; do : ; done

Instead of :, you can use sleep 1 (or 0.2) to ease the CPU.

The loop runs until grep finds the string in the command's output. -m 1 means "one match is enough", i.e. grep stops searching after it finds the first match.

You can also use grep -q which also quits after finding the first match, but without printing the matching line.

choroba
  • 18,638
  • 4
  • 48
  • 52
  • 1
    someone else mentioned `grep -q` which is another option. grep quits after finding the string. – Sun Mar 22 '19 at 05:01
  • 1
    note that this command will repeatedly run the command in question, which might or might not be desirable. – adrien Jun 12 '19 at 10:03
  • 1
    @A__: It's desirable, as stated in the OP under "Update". – choroba Jun 12 '19 at 10:52
  • I recommend inserting a sleep so the CPU is not overly taxed. Example: `until my_cmd | grep -q foo; do ; sleep 1; done; echo 'Found'`. Depending on the task you might want a much longer sleep period. – Keith Bennett May 04 '20 at 18:04
  • 1
    @KeithBennett: That's what I suggested on the third line. – choroba May 04 '20 at 19:09
16
watch -e "! my_cmd | grep -m 1 \"String Im Looking For\""
  • ! negates the exit code of the command pipeline
  • grep -m 1 exits when string is found
  • watch -e returns if any error has occured

But this can be improved to actually display that matched line, which is thrown away so far.

math
  • 2,633
  • 8
  • 31
  • 43
  • Thanks for the detailed explanation, but it doesn't work for me. My `watch` command (CentOS) doesn't have the `-e` flag (which shouldn't really matter). More importantly, though, when the string is found, watch continues to run and does not exit. It seems that when `grep -m` exits, it only exits kills `my_cmd`, but not `watch`. – gdw2 Jan 06 '12 at 14:05
  • Nope, is does matter!, the "-e" flag is ment for leaving watch when the command has an error code different from 0. Since its not present watch is about to continue on your platform. Anyhow, good to know, on my Ubuntu 11.10 installation everything is fine. I have also sometimes troubles with Mac OSX regarding very very outdated commandline tools and I am using mac ports so far to get more current software. – math Jan 09 '12 at 10:00
  • This stops if the pattern is found, but it doesn't show any output until that happens – Mark Jan 03 '16 at 19:33
  • 1
    You may employ `tee` for that, but this introduces a misleading newline, I don't know how to circumvent right now: `watch -n1 -e "! date | tee /dev/tty | grep --color -m 1 \"17\""` – math Jan 04 '16 at 10:02
  • Yeah, this didn't work for me. `watch` dutifully stops watching when the string is found, but it doesn't actually exit until you press a key. So close. – mlissner May 01 '18 at 18:19
  • But using `-g` instead seems to do the trick! – mlissner May 01 '18 at 18:50
  • I've tried various variations of this and can't get it to work. No matter what I do, even though the string is found in `stdout`, the command doesn't exit. Example: `watch -g "! docker run --tty --volume $(pwd)/test/jekyll-plantuml:/srv/jekyll --env DEBUG=true swedbankpay/jekyll-plantuml:1.3.5 | tee /dev/tty | grep -q \"Server running\""` – Asbjørn Ulsberg May 22 '20 at 23:04
11

For those who have a program that is continuously writing to stdout, all you need to do is pipe it to grep with the 'single match' option. Once grep finds the matching string, it will exit, which closes stdout on the process that is being piped to grep. This event should naturally cause the program to gracefully exit so long as the process writes again.

What will happen is that the process will receive a SIGPIPE when it tries writing to closed stdout after grep has exited. Here is an example with ping, which would otherwise run indefinitely:

$ ping superuser.com | grep -m 1 "icmp_seq"

This command will match the first successful 'pong', and then exit the next time ping tries writing to stdout.

However,

It's not always guaranteed that the process will write to stdout again and therefore might not cause a SIGPIPE to be raised (e.g., this can happen when tailing a log file). The best solution i've managed to come up with for this scenario involves writing to a file; please comment if you think you can improve:

$ { tail -f log_file & echo $! > pid; } | { grep -m1 "find_me" && kill -9 $(cat pid) && rm pid; }

Breaking this down:

  1. tail -f log_file & echo $! > pid - tails a file, attaches process to background, and saves the PID ($!) to a file. I tried exporting the PID to a variable instead, but it seems there's a race condition between here and when the PID is used again.
  2. { ... ;} - group these commands together so we can pipe the output to grep while keeping the current context (helps when saving and reusing variables, but wasn't able to get that part working)
  3. | - pipe left side's stdout to right side's stdin
  4. grep -m1 "find_me" - find the target string
  5. && kill -9 $(cat pid) - force kill (SIGKILL) the tail process after grep exits once it finds the matching string
  6. && rm pid - remove the file we created
Blake Regalia
  • 211
  • 2
  • 4
  • Thanks! This worked for me: `{ docker run --tty --volume $(pwd)/test/jekyll-plantuml:/srv/jekyll --env DEBUG=true swedbankpay/jekyll-plantuml:1.3.5 & echo $! > pid; } | { grep -m1 "Server running" && kill -9 $(cat pid) && rm pid; }` – Asbjørn Ulsberg May 22 '20 at 23:06
  • Also, if you'd like to read the entire output from the first command and not just what's matched by `grep`, you can inject the following between `}` and `{`: `} | tee /dev/tty | {`. If `tty` is not available (on a build server, for instance), replace `/dev/tty` with `/dev/stderr`. – Asbjørn Ulsberg May 25 '20 at 08:28
1

To watch the output without modifying it, and with a timeout there can be used a "waitforoutput" micro Python application.

pip install waitforoutput
waitforoutput 'String Im Looking For' --command 'watch my_cmd' --timeout 10

On the official README there is more detailed explanation on the behavior: https://pypi.org/project/waitforoutput/

Krzysztofa Krzysztof
  • 161
  • 1
  • 7
  • This may solve the problem– but this is no 'micro python' application if it installs 28 other package dependancies. For those who want to undo isntalling this either `pip uninstall waitforoutput Pygments SecretStorage attrs bleach cffi colorama cryptography docutils jeepney jsonschema keyring packaging pbr pycparser pyparsing pyrsistent requests rfc3986 rkd rkd tabulate tqdm twine typing waitforoutput webencodings readme-renderer python-dotenv importlib-metadata` or install _another_ package `pip install pip-autoremove` and `pip-autoremove waitforoutput` to safely remove this. – Tim O'Brien Nov 09 '21 at 19:36
0
my_cmd | tail +1f | sed '/String Im Looking For/q'

If tail doesn't support the +1f syntax, try tail -f -n +1. (The -n +1 tells it to start at the beginning; tail -f by default starts with the last 10 lines of output.)

Keith Thompson
  • 5,075
  • 2
  • 25
  • 33
0

Append the result of your program calls to a file. Then tail -f that file. That way it should work... I hope.

When you restart calling that program you'll have to erase the file or append some gibberish to it just so it doesn't match again right away what you were looking for.

Joanis
  • 386
  • 1
  • 10