ffmpeg waits to close the pipe in order to start processing data

Question

I'm struggling with the following issue, on a Windows system:

• I get a live stream from an SDK; I get chunks of bytes via a callback; the container is MPEG-PS
• I have to display this live stream to the end-user using VLC. But VLC cannot play this stream I receive
• My solution was to use ffmpeg to convert this to MPEG-TS (or something else) that can be played by VLC

I tried different possible solutions, like making a TCP server which exposes this stream, instructing ffmpeg to read the input from there. But ffmpeg is complaining that it cannot detect the container type. If I save to disk the bytes that I receive via that callback, I can see that the first bytes are ‘IMKH’, indicating a MPEG-PS container. If I then use ffmpeg to transcode it to MP4 (as a file), that works. But I need to do this ‘dynamically’, so a live transcoding of the stream that I receive.

Last thing that I have tried was to create a named pipe and write the received bytes in it, then used the following command to create the live stream: ffmpeg -i \.\pipe\testpipe -f mpegts -c copy -listen 1 "http://127.0.0.1:5002” and then play the URL “http://127.0.0.1:5002” in VLC.

Something strange is happening, because only when I close the pipe, the ffmpeg start to output (in console) the found streams and VLC can connect to its http server and starts playing correctly. I flush the pipe after writing each chunk of bytes.

I would expect ffmpeg to start processing the pipe input as it is received, and immediately output the result for VLC. I found something similar in the following post, but without an answer: https://ffmpeg.org/pipermail/ffmpeg-user/2016-December/034639.html

Why is ffmpeg waiting to close the pipe to start processing? Can it be configured to start a live transcoding of the received stream?

EDIT - MORE DETAILS ====================================

I cannot make available the source stream (it is isolated in an intranet), but as I mentioned I use a 3rd party C# SDK to access it (actually this is the only way to access it). That SDK provides me access to that stream via a simple callback which looks like this: callback(byte[] data). So using this callback I receive all the stream bytes. I saved these bytes to a file on disk. The received stream is MPEG-PS. If I try to convert this file to another file format (like avi) that works: ffmpeg.exe -i input.mpeg out.avi and VLC can play this out.avi. So I know that the callback data that I get is correct, so input.mpeg is a correct file, ffmpeg can deal with it.

For my testing I also use this input.mpeg file, I created a simple C# server which reads it, and exposes it in chunks via a named pipe using this code:

_pipeServer = new NamedPipeServerStream("testpipe", PipeDirection.Out);
_pipeServer.WaitForConnection(); // wait for ffmpeg to connect

byte[] allBytes = System.IO.File.ReadAllBytes(“input.mpeg”);

using (BinaryWriter bw = new BinaryWriter(_pipeServer))
{                    
   int index = 0;                       
    for(;;)
    {
      bw.Write(allBytes, idx, 100);
      index += 100;

      // added some delay to emulate a live stream
      System.Threading.Thread.Sleep(100);

     if (idx >= allBytes.Length) break;
    }
}

This is the exact description of the flow:

• I start the C# pipe server as above
• I start ffmpeg with the command line: ffmpeg -i \.\pipe\testpipe -f mpegts –c copy -listen 1 http://127.0.0.1:5002
• Then I start VLC with URL http://127.0.0.1:5002.

But nothing happens; only when the C# server completes sending (so the BinaryWriter object is disposed, so ffmpeg detects the end of input stream) VLC starts playing. It seems that ffmpeg accumulates the received stream and only provides the output when the input completes.

I made also the following test:

• I converted input.mpeg which has MPEG_PS format to MPEG-TS format using: ffmpeg.exe -i input.mpeg input.ts
• then I feed input.ts in the named pipe “testpipe” and start ffmpeg with the same args: ffmpeg -i \.\pipe\testpipe -f mpegts –c copy -listen 1 http://127.0.0.1:5002
• I start VLC with http://127.0.0.1:5002 and it starts immediately.

So I think this proves is not a pipe issue, but a format/container issue. When the pipe is fed with MPEG-PS, ffmpeg waits for EOF; but when is fed with MPEG-TS, ffmpeg starts producing output immediately (and this is what I want to achieve, live streaming).

BTW: if I kill the C# server above when it gets to the middle of the input.mpeg, the effect is the same, VLC starts playing the available stream. So I don’t think ffmpeg really needs the entire stream to be downloaded in order to start forwarding it to output.

Do you know how can I convince ffmpeg to start producing output immediately? Probably is some parameter for ffmpeg, but I could not find it … I have placed the input.mpeg file here:

https://www.dropbox.com/s/zsdktikfq4yuak0/input.mpeg?dl=0

Thank you!

Answer 1

I had the same issue in a slightly different context, and after much hair-pulling and deep-diving in the FFMPEG code, I finally found the reason for this hanging of the input pipe: FFMPEG tries to find as much information about the input stream as possible at the start, and to do this it reads and decodes the input until it is satisfied it has enough information.

Depending on your codec and format, this can require more or less data and in some cases (mine at least), 4 seconds worth of video wasn't enough (why is another subject).

The default maximum probe size is 500000 bytes, but it can be modified with the -probesize option (see https://ffmpeg.org/ffmpeg-formats.html), e.g.:

fmpeg -probesize 8192 -i input ...

Setting this value to the minimum data size you are sure to have available at the start of the stream will get you rolling (but you may have less information about the stream than you'd like).

Answer 2

You need to use a format that works on-the-fly. Like ogv, webm or better hls. Try for example: $ ffmpeg -i video.mp4 -f hls ./m3u8/video.m3u8 And, ffplay ./m3u8/video.m3u8 it's ok from the first time. I recommend the series of posts : https://www.martin-riedl.de/ffmpeg/

Answer 3

I can't make it work with VLC, but with FFplay or mpv, it's working without the need to convert the format.

We may write the data directly to stdin pipe of FFplay (or mpv) sub-process.

Here is a C# code sample:

using System;
using System.Diagnostics;


namespace PipeFfmpeg
{
    class Program
    {
        static void Main(string[] args)
        {
            //https://stackoverflow.com/questions/19658415/how-to-use-pipe-in-ffmpeg-within-c-sharp
            Process proc = new Process();

            //Get mpegps in stdin pipe, and convert to mpegts stream (the -bsf:v dump_extra may or may not be need).
            //proc.StartInfo.FileName = @"ffplay.exe";
            proc.StartInfo.FileName = @"mpv.exe";            
            proc.StartInfo.Arguments = String.Format(" - ");    //Get data from input pipe
            proc.StartInfo.UseShellExecute = false;
            proc.StartInfo.RedirectStandardInput = true;
            proc.StartInfo.RedirectStandardOutput = false;

            proc.Start();

            //https://superuser.com/questions/1708271/ffmpeg-waits-to-close-the-pipe-in-order-to-start-processing-data
            byte[] allBytes = System.IO.File.ReadAllBytes("input.mpeg");

            //Write the same content 500 times.
            for (int i = 0; i < 500; i++)
            {
                int idx = 0;
                for (;;)
                {
                    proc.StandardInput.BaseStream.Write(allBytes, idx, Math.Min(100, allBytes.Length - idx));
                    idx += 100;

                    if (idx >= allBytes.Length) break;
                }
            }
        }
    }
}

---

Sample output:

---

Update:

• Write the mpeg-ps data to stdin pipe of FFmpeg.
• FFmpeg generates new timestamps, converts to raw AVI format, and passes to stdout pipe.
• A thread reads data from stdout of FFmpeg and write it to stdin pipe of MPV.

Code sample:

using System;
using System.Diagnostics;
using System.Threading.Tasks;


namespace PipeFfmpeg
{
    class Program
    {
        static void Main(string[] args)
        {
            //https://stackoverflow.com/questions/19658415/how-to-use-pipe-in-ffmpeg-within-c-sharp
            //Process proc = new Process();

            //https://stackoverflow.com/questions/21213895/how-to-stream-live-videos-with-no-latency-ffplay-mplayer-and-what-kind-of-wra
            //https://github.com/mpv-player/mpv/issues/4213
            //proc.StartInfo.FileName = @"mpv.exe";
            //proc.StartInfo.Arguments = String.Format("--profile=low-latency -f mpeg -vcodec h264 -acodec pcm_alaw - ");    //Get data from input pipe
            //proc.StartInfo.Arguments = String.Format(" - ");    //Get data from input pipe

            Process ffmpeg_proc = new Process();
            ffmpeg_proc.StartInfo.FileName = @"ffmpeg.exe";
            ffmpeg_proc.StartInfo.Arguments = String.Format("-fflags +genpts+igndts+ignidx -probesize 1024 -i pipe: -vf setpts=PTS-STARTPTS -af asetpts=PTS-STARTPTS -vcodec rawvideo -acodec pcm_s16le -pix_fmt bgr24 -f avi pipe:");
            ffmpeg_proc.StartInfo.UseShellExecute = false;
            ffmpeg_proc.StartInfo.RedirectStandardInput = true;
            ffmpeg_proc.StartInfo.RedirectStandardOutput = true;

            Process mpv_proc = new Process();
            mpv_proc.StartInfo.FileName = @"mpv.exe";
            mpv_proc.StartInfo.Arguments = String.Format(" - ");    //Get data from input pipe
            mpv_proc.StartInfo.UseShellExecute = false;
            mpv_proc.StartInfo.RedirectStandardInput = true;
            mpv_proc.StartInfo.RedirectStandardOutput = false;


            //https://stackoverflow.com/questions/16658873/how-to-minimize-the-delay-in-a-live-streaming-with-ffmpeg
            //proc.StartInfo.FileName = @"ffplay.exe";
            //proc.StartInfo.Arguments = String.Format("-f mpegts -i pipe:");    //Get data from input pipe

            //proc.StartInfo.FileName = @"c:\\Program Files\\VideoLAN\\VLC\\vlc.exe";
            //proc.StartInfo.Arguments = String.Format(" - ");    //Get data from input pipe

            mpv_proc.Start();
            ffmpeg_proc.Start();
            
            // Added 3 seconds delay - wait for FFplay or MPV to start.
            System.Threading.Thread.Sleep(3000);

            byte[] allBytes = System.IO.File.ReadAllBytes("input.mpeg");    //12fps

            var helper = Task.Run(() =>
            {
                int n_bytes;
                byte[] buf = new byte[1024];

                while (true)
                {
                    n_bytes = ffmpeg_proc.StandardOutput.BaseStream.Read(buf, 0, 1024);

                    if (n_bytes > 0)
                    {
                        mpv_proc.StandardInput.BaseStream.Write(buf, 0, n_bytes);
                    }
                }
            });

            //Write the same content 500000 times.
            for (int i = 0; i < 500000; i++)
            {
                int idx = 0;

                while (true)
                {
                    int block_size = 500;

                    ffmpeg_proc.StandardInput.BaseStream.Write(allBytes, idx, Math.Min(block_size, allBytes.Length - idx));
                    ffmpeg_proc.StandardInput.BaseStream.Flush();
                    idx += block_size;

                    System.Threading.Thread.Sleep(50);  //Sleep 50msec (83.333msec applies 12fps, assume some overhead).

                    Console.WriteLine("idx = {0}", idx);

                    if (idx >= allBytes.Length) break;
                }
            }
        }
    }
}

---

Same code with FFplay instead of mpv player:

using System;
using System.Diagnostics;
using System.Threading.Tasks;


namespace PipeFfmpeg
{
    class Program
    {
        static void Main(string[] args)
        {
            Process ffmpeg_proc = new Process();
            ffmpeg_proc.StartInfo.FileName = @"ffmpeg.exe";
            ffmpeg_proc.StartInfo.Arguments = String.Format("-fflags +genpts+igndts+ignidx -probesize 1024 -i pipe: -vf setpts=PTS-STARTPTS -af asetpts=PTS-STARTPTS -vcodec rawvideo -acodec pcm_s16le -pix_fmt bgr24 -f avi pipe:");
            ffmpeg_proc.StartInfo.UseShellExecute = false;
            ffmpeg_proc.StartInfo.RedirectStandardInput = true;
            ffmpeg_proc.StartInfo.RedirectStandardOutput = true;

            //Process mpv_proc = new Process();
            //mpv_proc.StartInfo.FileName = @"mpv.exe";
            //mpv_proc.StartInfo.Arguments = String.Format(" - ");    //Get data from input pipe
            //mpv_proc.StartInfo.UseShellExecute = false;
            //mpv_proc.StartInfo.RedirectStandardInput = true;
            //mpv_proc.StartInfo.RedirectStandardOutput = false;

            Process ffplay_proc = new Process();
            ffplay_proc.StartInfo.FileName = @"ffplay.exe";
            ffplay_proc.StartInfo.Arguments = String.Format(" - ");    //Get data from input pipe
            ffplay_proc.StartInfo.UseShellExecute = false;
            ffplay_proc.StartInfo.RedirectStandardInput = true;
            ffplay_proc.StartInfo.RedirectStandardOutput = false;


            ffplay_proc.Start();
            ffmpeg_proc.Start();
            
            // Added 3 seconds delay - wait for FFplay or MPV to start.
            System.Threading.Thread.Sleep(3000);

            byte[] allBytes = System.IO.File.ReadAllBytes("input.mpeg");    //12fps

            var helper = Task.Run(() =>
            {
                int n_bytes;
                byte[] buf = new byte[1024];

                while (true)
                {
                    n_bytes = ffmpeg_proc.StandardOutput.BaseStream.Read(buf, 0, 1024);

                    if (n_bytes > 0)
                    {
                        //mpv_proc.StandardInput.BaseStream.Write(buf, 0, n_bytes);
                        ffplay_proc.StandardInput.BaseStream.Write(buf, 0, n_bytes);
                    }
                }
            });

            //Write the same content 500000 times.
            for (int i = 0; i < 500000; i++)
            {
                int idx = 0;

                while (true)
                {
                    int block_size = 500;

                    ffmpeg_proc.StandardInput.BaseStream.Write(allBytes, idx, Math.Min(block_size, allBytes.Length - idx));
                    ffmpeg_proc.StandardInput.BaseStream.Flush();
                    idx += block_size;

                    System.Threading.Thread.Sleep(50);  //Sleep 50msec (83.333msec applies 12fps, assume some overhead).

                    Console.WriteLine("idx = {0}", idx);

                    if (idx >= allBytes.Length) break;
                }
            }
        }
    }
}

---

VLC Example - re-encoding to H.264 (and aac) and re-muxing to mpegts:

using System;
using System.Diagnostics;
using System.Threading.Tasks;


namespace PipeFfmpeg
{
    class Program
    {
        static void Main(string[] args)
        {
            Process ffmpeg_proc = new Process();
            ffmpeg_proc.StartInfo.FileName = @"ffmpeg.exe";
            ffmpeg_proc.StartInfo.Arguments = String.Format("-fflags +genpts+igndts+ignidx -probesize 1024 -i pipe: -vf setpts=PTS-STARTPTS -af asetpts=PTS-STARTPTS -vcodec libx264 -crf 10 -acodec aac -pix_fmt yuv420p -f mpegts pipe:");
            ffmpeg_proc.StartInfo.UseShellExecute = false;
            ffmpeg_proc.StartInfo.RedirectStandardInput = true;
            ffmpeg_proc.StartInfo.RedirectStandardOutput = true;

            Process vlc_proc = new Process();
            vlc_proc.StartInfo.FileName = @"c:\\Program Files\\VideoLAN\\VLC\\vlc.exe";
            vlc_proc.StartInfo.Arguments = String.Format(" - ");    //Get data from input pipe
            vlc_proc.StartInfo.UseShellExecute = false;
            vlc_proc.StartInfo.RedirectStandardInput = true;
            vlc_proc.StartInfo.RedirectStandardOutput = false;

            vlc_proc.Start();
            ffmpeg_proc.Start();
            
            byte[] allBytes = System.IO.File.ReadAllBytes("input.mpeg");    //12fps

            var helper = Task.Run(() =>
            {
                int n_bytes;
                byte[] buf = new byte[1024];

                while (true)
                {
                    n_bytes = ffmpeg_proc.StandardOutput.BaseStream.Read(buf, 0, 1024);

                    if (n_bytes > 0)
                    {
                        vlc_proc.StandardInput.BaseStream.Write(buf, 0, n_bytes);
                    }
                }
            });

            //Write the same content 500000 times.
            for (int i = 0; i < 500000; i++)
            {
                int idx = 0;

                while (true)
                {
                    int block_size = 500;

                    ffmpeg_proc.StandardInput.BaseStream.Write(allBytes, idx, Math.Min(block_size, allBytes.Length - idx));
                    ffmpeg_proc.StandardInput.BaseStream.Flush();
                    idx += block_size;

                    System.Threading.Thread.Sleep(50);  //Sleep 50msec (83.333msec applies 12fps, assume some overhead).

                    Console.WriteLine("idx = {0}", idx);

                    if (idx >= allBytes.Length) break;
                }
            }
        }
    }
}