| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit optimizes HTTP performance by reducing forward seeks, instead
favoring a read-ahead and discard on the current connection (referred to
as a short seek) for seeks that are within a TCP window's worth of data.
This improves performance because with TCP flow control, a window's worth
of data will be in the local socket buffer already or in-flight from the
sender once congestion control on the sender is fully utilizing the window.
Note: this approach doesn't attempt to differentiate from a newly opened
connection which may not be fully utilizing the window due to congestion
control vs one that is. The receiver can't get at this information, so we
assume worst case; that full window is in use (we did advertise it after all)
and that data could be in-flight
The previous behavior of closing the connection, then opening a new
with a new HTTP range value results in a massive amounts of discarded
and re-sent data when large TCP windows are used. This has been observed
on MacOS/iOS which starts with an initial window of 256KB and grows up to
1MB depending on the bandwidth-product delay.
When seeking within a window's worth of data and we close the connection,
then open a new one within the same window's worth of data, we discard
from the current offset till the end of the window. Then on the new
connection the server ends up re-sending the previous data from new
offset till the end of old window.
Example (assumes full window utilization):
TCP window size: 64KB
Position: 32KB
Forward seek position: 40KB
* (Next window)
32KB |--------------| 96KB |---------------| 160KB
*
40KB |---------------| 104KB
Re-sent amount: 96KB - 40KB = 56KB
For a real world test example, I have MP4 file of ~25MB, which ffplay
only reads ~16MB and performs 177 seeks. With current ffmpeg, this results
in 177 HTTP GETs and ~73MB worth of TCP data communication. With this
patch, ffmpeg issues 4 HTTP GETs and 3 seeks for a total of ~22MB of TCP data
communication.
To support this feature, the short seek logic in avio_seek() has been
extended to call a function to get the short seek threshold value. This
callback has been plumbed to the URLProtocol structure, which now has
infrastructure in HTTP and TCP to get the underlying receiver window size
via SO_RCVBUF. If the underlying URL and protocol don't support returning
a short seek threshold, the default s->short_seek_threshold is used
This feature has been tested on Windows 7 and MacOS/iOS. Windows support
is slightly complicated by the fact that when TCP window auto-tuning is
enabled, SO_RCVBUF doesn't report the real window size, but it does if
SO_RCVBUF was manually set (disabling auto-tuning). So we can only use
this optimization on Windows in the later case
Signed-off-by: Joel Cunningham <joel.cunningham@me.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit adds the avio_get_dyn_buf function which allows accessing
the
content of a DynBuffer without destroying it.
This is required in matroskaenc for preliminary writing (correct) mkv
headers.
Context for this change is fixing regression bug #5977.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
|
|
|
|
| |
Signed-off-by: Stephan Holljes <klaxa1337@googlemail.com>
|
|
|
|
|
|
| |
Reviewed-by: Nicolas George <george@nsup.org>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
|
|
|
|
| |
Signed-off-by: James Almer <jamrial@gmail.com>
|
|\
| |
| |
| |
| |
| |
| | |
* commit 'db7968bff4851c2be79b15b2cb2ae747424d2fca':
avio: Allow custom IO users to get labels for the output bytestream
Merged-by: Matthieu Bouron <matthieu.bouron@stupeflix.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This allows callers with avio write callbacks to get the bytestream
positions that correspond to keyframes, suitable for live streaming.
In the simplest form, a caller could expect that a header is written
to the bytestream during the avformat_write_header, and the data
output to the avio context during e.g. av_write_frame corresponds
exactly to the current packet passed in.
When combined with av_interleaved_write_frame, and with muxers that
do buffering (most muxers that do some sort of fragmenting or
clustering), the mapping from input data to bytestream positions
is nontrivial.
This allows callers to get directly information about what part
of the bytestream is what, without having to resort to assumptions
about the muxer behaviour.
One keyframe/fragment/block can still be split into multiple (if
they are larger than the aviocontext buffer), which would call
the callback with e.g. AVIO_DATA_MARKER_SYNC_POINT, followed by
AVIO_DATA_MARKER_UNKNOWN for the second time it is called with
the following data.
Signed-off-by: Martin Storsjö <martin@martin.st>
|
|\|
| |
| |
| |
| |
| |
| | |
* commit 'fab8156b2f30666adabe227b3d7712fd193873b1':
avio: Copy URLContext generic options into child URLContexts
Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Since all URLContexts have the same AVOptions, such AVOptions
will be applied on the outermost context only and removed from the
dict, while they probably make sense on all contexts.
This makes sure that rw_timeout gets propagated to the innermost
URLContext (to make sure it gets passed to the tcp protocol, when
opening a http connection for instance).
Alternatively, such matching options would be kept in the dict
and only removed after the ffurl_connect call.
Signed-off-by: Martin Storsjö <martin@martin.st>
|
| |
| |
| |
| |
| |
| |
| |
| | |
Should make the default behaviour safer for careless callers that open
random untrusted files.
Bug-Id: CVE-2016-1897
Bug-Id: CVE-2016-1898
|
| |
| |
| |
| |
| | |
This way, the decisions about which protocols are available for use in
any given situations can be delegated to the caller.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This reverts commit 9f9ed79d4cb40e5d9093899f8a79086ff23da844.
The hlsopts member was never set anywhere and always NULL, furthermore
the HLS demuxer needs to retrieve the proper options from the underlying
http protocol (cookies, user-agent, etc), so a dummy context won't help.
Instead, use the AVIOContext directly to access the options.
|
| |
| |
| |
| | |
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
|
| |
| |
| |
| | |
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
|
|\|
| |
| |
| |
| |
| |
| | |
* commit 'cae448cfbf31d492cba782bc64fc4eed556ed83d':
aviobuf: add a private data struct for avio_open()ed contexts
Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
|
| |
| |
| |
| | |
It will be useful in the following commits.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Note to maintainers: update tools
Note to maintainers: set a default whitelist for your protocol
If that makes no sense then consider to set "none" and thus require the user to specify a white-list
for sub-protocols to be opened
Note, testing and checking for missing changes is needed
Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
|
| |
| |
| |
| | |
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
|
| |
| |
| |
| |
| |
| |
| |
| | |
Fixes out of array read
Fixes: 03c406ec9530e594a074ce2979f8a1f0/asan_heap-oob_7dec26_4664_37c52495b2870a2eaac65f53958e76c1.flac
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
|
| |
| |
| |
| |
| |
| |
| | |
Previous "currently size is limited" didn't give away
much in terms of useful info.
Signed-off-by: Reynaldo H. Verdejo Pinochet <reynaldo@osg.samsung.com>
|
| |
| |
| |
| | |
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
|
| |
| |
| |
| | |
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
|
| |
| |
| |
| | |
Signed-off-by: Stephan Holljes <klaxa1337@googlemail.com>
|
|\|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* commit '059a934806d61f7af9ab3fd9f74994b838ea5eba':
lavc: Consistently prefix input buffer defines
Conflicts:
doc/examples/decoding_encoding.c
libavcodec/4xm.c
libavcodec/aac_adtstoasc_bsf.c
libavcodec/aacdec.c
libavcodec/aacenc.c
libavcodec/ac3dec.h
libavcodec/asvenc.c
libavcodec/avcodec.h
libavcodec/avpacket.c
libavcodec/dvdec.c
libavcodec/ffv1enc.c
libavcodec/g2meet.c
libavcodec/gif.c
libavcodec/h264.c
libavcodec/h264_mp4toannexb_bsf.c
libavcodec/huffyuvdec.c
libavcodec/huffyuvenc.c
libavcodec/jpeglsenc.c
libavcodec/libxvid.c
libavcodec/mdec.c
libavcodec/motionpixels.c
libavcodec/mpeg4videodec.c
libavcodec/mpegvideo.c
libavcodec/noise_bsf.c
libavcodec/nuv.c
libavcodec/nvenc.c
libavcodec/options.c
libavcodec/parser.c
libavcodec/pngenc.c
libavcodec/proresenc_kostya.c
libavcodec/qsvdec.c
libavcodec/svq1enc.c
libavcodec/tiffenc.c
libavcodec/truemotion2.c
libavcodec/utils.c
libavcodec/utvideoenc.c
libavcodec/vc1dec.c
libavcodec/wmalosslessdec.c
libavformat/adxdec.c
libavformat/aiffdec.c
libavformat/apc.c
libavformat/apetag.c
libavformat/avidec.c
libavformat/bink.c
libavformat/cafdec.c
libavformat/flvdec.c
libavformat/id3v2.c
libavformat/isom.c
libavformat/matroskadec.c
libavformat/mov.c
libavformat/mpc.c
libavformat/mpc8.c
libavformat/mpegts.c
libavformat/mvi.c
libavformat/mxfdec.c
libavformat/mxg.c
libavformat/nutdec.c
libavformat/oggdec.c
libavformat/oggparsecelt.c
libavformat/oggparseflac.c
libavformat/oggparseopus.c
libavformat/oggparsespeex.c
libavformat/omadec.c
libavformat/rawdec.c
libavformat/riffdec.c
libavformat/rl2.c
libavformat/rmdec.c
libavformat/rtpdec_latm.c
libavformat/rtpdec_mpeg4.c
libavformat/rtpdec_qdm2.c
libavformat/rtpdec_svq3.c
libavformat/sierravmd.c
libavformat/smacker.c
libavformat/smush.c
libavformat/spdifenc.c
libavformat/takdec.c
libavformat/tta.c
libavformat/utils.c
libavformat/vqf.c
libavformat/westwood_vqa.c
libavformat/xmv.c
libavformat/xwma.c
libavformat/yop.c
Merged-by: Michael Niedermayer <michael@niedermayer.cc>
|
| |
| |
| |
| | |
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Calling ffio_ensure_seekback() if ffio_init_checksum() has been called
on the same context can lead to out of bounds memory accesses and
crashes. The reason is that ffio_ensure_seekback() does not update
checksum_ptr after reallocating the buffer, resulting in a dangling
pointer.
This effectively fixes potential crashes when opening mp3 files.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
|
| |
| |
| |
| |
| |
| | |
This allows us to adjust it internally.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
|
| |
| |
| |
| | |
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
|
|\|
| |
| |
| |
| |
| |
| | |
* commit 'bff0349d9da527084e1433167466d8afc9e25c7f':
avio: Add avio_read wrapper to simplify error checking
Merged-by: Michael Niedermayer <michaelni@gmx.at>
|
| |
| |
| |
| | |
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
|
| |
| |
| |
| |
| | |
Previous version reviewed-by: wm4 <nfxjfg@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
there is no read function
Without a read function there can never be more data read than the buffer could
hold
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
|
|\|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* commit 'b97f427fb56bdbf7de8e2d4b72d01114aa6eceda':
lavf: Explicitly convert types at function pointer assignment
Conflicts:
libavformat/avio.c
libavformat/aviobuf.c
libavformat/swfenc.c
See: a76a2ffe9d7773d59bb1bc228a6660e23672c490 and others
Merged-by: Michael Niedermayer <michaelni@gmx.at>
|
| |
| |
| |
| | |
This fixes a number of "assignment from incompatible pointer type" warnings.
|
| |
| |
| |
| | |
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
|
|\|
| |
| |
| |
| |
| |
| | |
* commit 'c86d8aed748adc3ba8df96fb1fe14aa45da77a31':
avio: Rename avclass symbols relating to avio
Merged-by: Michael Niedermayer <michaelni@gmx.at>
|
| |
| |
| |
| |
| |
| |
| | |
Don't prefix them ffio_url, which is misleading, sounding too
much like the urlprotocol layer (like ffurl_*).
Signed-off-by: Martin Storsjö <martin@martin.st>
|
|\|
| |
| |
| |
| |
| |
| | |
* commit '8a273a746061a112e5e35066a8fd8e146d821a62':
avio: Add an internal utility function for freeing dynamic buffers
Merged-by: Michael Niedermayer <michaelni@gmx.at>
|
| |
| |
| |
| | |
Signed-off-by: Martin Storsjö <martin@martin.st>
|
| |
| |
| |
| |
| |
| | |
This undoes the effect of 3c18a7b18807de81566381a1bcbe9f6103c0296b for reading
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
|
|\|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* commit '3c18a7b18807de81566381a1bcbe9f6103c0296b':
avio: Do not consider the end-of-buffer position valid
Conflicts:
libavformat/aviobuf.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Trigger a refill if the seek action moves the pointer
at the end of the buffer.
Before this patch the read action following the seek would trigger
the refill, while write action would write outside the buffer.
In the Libav codebase few muxers seek forward outside of what
already has been written so it is quite unlikely to experience
the problem with the default buffer size.
CC: libav-stable@libav.org
|
| |
| |
| |
| |
| |
| |
| | |
Fixes out of array access
Suggested-by: Andrew Scherkus <scherkus@google.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
|
| |
| |
| |
| |
| |
| | |
Fixes ticket4152
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
|
|\|
| |
| |
| |
| |
| |
| |
| | |
* commit 'd7f530b0d67fe3996dbfa990a47c865e0d1400a2':
aviobuf: check context before using it
See: 7441d1ec330da810a0ffd44a02b2fc60add5b719
Merged-by: Michael Niedermayer <michaelni@gmx.at>
|
| |
| |
| |
| |
| |
| |
| | |
Avoid a possible null pointer dereference.
CC: libav-stable@libav.org
Bug-Id: CID 1135769
|
| |
| |
| |
| |
| |
| |
| | |
avio_flush() did nothing useful for read streams. Fix it to behave as
expected, and discard the currently read buffer properly.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
|
| |
| |
| |
| | |
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
|
| |
| |
| |
| |
| |
| |
| | |
It's a public function and should use the avio_ namespace
Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
|