summaryrefslogtreecommitdiff
path: root/libavformat/aviobuf.c
Commit message (Collapse)AuthorAge
* avformat/aviobuf: add support for specifying minimum packet size and marking ↵Marton Balint2017-06-24
| | | | | | | flush points Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Marton Balint <cus@passwd.hu>
* avformat/aviobuf: fix flushing write buffers after seeking backward or forwardMarton Balint2017-06-24
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch makes aviobuf work more like traditinal file IO, which is how people think about it. For example, in the past, aviobuf only flushed buffers until the current buffer position, even if more data was written to it previously, and a backward seek was used to reposition the IO context. From now, aviobuf will keep track of the written data, so no explicit seek will be required till the end of the buffer, or till the end of file before flushing. This fixes at least one regression, fate-vsynth3-flv was broken if flush_packets option was set to false, an explicit seek was removed in 4e3cc4bdd8acedbcc703607ed0efbb64bb5c3cc4. Also from now on, if a forward seek in the write buffer were to cause a gap between the already written data and the new file position, a flush will happen. The must_flush varable is also removed, which might have caused needless flushes with multiple seeks whithin the write buffer. Since we know the amount of data written to it, we will know when to flush. Signed-off-by: Marton Balint <cus@passwd.hu>
* avformat/aviobuf: Only downscale the buffer once it has been usedMichael Niedermayer2017-05-19
| | | | | | The code mistook the first iteration sometimes as the end Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* Merge commit '3f75e5116b900f1428aa13041fc7d6301bf1988a'James Almer2017-04-13
|\ | | | | | | | | | | | | * commit '3f75e5116b900f1428aa13041fc7d6301bf1988a': avio: Keep track of the amount of data written Merged-by: James Almer <jamrial@gmail.com>
| * avio: Keep track of the amount of data writtenLuca Barbato2016-12-08
| | | | | | | | Make avio_size() work with any write AVIOContext.
* | Merge commit '8ea35af7620e4f73f9e8c072e1c0fac9a04ec161'James Almer2017-03-21
|\| | | | | | | | | | | | | * commit '8ea35af7620e4f73f9e8c072e1c0fac9a04ec161': avio: add a new flag for marking streams seekable by timestamp Merged-by: James Almer <jamrial@gmail.com>
| * avio: add a new flag for marking streams seekable by timestampAnton Khirnov2016-09-30
| |
* | Merge commit '83548fe894cdb455cc127f754d09905b6d23c173'James Almer2017-03-21
|\| | | | | | | | | | | | | * commit '83548fe894cdb455cc127f754d09905b6d23c173': lavf: fix usage of AVIOContext.seekable Merged-by: James Almer <jamrial@gmail.com>
| * lavf: fix usage of AVIOContext.seekableAnton Khirnov2016-09-30
| | | | | | | | | | | | | | | | It is supposed to be a flag. The only currently defined value is AVIO_SEEKABLE_NORMAL, but other ones may be added in the future. However all the current lavf code treats this field as a bool (mainly for historical reasons). Change all those cases to properly check for AVIO_SEEKABLE_NORMAL.
* | HTTP: improve performance by reducing forward seeksJoel Cunningham2017-02-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit optimizes HTTP performance by reducing forward seeks, instead favoring a read-ahead and discard on the current connection (referred to as a short seek) for seeks that are within a TCP window's worth of data. This improves performance because with TCP flow control, a window's worth of data will be in the local socket buffer already or in-flight from the sender once congestion control on the sender is fully utilizing the window. Note: this approach doesn't attempt to differentiate from a newly opened connection which may not be fully utilizing the window due to congestion control vs one that is. The receiver can't get at this information, so we assume worst case; that full window is in use (we did advertise it after all) and that data could be in-flight The previous behavior of closing the connection, then opening a new with a new HTTP range value results in a massive amounts of discarded and re-sent data when large TCP windows are used. This has been observed on MacOS/iOS which starts with an initial window of 256KB and grows up to 1MB depending on the bandwidth-product delay. When seeking within a window's worth of data and we close the connection, then open a new one within the same window's worth of data, we discard from the current offset till the end of the window. Then on the new connection the server ends up re-sending the previous data from new offset till the end of old window. Example (assumes full window utilization): TCP window size: 64KB Position: 32KB Forward seek position: 40KB * (Next window) 32KB |--------------| 96KB |---------------| 160KB * 40KB |---------------| 104KB Re-sent amount: 96KB - 40KB = 56KB For a real world test example, I have MP4 file of ~25MB, which ffplay only reads ~16MB and performs 177 seeks. With current ffmpeg, this results in 177 HTTP GETs and ~73MB worth of TCP data communication. With this patch, ffmpeg issues 4 HTTP GETs and 3 seeks for a total of ~22MB of TCP data communication. To support this feature, the short seek logic in avio_seek() has been extended to call a function to get the short seek threshold value. This callback has been plumbed to the URLProtocol structure, which now has infrastructure in HTTP and TCP to get the underlying receiver window size via SO_RCVBUF. If the underlying URL and protocol don't support returning a short seek threshold, the default s->short_seek_threshold is used This feature has been tested on Windows 7 and MacOS/iOS. Windows support is slightly complicated by the fact that when TCP window auto-tuning is enabled, SO_RCVBUF doesn't report the real window size, but it does if SO_RCVBUF was manually set (disabling auto-tuning). So we can only use this optimization on Windows in the later case Signed-off-by: Joel Cunningham <joel.cunningham@me.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* | libavformat/avio: Add avio_get_dyn_buf functionsoftworkz2017-01-07
| | | | | | | | | | | | | | | | | | | | | | | | | | This commit adds the avio_get_dyn_buf function which allows accessing the content of a DynBuffer without destroying it. This is required in matroskaenc for preliminary writing (correct) mkv headers. Context for this change is fixing regression bug #5977. Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* | lavf/aviobuf.c: Adapt avio_accept and avio_handshake to new AVIOContext APIStephan Holljes2016-10-03
| | | | | | | | Signed-off-by: Stephan Holljes <klaxa1337@googlemail.com>
* | avformat/avio: add a crc callback function for CRC-32 IEEE-LEJames Almer2016-08-04
| | | | | | | | | | | | Reviewed-by: Nicolas George <george@nsup.org> Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: James Almer <jamrial@gmail.com>
* | cosmetics: fix some misspelled wordsJames Almer2016-07-17
| | | | | | | | Signed-off-by: James Almer <jamrial@gmail.com>
* | Merge commit 'db7968bff4851c2be79b15b2cb2ae747424d2fca'Matthieu Bouron2016-06-23
|\| | | | | | | | | | | | | * commit 'db7968bff4851c2be79b15b2cb2ae747424d2fca': avio: Allow custom IO users to get labels for the output bytestream Merged-by: Matthieu Bouron <matthieu.bouron@stupeflix.com>
| * avio: Allow custom IO users to get labels for the output bytestreamMartin Storsjö2016-05-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows callers with avio write callbacks to get the bytestream positions that correspond to keyframes, suitable for live streaming. In the simplest form, a caller could expect that a header is written to the bytestream during the avformat_write_header, and the data output to the avio context during e.g. av_write_frame corresponds exactly to the current packet passed in. When combined with av_interleaved_write_frame, and with muxers that do buffering (most muxers that do some sort of fragmenting or clustering), the mapping from input data to bytestream positions is nontrivial. This allows callers to get directly information about what part of the bytestream is what, without having to resort to assumptions about the muxer behaviour. One keyframe/fragment/block can still be split into multiple (if they are larger than the aviocontext buffer), which would call the callback with e.g. AVIO_DATA_MARKER_SYNC_POINT, followed by AVIO_DATA_MARKER_UNKNOWN for the second time it is called with the following data. Signed-off-by: Martin Storsjö <martin@martin.st>
* | Merge commit 'fab8156b2f30666adabe227b3d7712fd193873b1'Derek Buitenhuis2016-04-21
|\| | | | | | | | | | | | | * commit 'fab8156b2f30666adabe227b3d7712fd193873b1': avio: Copy URLContext generic options into child URLContexts Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
| * avio: Copy URLContext generic options into child URLContextsMartin Storsjö2016-03-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since all URLContexts have the same AVOptions, such AVOptions will be applied on the outermost context only and removed from the dict, while they probably make sense on all contexts. This makes sure that rw_timeout gets propagated to the innermost URLContext (to make sure it gets passed to the tcp protocol, when opening a http connection for instance). Alternatively, such matching options would be kept in the dict and only removed after the ffurl_connect call. Signed-off-by: Martin Storsjö <martin@martin.st>
| * lavf: add a protocol whitelist/blacklist for file opened internallyAnton Khirnov2016-02-22
| | | | | | | | | | | | | | | | Should make the default behaviour safer for careless callers that open random untrusted files. Bug-Id: CVE-2016-1897 Bug-Id: CVE-2016-1898
| * urlprotocol: receive a list of protocols from the callerAnton Khirnov2016-02-22
| | | | | | | | | | This way, the decisions about which protocols are available for use in any given situations can be delegated to the caller.
* | hls: read protocol options through the AVIOContextHendrik Leppkes2016-03-16
| | | | | | | | | | | | | | | | | | | | This reverts commit 9f9ed79d4cb40e5d9093899f8a79086ff23da844. The hlsopts member was never set anywhere and always NULL, furthermore the HLS demuxer needs to retrieve the proper options from the underlying http protocol (cookies, user-agent, etc), so a dummy context won't help. Instead, use the AVIOContext directly to access the options.
* | avformat: Add a protocol blacklisting APIDerek Buitenhuis2016-03-04
| | | | | | | | Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
* | hls: Add and use a memebr of AVIOInternal rather than abuse opaqueDerek Buitenhuis2016-02-29
| | | | | | | | Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
* | Merge commit 'cae448cfbf31d492cba782bc64fc4eed556ed83d'Derek Buitenhuis2016-02-29
|\| | | | | | | | | | | | | * commit 'cae448cfbf31d492cba782bc64fc4eed556ed83d': aviobuf: add a private data struct for avio_open()ed contexts Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
| * aviobuf: add a private data struct for avio_open()ed contextsAnton Khirnov2016-02-22
| | | | | | | | It will be useful in the following commits.
* | avformat: add protocol_whitelistMichael Niedermayer2016-02-02
| | | | | | | | | | | | | | | | | | | | | | | | | | Note to maintainers: update tools Note to maintainers: set a default whitelist for your protocol If that makes no sense then consider to set "none" and thus require the user to specify a white-list for sub-protocols to be opened Note, testing and checking for missing changes is needed Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* | avformat/avio: free url/avio optionsMichael Niedermayer2016-01-29
| | | | | | | | Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* | avformat/aviobuf: Fix end check in put_str16()Michael Niedermayer2016-01-13
| | | | | | | | | | | | | | | | Fixes out of array read Fixes: 03c406ec9530e594a074ce2979f8a1f0/asan_heap-oob_7dec26_4664_37c52495b2870a2eaac65f53958e76c1.flac Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* | avio: add detail to avio_printf() size warningReynaldo H. Verdejo Pinochet2015-12-27
| | | | | | | | | | | | | | Previous "currently size is limited" didn't give away much in terms of useful info. Signed-off-by: Reynaldo H. Verdejo Pinochet <reynaldo@osg.samsung.com>
* | avformat/aviobuf: Simplify avio_read() and avio_seek()Bryan Huh2015-11-16
| | | | | | | | Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* | avformat/aviobuf: Improve readability of aviobuf (Add comments and docs)Bryan Huh2015-11-16
| | | | | | | | Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* | lavf/avio: add avio_accept and avio_handshakeStephan Holljes2015-08-01
| | | | | | | | Signed-off-by: Stephan Holljes <klaxa1337@googlemail.com>
* | Merge commit '059a934806d61f7af9ab3fd9f74994b838ea5eba'Michael Niedermayer2015-07-27
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit '059a934806d61f7af9ab3fd9f74994b838ea5eba': lavc: Consistently prefix input buffer defines Conflicts: doc/examples/decoding_encoding.c libavcodec/4xm.c libavcodec/aac_adtstoasc_bsf.c libavcodec/aacdec.c libavcodec/aacenc.c libavcodec/ac3dec.h libavcodec/asvenc.c libavcodec/avcodec.h libavcodec/avpacket.c libavcodec/dvdec.c libavcodec/ffv1enc.c libavcodec/g2meet.c libavcodec/gif.c libavcodec/h264.c libavcodec/h264_mp4toannexb_bsf.c libavcodec/huffyuvdec.c libavcodec/huffyuvenc.c libavcodec/jpeglsenc.c libavcodec/libxvid.c libavcodec/mdec.c libavcodec/motionpixels.c libavcodec/mpeg4videodec.c libavcodec/mpegvideo.c libavcodec/noise_bsf.c libavcodec/nuv.c libavcodec/nvenc.c libavcodec/options.c libavcodec/parser.c libavcodec/pngenc.c libavcodec/proresenc_kostya.c libavcodec/qsvdec.c libavcodec/svq1enc.c libavcodec/tiffenc.c libavcodec/truemotion2.c libavcodec/utils.c libavcodec/utvideoenc.c libavcodec/vc1dec.c libavcodec/wmalosslessdec.c libavformat/adxdec.c libavformat/aiffdec.c libavformat/apc.c libavformat/apetag.c libavformat/avidec.c libavformat/bink.c libavformat/cafdec.c libavformat/flvdec.c libavformat/id3v2.c libavformat/isom.c libavformat/matroskadec.c libavformat/mov.c libavformat/mpc.c libavformat/mpc8.c libavformat/mpegts.c libavformat/mvi.c libavformat/mxfdec.c libavformat/mxg.c libavformat/nutdec.c libavformat/oggdec.c libavformat/oggparsecelt.c libavformat/oggparseflac.c libavformat/oggparseopus.c libavformat/oggparsespeex.c libavformat/omadec.c libavformat/rawdec.c libavformat/riffdec.c libavformat/rl2.c libavformat/rmdec.c libavformat/rtpdec_latm.c libavformat/rtpdec_mpeg4.c libavformat/rtpdec_qdm2.c libavformat/rtpdec_svq3.c libavformat/sierravmd.c libavformat/smacker.c libavformat/smush.c libavformat/spdifenc.c libavformat/takdec.c libavformat/tta.c libavformat/utils.c libavformat/vqf.c libavformat/westwood_vqa.c libavformat/xmv.c libavformat/xwma.c libavformat/yop.c Merged-by: Michael Niedermayer <michael@niedermayer.cc>
| * lavc: Consistently prefix input buffer definesVittorio Giovara2015-07-27
| | | | | | | | Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
| * avio: Add avio_put_str16beLuca Barbato2015-06-07
| |
* | avio: fix potential crashes when combining ffio_ensure_seekback + crcwm42015-06-17
| | | | | | | | | | | | | | | | | | | | | | | | Calling ffio_ensure_seekback() if ffio_init_checksum() has been called on the same context can lead to out of bounds memory accesses and crashes. The reason is that ffio_ensure_seekback() does not update checksum_ptr after reallocating the buffer, resulting in a dangling pointer. This effectively fixes potential crashes when opening mp3 files. Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* | avformat/avio: move short seek threshold to the contextMichael Niedermayer2015-06-11
| | | | | | | | | | | | This allows us to adjust it internally. Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* | avformat/aviobuf: Check for ffio_set_buf_size() failureMichael Niedermayer2015-06-07
| | | | | | | | Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* | Merge commit 'bff0349d9da527084e1433167466d8afc9e25c7f'Michael Niedermayer2015-06-01
|\| | | | | | | | | | | | | * commit 'bff0349d9da527084e1433167466d8afc9e25c7f': avio: Add avio_read wrapper to simplify error checking Merged-by: Michael Niedermayer <michaelni@gmx.at>
| * avio: Add avio_read wrapper to simplify error checkingVittorio Giovara2015-05-31
| | | | | | | | Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
* | avformat: add callback for opening further filesMichael Niedermayer2015-05-11
| | | | | | | | | | Previous version reviewed-by: wm4 <nfxjfg@googlemail.com> Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* | avformat/aviobuf: Do not resize the buffer in ffio_ensure_seekback() if ↵Michael Niedermayer2015-04-27
| | | | | | | | | | | | | | | | | | there is no read function Without a read function there can never be more data read than the buffer could hold Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* | Merge commit 'b97f427fb56bdbf7de8e2d4b72d01114aa6eceda'Michael Niedermayer2015-03-12
|\| | | | | | | | | | | | | | | | | | | | | | | | | * commit 'b97f427fb56bdbf7de8e2d4b72d01114aa6eceda': lavf: Explicitly convert types at function pointer assignment Conflicts: libavformat/avio.c libavformat/aviobuf.c libavformat/swfenc.c See: a76a2ffe9d7773d59bb1bc228a6660e23672c490 and others Merged-by: Michael Niedermayer <michaelni@gmx.at>
| * lavf: Explicitly convert types at function pointer assignmentDiego Biurrun2015-03-11
| | | | | | | | This fixes a number of "assignment from incompatible pointer type" warnings.
* | libavformat/avio: added avio_put_str16beMark Reid2015-03-03
| | | | | | | | Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* | Merge commit 'c86d8aed748adc3ba8df96fb1fe14aa45da77a31'Michael Niedermayer2015-02-28
|\| | | | | | | | | | | | | * commit 'c86d8aed748adc3ba8df96fb1fe14aa45da77a31': avio: Rename avclass symbols relating to avio Merged-by: Michael Niedermayer <michaelni@gmx.at>
| * avio: Rename avclass symbols relating to avioMartin Storsjö2015-02-28
| | | | | | | | | | | | | | Don't prefix them ffio_url, which is misleading, sounding too much like the urlprotocol layer (like ffurl_*). Signed-off-by: Martin Storsjö <martin@martin.st>
* | Merge commit '8a273a746061a112e5e35066a8fd8e146d821a62'Michael Niedermayer2015-02-25
|\| | | | | | | | | | | | | * commit '8a273a746061a112e5e35066a8fd8e146d821a62': avio: Add an internal utility function for freeing dynamic buffers Merged-by: Michael Niedermayer <michaelni@gmx.at>
| * avio: Add an internal utility function for freeing dynamic buffersMartin Storsjö2015-02-24
| | | | | | | | Signed-off-by: Martin Storsjö <martin@martin.st>
* | avformat/aviobuf: Allow seeking to the end of the buffer for read only buffersMichael Niedermayer2015-01-28
| | | | | | | | | | | | This undoes the effect of 3c18a7b18807de81566381a1bcbe9f6103c0296b for reading Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-23 21:59:18 +0000