summaryrefslogtreecommitdiff
path: root/libavformat/aviobuf.c
Commit message (Collapse)AuthorAge
* avformat/aviobuf: Also return truncated buffer in avio_get_dyn_buf()Andreas Rheinhardt2020-06-11
| | | | | | | | | | | | | | | | | Two kinds of errors can happen when working with dynamic buffers: (Re)allocation errors or truncation errors (one has to truncate the buffer to a size of INT_MAX because avio_close_dyn_buf() and avio_get_dyn_buf() both return an int). Right now, avio_get_dyn_buf() returns an empty buffer in either case. But given that avio_get_dyn_buf() does not destroy the dynamic buffer, one can return the buffer in case of truncation and let the user check the error flags and decide for himself instead of hardcoding a single way to proceed in case of truncation. (This actually restores the behaviour from before commit 163bb9ac0af495a5cb95441bdb5c02170440d28c.) Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avformat/aviobuf: Return better error codesAndreas Rheinhardt2020-06-11
| | | | Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avformat/aviobuf: Stop restricting dynamic buffer sizes to INT_MAX/2Andreas Rheinhardt2020-06-11
| | | | | | | | This has originally been done in 568e18b15e2ddf494fd8926707d34ca08c8edce5 as a precaution against integer overflows, but it is actually easy to support the full range of int without overflows. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avformat/aviobuf: Simplify dyn_buf_write() a bitAndreas Rheinhardt2020-06-11
| | | | Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avformat/aviobuf: Don't check for overflow after it happenedAndreas Rheinhardt2020-06-11
| | | | | | | | If adding two ints overflows, it doesn't matter whether the result will be stored in an unsigned or not; and checking afterwards does not make it retroactively defined. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* AVIOContext: switch to child_class_iterate()Anton Khirnov2020-06-10
|
* avformat/aviobuf, nutenc: Move ff_puv_v, ff_get_v_length to nutenc.cAndreas Rheinhardt2020-05-05
| | | | | | | | | | | | | | and make it static again. These functions have been moved from nutenc to aviobuf and internal.h in f8280ff4c00eeaa245085fa9691035203abd168c in order to use them in a forthcoming patch in utils.c. Said patch never happened, so this commit moves them back and makes them static, effectively reverting said commit as well as f8280ff4c00eeaa245085fa9691035203abd168c (which added the ff-prefix to these functions). Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avformat/aviobuf: Add function to reset dynamic bufferAndreas Rheinhardt2020-03-30
| | | | | | | | | Resetting a dynamic buffer means to keep the AVIOContext and the internal buffer used by the dynamic buffer. This is done in order to save (re)allocations when one has a workflow where one opens and closes dynamic buffers in sequence. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* remove CHAR_MIN/CHAR_MAX usagePaul B Mahol2020-03-17
| | | | It is not needed at all.
* avformat/aviobuf: Remove AVIOInternal and one level of indirectionAndreas Rheinhardt2020-01-30
| | | | | | | | | | | | | | | | | | | | | | | In the Libav commit cae448cf, the opaque of every AVIOContext opened by ffio_fdopen() (which is used internally by avio_open() and avio_open2()) changed: It was a simple pointer to an URLContext before, but now it was a structure (namely AVIOInternal) containing a pointer to an URLContext as its only member. The next commits (namely 8c0ceafb and ec4c4839) added members to AVIOInternal to allow white-/blacklisting of protocols. But these two commits were never merged into FFmpeg (they were only merged as no-ops in 510046c2 and 063b26d3), because FFmpeg chose a different way to implement this (in 93629735); and so our AVIOInternal still has exactly one member. This of course means that it is unnecessary to use AVIOInternal as opaque as it is just adding a level of indirection (not only pointer dereference, but also wrapper functions). Therefore this commit removes AVIOInternal entirely and essentially reverts cae448cf. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avformat/aviobuf: Honor avio_open[2] documentationAndreas Rheinhardt2020-01-07
| | | | | | | | | | | The documentation of both avio_open() as well as avio_open2() states that on failure, the pointer to an AVIOContext given to this function (via a pointer to a pointer to an AVIOContext) will be set to NULL. Yet it didn't happen upon failure of ffurl_open_whitelist() or when allocating the internal buffer failed. This commit changes this. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avformat/aviobuf: Avoid allocating buffer when using dynamic bufferAndreas Rheinhardt2019-11-28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Up until now, using a dynamic buffer entailed at least three allocations: One for the AVIOContext, one for the AVIOContext's opaque (which, among other things, contains the small write buffer), and one for the big buffer that is independently allocated that is returned when calling avio_close_dyn_buf(). It is possible to avoid the third allocation if one doesn't use a packetized dynamic buffer, if all the data written so far fit into the write buffer and if one does not require the actual (big) buffer to have an indefinite lifetime. This is done by making avio_get_dyn_buf() return a pointer to the data in the write buffer if nothing has been written to the main buffer yet. The dynamic buffer will then be freed using ffio_free_dynamic_buffer (which needed to be modified not to call avio_close_dyn_buf() internally). So a typical use-case like: size = avio_close_dyn_buf(dyn_pb, &buf); do something with buf av_free(buf); can be converted to: size = avio_get_dyn_buf(dyn_pb, &buf); do something with buf ffio_free_dynamic_buffer(&dyn_pb); In more complex scenarios this can simplify freeing as well, because it is now clear that freeing always has to be performed via ffio_free_dynamic_buffer(). Of course, in case this saves an allocation it also saves a memcpy. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avformat/avio: remove 4k limit from avio_printfMarton Balint2019-08-17
| | | | | | | | We do this by switching to AVBPrint. v2: Also set IO context error flag in case of ENOMEM. Signed-off-by: Marton Balint <cus@passwd.hu>
* avformat/avio: add avio_print_string_array and avio_printMarton Balint2019-08-17
| | | | | | | These functions can be used to print a variable number of strings consecutively to the IO context. Unlike av_bprintf, no temporary buffer is necessary. Signed-off-by: Marton Balint <cus@passwd.hu>
* lavf/avio: add a ffio_realloc_buf API for AVIO buffer realloctomajsjiang2019-08-15
| | | | | | Add new API ffio_realloc_buf for AVIO buffer realloc. Signed-off-by: Zhongxing Jiang <tomajsjiang@tencent.com>
* lavf/aviobuf: make AVSEEK_SIZE usable from outside.Nicolas George2019-08-13
|
* lavf/avio: remove ffio_open2_wrapper functionJun Zhao2019-07-21
| | | | | | Remove the function ffio_open2_wrapper, it's not being used anymore. Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
* avformat/aviobuf: Delay buffer downsizing until asserts are metMichael Niedermayer2019-06-30
| | | | | | | | | Fixes: Assertion failure Fixes: 15151/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5757079496687616 Fixes: 15205/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5767573242642432 May fix: Ticket7094 Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* lavf/aviobuf: Increase Statistics verbosity to AV_LOG_VERBOSE.Carl Eugen Hoyos2018-06-16
|
* lavf/aviobuf: add ff_get_chomp_lineJun Zhao2018-04-12
| | | | | | | Same as ff_get_line but strip the white-space characters in the string tail. Signed-off-by: Jun Zhao <mypopydev@gmail.com>
* avformat/aviobuf: add ff_read_line_to_bprint and ↵Marton Balint2018-02-24
| | | | | | | | ff_read_line_to_bprint_overwrite functions To be able to read lines longer than a static buffer size. Signed-off-by: Marton Balint <cus@passwd.hu>
* avformat/aviobuf: zero initialize the AVIOContext in ffio_init_context()James Almer2018-02-14
| | | | | | | | This makes sure no field is ever used uninitialized. Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com> Reviewed-by: wm4 <nfxjfg@googlemail.com> Signed-off-by: James Almer <jamrial@gmail.com>
* libavformat/avio: Utility function to return URLContextJeyapal, Karthick2017-11-29
|
* lavf/aviobuf: return EINVAL when reading from a write-only context.Nicolas George2017-10-29
| | | | Signed-off-by: Nicolas George <george@nsup.org>
* lavf/avio: temporarily accept 0 as EOF.Nicolas George2017-10-29
| | | | | | | | Print a warning to let applicatios fix their use. After a deprecation period, check with a low-level assert. Also make the constraint explicit in the doxygen comment. Signed-off-by: Nicolas George <george@nsup.org>
* avformat: Drop deprecated feof() AVIO fuctionJames Almer2017-10-21
| | | | Deprecated in 08/2014.
* libavformat: not treat 0 as EOFDaniel Kucera2017-10-19
| | | | | | | | | | transfer_func variable passed to retry_transfer_wrapper are h->prot->url_read and h->prot->url_write functions. These need to return EOF or other error properly. In case of returning >= 0, url_read/url_write is retried until error is returned. Signed-off-by: Daniel Kucera <daniel.kucera@gmail.com>
* lavf: make avio_read_partial() publicwm42017-09-01
| | | | | | | | | Main use-case is proxying avio through a foreign I/O layer and a custom AVIO context, without losing latency and performance characteristics. Signed-off-by: Luca Barbato <lu_zero@gentoo.org> Merged from Libav commit 173b56218f39c64.
* avio: add a destructor for AVIOContextAnton Khirnov2017-09-01
| | | | | | | | Before this commit, AVIOContext is to be freed with a plain av_free(), which prevents us from adding any deeper structure to it. (cherry picked from commit 99684f3ae752fc8bfb44a2dd1482f8d7a3d8536d) Signed-off-by: James Almer <jamrial@gmail.com>
* avformat/aviobuf: Fix signed integer overflow in avio_seek()Vitaly Buka2017-08-23
| | | | | | | | Signed integer overflow is undefined behavior. Detected with clang and -fsanitize=signed-integer-overflow Signed-off-by: Vitaly Buka <vitalybuka@google.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avformat/aviobuf: add support for specifying minimum packet size and marking ↵Marton Balint2017-06-24
| | | | | | | flush points Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Marton Balint <cus@passwd.hu>
* avformat/aviobuf: fix flushing write buffers after seeking backward or forwardMarton Balint2017-06-24
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch makes aviobuf work more like traditinal file IO, which is how people think about it. For example, in the past, aviobuf only flushed buffers until the current buffer position, even if more data was written to it previously, and a backward seek was used to reposition the IO context. From now, aviobuf will keep track of the written data, so no explicit seek will be required till the end of the buffer, or till the end of file before flushing. This fixes at least one regression, fate-vsynth3-flv was broken if flush_packets option was set to false, an explicit seek was removed in 4e3cc4bdd8acedbcc703607ed0efbb64bb5c3cc4. Also from now on, if a forward seek in the write buffer were to cause a gap between the already written data and the new file position, a flush will happen. The must_flush varable is also removed, which might have caused needless flushes with multiple seeks whithin the write buffer. Since we know the amount of data written to it, we will know when to flush. Signed-off-by: Marton Balint <cus@passwd.hu>
* avformat/aviobuf: Only downscale the buffer once it has been usedMichael Niedermayer2017-05-19
| | | | | | The code mistook the first iteration sometimes as the end Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* Merge commit '3f75e5116b900f1428aa13041fc7d6301bf1988a'James Almer2017-04-13
|\ | | | | | | | | | | | | * commit '3f75e5116b900f1428aa13041fc7d6301bf1988a': avio: Keep track of the amount of data written Merged-by: James Almer <jamrial@gmail.com>
| * avio: Keep track of the amount of data writtenLuca Barbato2016-12-08
| | | | | | | | Make avio_size() work with any write AVIOContext.
* | Merge commit '8ea35af7620e4f73f9e8c072e1c0fac9a04ec161'James Almer2017-03-21
|\| | | | | | | | | | | | | * commit '8ea35af7620e4f73f9e8c072e1c0fac9a04ec161': avio: add a new flag for marking streams seekable by timestamp Merged-by: James Almer <jamrial@gmail.com>
| * avio: add a new flag for marking streams seekable by timestampAnton Khirnov2016-09-30
| |
* | Merge commit '83548fe894cdb455cc127f754d09905b6d23c173'James Almer2017-03-21
|\| | | | | | | | | | | | | * commit '83548fe894cdb455cc127f754d09905b6d23c173': lavf: fix usage of AVIOContext.seekable Merged-by: James Almer <jamrial@gmail.com>
| * lavf: fix usage of AVIOContext.seekableAnton Khirnov2016-09-30
| | | | | | | | | | | | | | | | It is supposed to be a flag. The only currently defined value is AVIO_SEEKABLE_NORMAL, but other ones may be added in the future. However all the current lavf code treats this field as a bool (mainly for historical reasons). Change all those cases to properly check for AVIO_SEEKABLE_NORMAL.
* | HTTP: improve performance by reducing forward seeksJoel Cunningham2017-02-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit optimizes HTTP performance by reducing forward seeks, instead favoring a read-ahead and discard on the current connection (referred to as a short seek) for seeks that are within a TCP window's worth of data. This improves performance because with TCP flow control, a window's worth of data will be in the local socket buffer already or in-flight from the sender once congestion control on the sender is fully utilizing the window. Note: this approach doesn't attempt to differentiate from a newly opened connection which may not be fully utilizing the window due to congestion control vs one that is. The receiver can't get at this information, so we assume worst case; that full window is in use (we did advertise it after all) and that data could be in-flight The previous behavior of closing the connection, then opening a new with a new HTTP range value results in a massive amounts of discarded and re-sent data when large TCP windows are used. This has been observed on MacOS/iOS which starts with an initial window of 256KB and grows up to 1MB depending on the bandwidth-product delay. When seeking within a window's worth of data and we close the connection, then open a new one within the same window's worth of data, we discard from the current offset till the end of the window. Then on the new connection the server ends up re-sending the previous data from new offset till the end of old window. Example (assumes full window utilization): TCP window size: 64KB Position: 32KB Forward seek position: 40KB * (Next window) 32KB |--------------| 96KB |---------------| 160KB * 40KB |---------------| 104KB Re-sent amount: 96KB - 40KB = 56KB For a real world test example, I have MP4 file of ~25MB, which ffplay only reads ~16MB and performs 177 seeks. With current ffmpeg, this results in 177 HTTP GETs and ~73MB worth of TCP data communication. With this patch, ffmpeg issues 4 HTTP GETs and 3 seeks for a total of ~22MB of TCP data communication. To support this feature, the short seek logic in avio_seek() has been extended to call a function to get the short seek threshold value. This callback has been plumbed to the URLProtocol structure, which now has infrastructure in HTTP and TCP to get the underlying receiver window size via SO_RCVBUF. If the underlying URL and protocol don't support returning a short seek threshold, the default s->short_seek_threshold is used This feature has been tested on Windows 7 and MacOS/iOS. Windows support is slightly complicated by the fact that when TCP window auto-tuning is enabled, SO_RCVBUF doesn't report the real window size, but it does if SO_RCVBUF was manually set (disabling auto-tuning). So we can only use this optimization on Windows in the later case Signed-off-by: Joel Cunningham <joel.cunningham@me.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* | libavformat/avio: Add avio_get_dyn_buf functionsoftworkz2017-01-07
| | | | | | | | | | | | | | | | | | | | | | | | | | This commit adds the avio_get_dyn_buf function which allows accessing the content of a DynBuffer without destroying it. This is required in matroskaenc for preliminary writing (correct) mkv headers. Context for this change is fixing regression bug #5977. Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* | lavf/aviobuf.c: Adapt avio_accept and avio_handshake to new AVIOContext APIStephan Holljes2016-10-03
| | | | | | | | Signed-off-by: Stephan Holljes <klaxa1337@googlemail.com>
* | avformat/avio: add a crc callback function for CRC-32 IEEE-LEJames Almer2016-08-04
| | | | | | | | | | | | Reviewed-by: Nicolas George <george@nsup.org> Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: James Almer <jamrial@gmail.com>
* | cosmetics: fix some misspelled wordsJames Almer2016-07-17
| | | | | | | | Signed-off-by: James Almer <jamrial@gmail.com>
* | Merge commit 'db7968bff4851c2be79b15b2cb2ae747424d2fca'Matthieu Bouron2016-06-23
|\| | | | | | | | | | | | | * commit 'db7968bff4851c2be79b15b2cb2ae747424d2fca': avio: Allow custom IO users to get labels for the output bytestream Merged-by: Matthieu Bouron <matthieu.bouron@stupeflix.com>
| * avio: Allow custom IO users to get labels for the output bytestreamMartin Storsjö2016-05-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows callers with avio write callbacks to get the bytestream positions that correspond to keyframes, suitable for live streaming. In the simplest form, a caller could expect that a header is written to the bytestream during the avformat_write_header, and the data output to the avio context during e.g. av_write_frame corresponds exactly to the current packet passed in. When combined with av_interleaved_write_frame, and with muxers that do buffering (most muxers that do some sort of fragmenting or clustering), the mapping from input data to bytestream positions is nontrivial. This allows callers to get directly information about what part of the bytestream is what, without having to resort to assumptions about the muxer behaviour. One keyframe/fragment/block can still be split into multiple (if they are larger than the aviocontext buffer), which would call the callback with e.g. AVIO_DATA_MARKER_SYNC_POINT, followed by AVIO_DATA_MARKER_UNKNOWN for the second time it is called with the following data. Signed-off-by: Martin Storsjö <martin@martin.st>
* | Merge commit 'fab8156b2f30666adabe227b3d7712fd193873b1'Derek Buitenhuis2016-04-21
|\| | | | | | | | | | | | | * commit 'fab8156b2f30666adabe227b3d7712fd193873b1': avio: Copy URLContext generic options into child URLContexts Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
| * avio: Copy URLContext generic options into child URLContextsMartin Storsjö2016-03-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since all URLContexts have the same AVOptions, such AVOptions will be applied on the outermost context only and removed from the dict, while they probably make sense on all contexts. This makes sure that rw_timeout gets propagated to the innermost URLContext (to make sure it gets passed to the tcp protocol, when opening a http connection for instance). Alternatively, such matching options would be kept in the dict and only removed after the ffurl_connect call. Signed-off-by: Martin Storsjö <martin@martin.st>
| * lavf: add a protocol whitelist/blacklist for file opened internallyAnton Khirnov2016-02-22
| | | | | | | | | | | | | | | | Should make the default behaviour safer for careless callers that open random untrusted files. Bug-Id: CVE-2016-1897 Bug-Id: CVE-2016-1898
| * urlprotocol: receive a list of protocols from the callerAnton Khirnov2016-02-22
| | | | | | | | | | This way, the decisions about which protocols are available for use in any given situations can be delegated to the caller.
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-16 21:37:16 +0000