mem: Consistently return NULL for av_malloc(0)

Plain POSIX malloc(0) is allowed to return either NULL or a
non-NULL pointer. The calling code should be ready to handle
a NULL return as a correct return (instead of a failure) if the size
to allocate was 0 - this makes sure the condition is handled
in a consistent way across platforms.

This also avoids calling posix_memalign(&ptr, 32, 0) on OS X,
which returns an invalid pointer (a non-NULL pointer that causes
crashes when passed to av_free).

Abort in debug mode, to help track down issues related to
incorrect handling of this case.

Signed-off-by: Martin Storsjö <martin@martin.st>

1 files changed, 3 insertions, 1 deletions

diff --git a/libavutil/mem.c b/libavutil/mem.c
index b6230cf0e3..bf1a542db8 100644
--- a/libavutil/mem.c
+++ b/libavutil/mem.c
@@ -68,8 +68,10 @@ void *av_malloc(size_t size)
     long diff;
 #endif
 
+    assert(size);
+
     /* let's disallow possible ambiguous cases */
-    if(size > (INT_MAX-32) )
+    if (size > (INT_MAX-32) || !size)
         return NULL;
 
 #if CONFIG_MEMALIGN_HACK