summaryrefslogtreecommitdiff
path: root/libavfilter/dnn
Commit message (Collapse)AuthorAge
* Replace all occurences of av_mallocz_array() by av_calloc()Andreas Rheinhardt2021-09-20
| | | | | | | They do the same. Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
* lavfi/dnn: Rename InferenceItem to LastLevelTaskItemShubhanshu Saxena2021-08-28
| | | | | | | | | | | | | | | This patch renames the InferenceItem to LastLevelTaskItem in the three backends to avoid confusion among the meanings of these structs. The following are the renames done in this patch: 1. extract_inference_from_task -> extract_lltask_from_task 2. InferenceItem -> LastLevelTaskItem 3. inference_queue -> lltask_queue 4. inference -> lltask 5. inference_count -> lltask_count Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* libavfilter: Remove Async Flag from DNN Filter SideShubhanshu Saxena2021-08-28
| | | | | | | Remove async flag from filter's perspective after the unification of async and sync modes in the DNN backend. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* libavfilter: Unify Execution Modes in DNN FiltersShubhanshu Saxena2021-08-28
| | | | | | | | | | | | | | | | | | | | | | | This commit unifies the async and sync mode from the DNN filters' perspective. As of this commit, the Native backend only supports synchronous execution mode. Now the user can switch between async and sync mode by using the 'async' option in the backend_configs. The values can be 1 for async and 0 for sync mode of execution. This commit affects the following filters: 1. vf_dnn_classify 2. vf_dnn_detect 3. vf_dnn_processing 4. vf_sr 5. vf_derain This commit also updates the filters vf_dnn_detect and vf_dnn_classify to send only the input frame and send NULL as output frame instead of input frame to the DNN backends. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: Task-based Inference in Native BackendShubhanshu Saxena2021-08-28
| | | | | | | This commit rearranges the code in Native Backend to use the TaskItem for inference. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: DNNAsyncExecModule Execution Failure HandlingShubhanshu Saxena2021-08-10
| | | | | | | | | This commit adds the case handling if the asynchronous execution of a request fails by checking the exit status of the thread when joining before starting another execution. On failure, it does the cleanup as well. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_tf: Error Handling for tf_create_inference_requestShubhanshu Saxena2021-08-10
| | | | | | | This commit includes the check for the case when the newly created TFInferRequest is NULL. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: Extract Common Parts from get_output functionsShubhanshu Saxena2021-08-10
| | | | | | | | The frame allocation and filling the TaskItem with execution parameters is common in the three backends. This commit shifts this logic to dnn_backend_common. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_tf: Add TF_Status to TFRequestItemShubhanshu Saxena2021-08-10
| | | | | | | | | Since requests are running in parallel, there is inconsistency in the status of the execution. To resolve it, we avoid using mutex as it would result in single TF_Session running at a time. So add TF_Status to the TFRequestItem Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_tf: Error Handling for execute_model_tfShubhanshu Saxena2021-08-10
| | | | | | | | This patch adds error handling for cases where the execute_model_tf fails, clears the used memory in the TFRequestItem and finally pushes it back to the request queue. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: Async Support for TensorFlow BackendShubhanshu Saxena2021-08-10
| | | | | | | | | | | | | | | | | | | This commit enables async execution in the TensorFlow backend and adds function to flush extra frames. The async execution mechanism executes the TFInferRequests on a separate thread which is joined before the next execution of same TFRequestItem/while freeing the model. The following is the comparison of this mechanism with the existing sync mechanism on TensorFlow C API 2.5 CPU variant. Async Mode: 4m32.846s Sync Mode: 5m17.582s The above was performed on super resolution filter using SRCNN model. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_tf: TFInferRequest Execution and DocumentationShubhanshu Saxena2021-08-10
| | | | | | | This commit adds a function for execution of TFInferRequest and documentation for functions related to TFInferRequest. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: Common Function to Get Async Result in DNN BackendsShubhanshu Saxena2021-08-10
| | | | | | | This commits refactors the get async result function for common use in all three backends. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: Add Async Execution Mechanism and DocumentationShubhanshu Saxena2021-08-10
| | | | | | | | | This commit adds an async execution mechanism for common use in the TensorFlow and Native backends. This commit also adds the documentation of typedefs and functions in the async module for common use in DNN backends. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* avfilter/internal: Don't include libavcodec/(avcodec|internal).hAndreas Rheinhardt2021-08-05
| | | | | | | | | The reasons for including them don't exist any longer: ff_tlog() has been moved to libavutil/internal.h and FF_QSCALE_TYPE_* has been moved to qp_table.h. Reviewed-by: Nicolas George <george@nsup.org> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
* Remove unnecessary avassert.h inclusionsAndreas Rheinhardt2021-07-22
| | | | Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
* avcodec/avcodec: Don't include cpu.hAndreas Rheinhardt2021-07-22
| | | | | | | It is not used here at all; instead, add it where it is used without including it or any of the arch-specific CPU headers. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
* lavfi/dnn_backend_ov: Rename RequestItem to OVRequestItemShubhanshu Saxena2021-07-22
| | | | | | | Rename RequestItem to OVRequestItem in the OpenVINO backend to avoid confusion. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_openvino.c: Fix Memory Leak in execute_model_ovShubhanshu Saxena2021-07-22
| | | | | | | | In cases where the execution inside the function execute_model_ov fails, the OVRequestItem must be pushed back to the request_queue before returning the error. In case pushing back fails, release the allocated memory. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_tf: Error HandlingShubhanshu Saxena2021-07-11
| | | | | | | This commit adds handling for cases where an error may occur, clearing the allocated memory resources. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_tf: Separate function for Completion CallbackShubhanshu Saxena2021-07-11
| | | | | | | This commit rearranges the existing code to create a separate function for the completion callback in execute_model_tf. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_tf: Separate function for filling RequestItemShubhanshu Saxena2021-07-11
| | | | | | | This commit rearranges the existing code to create separate function for filling request with execution data. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_tf: Request-based ExecutionShubhanshu Saxena2021-07-11
| | | | | | | | This commit uses TFRequestItem and the existing sync execution mechanism to use request-based execution. It will help in adding async functionality to the TensorFlow backend later. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_tf: Add TFInferRequest and TFRequestItemShubhanshu Saxena2021-07-11
| | | | | | | | This commit introduces a typedef TFInferRequest to store execution parameters for a single call to the TensorFlow C API. This typedef is used in the TFRequestItem. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_tf: TaskItem Based InferenceShubhanshu Saxena2021-07-11
| | | | | | | This commit uses the common TaskItem and InferenceItem typedefs for execution in TensorFlow backend. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_openvino.c: Fix Memory Leak in execute_model_ovShubhanshu Saxena2021-07-04
| | | | | | | | In cases where the execution inside the function execute_model_ov fails, push the RequestItem back to the request_queue before returning the error. In case pushing back fails, release the allocated memory. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_openvino.c: fix crash when target is not specifiedGuo Yejun2021-06-19
|
* lavfi/dnn_backend_openvino.c: Fix Memory Leak for RequestItemShubhanshu Saxena2021-06-18
| | | | | | | Fix memory leak for RequestItem upon error while pushing to the request_queue in the completion callback. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: Fill Task using Common FunctionShubhanshu Saxena2021-06-12
| | | | | | | This commit adds a common function for filling the TaskItems in all three backends. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: Use uint8_t for async and do_ioproc in TaskItemsShubhanshu Saxena2021-06-12
| | | | | | | These properties have values either 0 or 1, so using uint8_t is a better option as compared to int. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: Add nb_output to TaskItemShubhanshu Saxena2021-06-12
| | | | | | | Add nb_output property to TaskItem for use in TensorFlow backend and Native backend. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: Convert output_name to char** in TaskItemShubhanshu Saxena2021-06-12
| | | | | | | Convert output_name to char **output_names in TaskItem and use it as a pointer to array of output names in the DNN backend. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: Extract TaskItem and InferenceItem from OpenVino BackendShubhanshu Saxena2021-06-12
| | | | | | | Extract TaskItem and InferenceItem from OpenVino backend and convert ov_model to void in TaskItem. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn/dnn_io_proc.c: fix CID 1484955Guo Yejun2021-06-05
| | | | CID 1484955: Memory - corruptions (ARRAY_VS_SINGLETON)
* lavfi/dnn_backend_openvino.c: Correct Pointer Type while FreeingShubhanshu Saxena2021-05-28
| | | | | | | This commit corrects the type of pointer of elements from the inference queue in ff_dnn_free_model_ov. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: refine code to separate processing and detection in backendsGuo, Yejun2021-05-24
|
* lavfi/dnn/dnn_io_proc.c: fix Improper use of negative value (NEGATIVE_RETURNS)Guo, Yejun2021-05-18
| | | | fix coverity CID 1473511 and 1473566
* lavfi/dnn/dnn_io_proc.c: Fix Out-of-bounds access (ARRAY_VS_SINGLETON)Guo, Yejun2021-05-18
| | | | fix coverity CID 1473571, 1473577 and 1482089
* lavfi/dnn_backend_native_layer_mathunary.h: DocumentationShubhanshu Saxena2021-05-17
| | | | | | Add documentation for Unary Math Layer Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_native_layer_depth2space.h: DocumentationShubhanshu Saxena2021-05-17
| | | | | | Add documentation for Depth to Space Layer Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_native_layer_dense.h: DocumentationShubhanshu Saxena2021-05-17
| | | | | | Add documentation for Dense Layer Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_native_layer_conv2d.h: DocumentationShubhanshu Saxena2021-05-17
| | | | | | Add documentation for 2D Convolution Layer Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_native_layer_avgpool.h: DocumentationShubhanshu Saxena2021-05-14
| | | | | | Add documentation for Average Pool Layer Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* avfilter/dnn/dnn_backend_tf: fix cross library usageLimin Wang2021-05-11
| | | | | | | | duplicate ff_hex_to_data() function from avformat and rename it to hex_to_data() as static function. Reviewed-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
* lavfi/dnn_backend_tensorflow: support detect modelTing Fu2021-05-11
| | | | Signed-off-by: Ting Fu <ting.fu@intel.com>
* lavfi/dnn_backend_tensorflow: add multiple outputs supportTing Fu2021-05-11
| | | | Signed-off-by: Ting Fu <ting.fu@intel.com>
* dnn: add DCO_RGB color order to enum DNNColorOrderTing Fu2021-05-11
| | | | | | | Adding DCO_RGB color order to DNNColorOrder, since tensorflow model needs this kind of color oder as input. Signed-off-by: Ting Fu <ting.fu@intel.com>
* lavfi/dnn: add classify support with openvino backendGuo, Yejun2021-05-06
| | | | Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
* lavfi/dnn: refine dnn interface to add DNNExecBaseParamsGuo, Yejun2021-05-06
| | | | | | | | | | | | | Different function type of model requires different parameters, for example, object detection detects lots of objects (cat/dog/...) in the frame, and classifcation needs to know which object (cat or dog) it is going to classify. The current interface needs to add a new function with more parameters to support new requirement, with this change, we can just add a new struct (for example DNNExecClassifyParams) based on DNNExecBaseParams, and so we can continue to use the current interface execute_model just with params changed.
* lavfi/dnn_backend_openvino.c: move the logic for batch mode earlierGuo, Yejun2021-05-06
|
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-29 21:48:57 +0000