summaryrefslogtreecommitdiff
path: root/libavfilter/dnn/dnn_backend_native.c
Commit message (Collapse)AuthorAge
* libavfilter: Remove DNNReturnType from DNN ModuleShubhanshu Saxena2022-03-12
| | | | | | | | | | This patch removes all occurences of DNNReturnType from the DNN module. This commit replaces DNN_SUCCESS by 0 (essentially the same), so the functions with DNNReturnType now return 0 in case of success, the negative values otherwise. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com> Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn_backend_native: Return Specific Error CodesShubhanshu Saxena2022-03-12
| | | | | | | Switch to returning specific error codes or DNN_GENERIC_ERROR when an error is encountered. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: Rename InferenceItem to LastLevelTaskItemShubhanshu Saxena2021-08-28
| | | | | | | | | | | | | | | This patch renames the InferenceItem to LastLevelTaskItem in the three backends to avoid confusion among the meanings of these structs. The following are the renames done in this patch: 1. extract_inference_from_task -> extract_lltask_from_task 2. InferenceItem -> LastLevelTaskItem 3. inference_queue -> lltask_queue 4. inference -> lltask 5. inference_count -> lltask_count Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* libavfilter: Unify Execution Modes in DNN FiltersShubhanshu Saxena2021-08-28
| | | | | | | | | | | | | | | | | | | | | | | This commit unifies the async and sync mode from the DNN filters' perspective. As of this commit, the Native backend only supports synchronous execution mode. Now the user can switch between async and sync mode by using the 'async' option in the backend_configs. The values can be 1 for async and 0 for sync mode of execution. This commit affects the following filters: 1. vf_dnn_classify 2. vf_dnn_detect 3. vf_dnn_processing 4. vf_sr 5. vf_derain This commit also updates the filters vf_dnn_detect and vf_dnn_classify to send only the input frame and send NULL as output frame instead of input frame to the DNN backends. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: Task-based Inference in Native BackendShubhanshu Saxena2021-08-28
| | | | | | | This commit rearranges the code in Native Backend to use the TaskItem for inference. Signed-off-by: Shubhanshu Saxena <shubhanshu.e01@gmail.com>
* lavfi/dnn: refine code to separate processing and detection in backendsGuo, Yejun2021-05-24
|
* lavfi/dnn: refine dnn interface to add DNNExecBaseParamsGuo, Yejun2021-05-06
| | | | | | | | | | | | | Different function type of model requires different parameters, for example, object detection detects lots of objects (cat/dog/...) in the frame, and classifcation needs to know which object (cat or dog) it is going to classify. The current interface needs to add a new function with more parameters to support new requirement, with this change, we can just add a new struct (for example DNNExecClassifyParams) based on DNNExecBaseParams, and so we can continue to use the current interface execute_model just with params changed.
* lavfi/dnn: refine code for frame pre/proc processingGuo, Yejun2021-04-08
|
* dnn/dnn_backend_native: Fix typo in log messageAndreas Rheinhardt2021-03-11
| | | | | Reviewed-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* dnn/dnn_backend_native: Don't use asserts for checksAndreas Rheinhardt2021-03-11
| | | | | | | | | | asserts should not be used instead of ordinary input checks. Yet the native DNN backend did it: get_input_native() asserted that the first dimension was one, despite this value coming directly from the input file without having been sanitized. Reviewed-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* dnn/dnn_backend_native: Fix leak in case parsing options failsAndreas Rheinhardt2021-03-11
| | | | | Reviewed-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* dnn/dnn_backend_native: Avoid allocation for checking file magicAndreas Rheinhardt2021-03-11
| | | | | Reviewed-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* dnn: add color conversion for analytic caseGuo, Yejun2021-02-18
| | | | Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
* dnn: add function type for modelGuo, Yejun2021-02-18
| | | | | | | | So the backend knows the usage of model is for frame processing, detect, classify, etc. Each function type has different behavior in backend when handling the input/output data of the model. Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
* dnn: remove type cast which is not necessaryGuo, Yejun2021-01-28
|
* dnn: Add ff_ prefix to unnamespaced globalsMark Thompson2021-01-22
| | | | Reviewed-By: Guo, Yejun <yejun.guo@intel.com>
* dnn_backend_native.c: Add missing static to local variableMark Thompson2021-01-22
|
* libavfilter/dnn: add prefix ff_ for internal functionsGuo, Yejun2021-01-22
| | | | | | | from proc_from_frame_to_dnn to ff_proc_from_frame_to_dnn, and from proc_from_dnn_to_frame to ff_proc_from_dnn_to_frame. Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
* libavfilter/dnn: use avpriv_report_missing_feature for unsupported featuresGuo, Yejun2021-01-22
| | | | Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
* dnn_interface: change from 'void *userdata' to 'AVFilterContext *filter_ctx'Guo, Yejun2020-12-29
| | | | | | | | | | 'void *' is too flexible, since we can derive info from AVFilterContext*, so we just unify the interface with this data structure. Signed-off-by: Xie, Lin <lin.xie@intel.com> Signed-off-by: Wu Zhiwen <zhiwen.wu@intel.com> Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
* libavfilter/dnn_backend_native: check mem allocationChris Miceli2020-10-14
| | | | check that frame allocations return non-null.
* dnn: add a new interface DNNModel.get_outputGuo, Yejun2020-09-21
| | | | | | | | | | for some cases (for example, super resolution), the DNN model changes the frame size which impacts the filter behavior, so the filter needs to know the out frame size at very beginning. Currently, the filter reuses DNNModule.execute_model to query the out frame size, it is not clear from interface perspective, so add a new explict interface DNNModel.get_output for such query.
* dnn: put DNNModel.set_input and DNNModule.execute_model togetherGuo, Yejun2020-09-21
| | | | | | | | | | | | | | suppose we have a detect and classify filter in the future, the detect filter generates some bounding boxes (BBox) as AVFrame sidedata, and the classify filter executes DNN model for each BBox. For each BBox, we need to crop the AVFrame, copy data to DNN model input and do the model execution. So we have to save the in_frame at DNNModel.set_input and use it at DNNModule.execute_model, such saving is not feasible when we support async execute_model. This patch sets the in_frame as execution_model parameter, and so all the information are put together within the same function for each inference. It also makes easy to support BBox async inference.
* dnn: change dnn interface to replace DNNData* with AVFrame*Guo, Yejun2020-09-21
| | | | | | | | | | | | Currently, every filter needs to provide code to transfer data from AVFrame* to model input (DNNData*), and also from model output (DNNData*) to AVFrame*. Actually, such transfer can be implemented within DNN module, and so filter can focus on its own business logic. DNN module also exports the function pointer pre_proc and post_proc in struct DNNModel, just in case that a filter has its special logic to transfer data between AVFrame* and DNNData*. The default implementation within DNN module is used if the filter does not set pre/post_proc.
* dnn: add userdata for load model parameterGuo, Yejun2020-09-21
| | | | the userdata will be used for the interaction between AVFrame and DNNData
* dnn_backend_native.c: parse options in native backendXu Jun2020-09-09
| | | | | Signed-off-by: Xu Jun <xujunzz@sjtu.edu.cn> Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
* dnn/native: add log error messageTing Fu2020-08-25
| | | | Signed-off-by: Ting Fu <ting.fu@intel.com>
* dnn/native: unify error return to DNN_ERRORTing Fu2020-08-25
| | | | | | | Unify all error return as DNN_ERROR, in order to cease model executing when return error in ff_dnn_execute_model_native layer_func.pf_exec Signed-off-by: Ting Fu <ting.fu@intel.com>
* dnn: move output name from DNNModel.set_input_output to DNNModule.execute_modelGuo, Yejun2020-08-25
| | | | | | | | | | | currently, output is set both at DNNModel.set_input_output and DNNModule.execute_model, it makes sense that the output name is provided at model inference time so all the output info is set at a single place. and so DNNModel.set_input_output is renamed to DNNModel.set_input Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
* dnn/native: rename struct ConvolutionalNetwork to NativeModelTing Fu2020-08-21
| | | | | Signed-off-by: Ting Fu <ting.fu@intel.com> Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
* dnn: add backend options when load the modelGuo, Yejun2020-08-12
| | | | | | | different backend might need different options for a better performance, so, add the parameter into dnn interface, as a preparation. Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
* dnn_backend_native: Add overflow check for length calculation.Reimar Döffinger2020-07-06
| | | | | | | | | We should not silently allocate an incorrect sized buffer. Fixes trac issue #8718. Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de> Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
* dnn_backend_native: check operand indexGuo Yejun2020-06-17
| | | | it fixed the issue in https://trac.ffmpeg.org/ticket/8716
* dnn_backend_native.c: refine code for fail caseGuo Yejun2020-06-17
|
* avfilter/dnn: add a new interface to query dnn model's input infoGuo, Yejun2019-10-30
| | | | | | | | | | | | to support dnn networks more general, we need to know the input info of the dnn model. background: The data type of dnn model's input could be float32, uint8 or fp16, etc. And the w/h of input image could be fixed or variable. Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* avfilter/dnn: get the data type of network output from dnn execution resultGuo, Yejun2019-10-30
| | | | | | | | | | | so, we can make a filter more general to accept different network models, by adding a data type convertion after getting data from network. After we add dt field into struct DNNData, it becomes the same as DNNInputData, so merge them with one struct: DNNData. Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* dnn: add tf.nn.conv2d support for native modelGuo, Yejun2019-10-30
| | | | | | | | | | | | | Unlike other tf.*.conv2d layers, tf.nn.conv2d does not create many nodes (within a scope) in the graph, it just acts like other layers. tf.nn.conv2d only creates one node in the graph, and no internal nodes such as 'kernel' are created. The format of native model file is also changed, a flag named has_bias is added, so change the version number. Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* avfilter/dnn: unify the layer load function in native modeGuo, Yejun2019-10-15
| | | | | Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* avfilter/dnn: unify the layer execution function in native modeGuo, Yejun2019-10-15
| | | | | Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* avfilter/dnn: add DLT prefix for enum DNNLayerType to avoid potential conflictsGuo, Yejun2019-10-15
| | | | | | | and also change CONV to DLT_CONV2D for better description Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* libavfilter/dnn: support multiple outputs for native modeGuo, Yejun2019-09-20
| | | | | Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* libavfilter/dnn/dnn_backend_native: find the input operand according to ↵Guo, Yejun2019-09-20
| | | | | | | input name Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* libavfilter/dnn: add layer maximum for native mode.Guo, Yejun2019-09-20
| | | | | | | | | The reason to add this layer is that it is used by srcnn in vf_sr. This layer is currently ignored in native mode. After this patch, we can add multiple outputs support for native mode. Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* libavfilter/dnn: separate depth_to_space layer from dnn_backend_native.c to ↵Guo, Yejun2019-09-19
| | | | | | | | | | a new file the logic is that one layer in one separated source file to make the source files simple for maintaining. Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* libavfilter/dnn: separate conv2d layer from dnn_backend_native.c to a new fileGuo, Yejun2019-09-19
| | | | | | | | the logic is that one layer in one separated source file to make the source files simple for maintaining. Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* libavfilter/dnn: add header into native model fileGuo, Yejun2019-09-04
| | | | | Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* dnn: export operand info in python script and load in c codeGuo, Yejun2019-08-30
| | | | | Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* dnn: change .model file format to put layer number at the end of fileGuo, Yejun2019-08-30
| | | | | | | | | | currently, the layer number is at the beginning of the .model file, so we have to scan twice in python script, the first scan to get the layer number. Only one scan needed after put the layer number at the end of .model file. Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* dnn: introduce dnn operand (in c code) to hold operand infos within networkGuo, Yejun2019-08-30
| | | | | | | | | | | the info can be saved in dnn operand object without regenerating again and again, and it is also needed for layer split/merge, and for memory reuse. to make things step by step, this patch just focuses on c code, the change within python script will be added later. Signed-off-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
* lavfi/dnn/dnn_backend_native: fix memory leak in error pathJun Zhao2019-08-20
| | | | | | | fix memory leak in error path Reviewed-by: Guo, Yejun <yejun.guo@intel.com> Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-15 18:50:10 +0000