yolo v5 使用--batch-size 大于1时量化报错

ICpachong LV 5

在量化yolo v5模型时使用命令：
pegasus quantize --model yolov5s-sim.json --model-data yolov5s-sim.data --batch-size 5 --device CPU --with-input-meta yolov5s-sim_inputmeta.yml --rebuild --model-quantize yolov5s-sim.quantize --quantizer asymmetric_affine --qtype uint8
dataset.txt的配置和图片的位置都是正确的，当--batch-size 为1是成功的，但是当--batch-size 为5或其他大于1的值时会报如下错误，请各位大佬帮忙看看：

I Build torch-jit-export complete.
D *********** Setup database (1) ***********
D Setup dataset "TEXT":
D Lids: ['images_208']
D Layouts: ['nchw']
D Shapes: [[5, 3, 640, 640]]
D Data types: ['float32']
D Sparse tensors: []
D Tensor names(H5FS only): []
W:tensorflow:AutoGraph could not transform <acuitylib.core.execution.BackendFunction object at 0x7f5bcde20ee0> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: Unable to locate the source code of <function BackendFunction.__call__ at 0x7f5bced43670>. Note that functions defined in certain environments, like the interactive Python shell do not expose their source code. If that is the case, you should to define them in a .py source file. If you are certain the code is graph-compatible, wrap the call using @tf.autograph.do_not_convert. Original error: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
Traceback (most recent call last):
  File "tensorflow/python/autograph/pyct/parser.py", line 204, in parse_entity
  File "tensorflow/python/autograph/pyct/inspect_utils.py", line 147, in getimmediatesource
  File "inspect.py", line 798, in findsource
OSError: could not get source code

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tensorflow/python/autograph/impl/api.py", line 584, in converted_call
  File "tensorflow/python/autograph/impl/conversion.py", line 118, in convert
  File "tensorflow/python/autograph/pyct/transpiler.py", line 411, in transform_function
  File "tensorflow/python/autograph/pyct/transpiler.py", line 373, in _transformed_factory
  File "tensorflow/python/autograph/pyct/transpiler.py", line 303, in _transform_function
  File "tensorflow/python/autograph/pyct/parser.py", line 206, in parse_entity
ValueError: Unable to locate the source code of <function BackendFunction.__call__ at 0x7f5bced43670>. Note that functions defined in certain environments, like the interactive Python shell do not expose their source code. If that is the case, you should to define them in a .py source file. If you are certain the code is graph-compatible, wrap the call using @tf.autograph.do_not_convert. Original error: could not get source code

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tensorflow/python/framework/ops.py", line 1812, in _create_c_op
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot reshape a tensor with 8160000 elements to shape [1,3,85,80,80] (1632000 elements) for '{{node Reshape_Reshape_213_8}} = Reshape[T=DT_FLOAT, Tshape=DT_INT32](Reshape_Reshape_213_8_acuity_mark_perm_211, Reshape_Reshape_213_8/shape)' with input shapes: [5,255,80,80], [5] and with input tensors computed as partial shapes: input[1] = [1,3,85,80,80].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "pegasus.py", line 131, in <module>
  File "pegasus.py", line 108, in main
  File "acuitylib/app/medusa/commands.py", line 212, in execute
  File "acuitylib/vsi_nn.py", line 589, in quantize
  File "acuitylib/app/medusa/quantization.py", line 151, in run
  File "acuitylib/app/medusa/quantization.py", line 54, in _run_quantization
  File "acuitylib/app/medusa/workspace.py", line 120, in run
  File "acuitylib/app/medusa/workspace.py", line 92, in _setup_graph
  File "acuitylib/acuitynetbuilder.py", line 83, in build_normal_graph
  File "acuitylib/acuitynetbuilder.py", line 69, in _build_backend_execution
  File "acuitylib/core/execution.py", line 557, in __init__
  File "tensorflow/python/eager/def_function.py", line 1167, in get_concrete_function
  File "tensorflow/python/eager/def_function.py", line 1073, in _get_concrete_function_garbage_collected
  File "tensorflow/python/eager/def_function.py", line 696, in _initialize
  File "tensorflow/python/eager/function.py", line 2855, in _get_concrete_function_internal_garbage_collected
  File "tensorflow/python/eager/function.py", line 3213, in _maybe_define_function
  File "tensorflow/python/eager/function.py", line 3065, in _create_graph_function
  File "tensorflow/python/framework/func_graph.py", line 986, in func_graph_from_py_func
  File "tensorflow/python/eager/def_function.py", line 600, in wrapped_fn
  File "tensorflow/python/framework/func_graph.py", line 962, in wrapper
  File "tensorflow/python/autograph/impl/api.py", line 591, in converted_call
  File "tensorflow/python/autograph/impl/api.py", line 398, in _fall_back_unconverted
  File "tensorflow/python/autograph/impl/api.py", line 339, in _call_unconverted
  File "acuitylib/core/execution.py", line 413, in __call__
  File "acuitylib/core/execution.py", line 409, in _build_backend_operations
  File "acuitylib/core/execution.py", line 401, in _build_layer_backend
  File "acuitylib/core/execution.py", line 19, in __call__
  File "acuitylib/layer/reshapelayer.py", line 89, in compute_out_tensor
  File "tensorflow/python/util/dispatch.py", line 201, in wrapper
  File "tensorflow/python/ops/array_ops.py", line 195, in reshape
  File "tensorflow/python/ops/gen_array_ops.py", line 8233, in reshape
  File "tensorflow/python/framework/op_def_library.py", line 742, in _apply_op_helper
  File "tensorflow/python/framework/func_graph.py", line 591, in _create_op_internal
  File "tensorflow/python/framework/ops.py", line 3477, in _create_op_internal
  File "tensorflow/python/framework/ops.py", line 1974, in __init__
  File "tensorflow/python/framework/ops.py", line 1815, in _create_c_op
ValueError: Cannot reshape a tensor with 8160000 elements to shape [1,3,85,80,80] (1632000 elements) for '{{node Reshape_Reshape_213_8}} = Reshape[T=DT_FLOAT, Tshape=DT_INT32](Reshape_Reshape_213_8_acuity_mark_perm_211, Reshape_Reshape_213_8/shape)' with input shapes: [5,255,80,80], [5] and with input tensors computed as partial shapes: input[1] = [1,3,85,80,80].
[107616] Failed to execute script 'pegasus' due to unhandled exception!

柚木鉉 LV 9

固定为1就行了，这个batch_size是模型定的。如果原始网络使用固定的batch_size，请使用固定的batch_size，如果原始网络使用可变batch_size，请将此参数设置为1

ICpachong LV 5

@yuzukitsuru 感谢回答！
如果在量化和预推理时循环多次的话，需要加入--iterations 6参数