I run yolort_trt for inference with c++, and this is a question about the file format specified by the --class_names argument. #445

leeyunhome · 2022-07-18T07:23:22Z

leeyunhome
Jul 18, 2022

I run yolort_trt for inference with c++, and this is a question about the file format specified by the --class_names argument.

When executing yolort_trt with the following command, a file called coco.names was specified as an argument of the --class_names option.
The file contents are as follows.

I can't seem to make an inference when I specify this, can you give me some advice on how to solve it?

./yolort_trt --image /home/manager/Pictures/girl.jpg --model_path ~/yolov5s.engine --class_names coco.names

coco.names file content
person bicycle car motorbike aeroplane bus train truck boat traffic light fire hydrant stop sign parking meter bench bird ...

manager@manager-desktop:~/coding/github-repository/yolov5-rt-stack/deployment/tensorrt/build$ ./yolort_trt --image /home/manager/Pictures/girl.jpg --model_path ~/yolov5s.engine --class_names coco.names
Loading engine from file: /home/manager/yolov5s.engine
3: Cannot find binding of given name: num_detections
3: Cannot find binding of given name: detection_boxes
3: Cannot find binding of given name: detection_scores
3: Cannot find binding of given name: detection_classes
3: [executionContext.cpp::setBindingDimensions::945] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::setBindingDimensions::945, condition: profileMinDims.d[i] <= dimensions.d[i]. Supplied binding dimension [1,3,640,640] for bindings[0] exceed min ~ max range at index 0, maximum dimension in profile is 10, minimum dimension in profile is 10, but supplied dimension is 1.
)
Bind[0] {Name:images, Datatype:0, Shape:(10,3,640,640,)}
Bind[1] {Name:output, Datatype:0, Shape:(10,25200,85,)}
Detected 0 objects.

Thank you.

zhiqwang · 2022-07-18T07:38:24Z

zhiqwang
Jul 18, 2022
Maintainer

Hi @leeyunhome , The way we generate the tensorrt engine is a little different from the official YOLOv5 one, did you export the tensorrt serialization model as follows?

python3 tools/export_model.py --checkpoint_path {path/to/your/best.pt} --include engine

https://github.com/zhiqwang/yolov5-rt-stack/tree/main/deployment/tensorrt#usage

BTW, you can find the coco.names sample at belows:

https://github.com/zhiqwang/yolov5-rt-stack/blob/main/notebooks/assets/coco.names

11 replies

zhiqwang Jul 19, 2022
Maintainer

Hi @leeyunhome , please add the --include engine argument to export the ONNX for TensorRT, The ONNX exported by default is for ONNX Runtime use.

python3 tools/export_model.py --checkpoint_path {path/to/your/best.pt} --include engine

And the main branch is unstable now, you can install the stable 0.6.3 version as belows, and move the export_model.py somewhere to avoid confusing it with the codes at main branch.

pip install yolort==0.6.3

leeyunhome Jul 19, 2022
Author

Hi @leeyunhome , please add the --include engine argument to export the ONNX for TensorRT, The ONNX exported by default is for ONNX Runtime use.
python3 tools/export_model.py --checkpoint_path {path/to/your/best.pt} --include engine
And the main branch is unstable now, you can install the stable 0.6.3 version as belows, and move the export_model.py somewhere to avoid confusing it with the codes at main branch.
pip install yolort==0.6.3

Hello, @zhiqwang

Installation using the command pip install yolort==0.6.3 fails. What's the problem?

manager@manager-desktop:~/coding/github-repository/yolov5-rt-stack/deployment/tensorrt/build$ ./yolort_trt --image /home/manager/Pictures/power_mode.png --model_path ~/yolov5s.engine --class_names coco.names 
Loading engine from file: /home/manager/yolov5s.engine
1: [stdArchiveReader.cpp::StdArchiveReader::40] Error Code 1: Serialization (Serialization assertion stdVersionRead == serializationVersion failed.Version tag does not match. Note: Current Version: 205, Serialized Engine Version: 213)
4: [runtime.cpp::deserializeCudaEngine::50] Error Code 4: Internal Error (Engine deserialization failed.)
Segmentation fault (core dumped)
manager@manager-desktop:~/coding/github-repository/yolov5-rt-stack/deployment/tensorrt/build$ ./yolort_trt --image /home/manager/Pictures/power_mode.png --model_path ~/yolov5s.engine --class_names coco.names ^C
manager@manager-desktop:~/coding/github-repository/yolov5-rt-stack/deployment/tensorrt/build$ pip3 uninstall yolort
Uninstalling yolort-0.7.0a0+c6482cf:
  /home/manager/.local/lib/python3.6/site-packages/yolort.egg-link
Proceed (y/n)? y
  Successfully uninstalled yolort-0.7.0a0+c6482cf
manager@manager-desktop:~/coding/github-repository/yolov5-rt-stack/deployment/tensorrt/build$ pip3 install yolort==0.6.3
Collecting yolort==0.6.3
  Using cached https://files.pythonhosted.org/packages/b1/0f/e331e7ddbb141f908c07d77ad2a70eb514cbdcfff66953ae784f96e6776e/yolort-0.6.3-py3-none-any.whl
Collecting ipython (from yolort==0.6.3)
  Using cached https://files.pythonhosted.org/packages/53/09/958a4802489d28b2484114ee6414c7502ef57de6f2dbc9095b718640060c/ipython-7.16.3-py3-none-any.whl
Collecting Pillow>=8.0.0 (from yolort==0.6.3)
Collecting onnx>=1.8.0 (from yolort==0.6.3)
  Using cached https://files.pythonhosted.org/packages/2c/6a/39b0580858589a67c3322aabc2634f158391ffbf98fa410127533e7f1495/onnx-1.12.0.tar.gz
Collecting thop (from yolort==0.6.3)
  Using cached https://files.pythonhosted.org/packages/15/d3/b2c788a51c55a26f3785625128285baf9461078a6a5c03836d9c6c7477c5/thop-0.1.1.post2207130030-py3-none-any.whl
Collecting tabulate (from yolort==0.6.3)
  Using cached https://files.pythonhosted.org/packages/92/4e/e5a13fdb3e6f81ce11893523ff289870c87c8f1f289a7369fb0e9840c3bb/tabulate-0.8.10-py3-none-any.whl
Collecting scipy>=1.4.1 (from yolort==0.6.3)
  Using cached https://files.pythonhosted.org/packages/aa/d5/dd06fe0e274e579e1dff21aa021219c039df40e39709fabe559faed072a5/scipy-1.5.4.tar.gz
Collecting tqdm>=4.41.0 (from yolort==0.6.3)
  Using cached https://files.pythonhosted.org/packages/8a/c4/d15f1e627fff25443ded77ea70a7b5532d6371498f9285d44d62587e209c/tqdm-4.64.0-py2.py3-none-any.whl
Collecting PyYAML>=5.3.1 (from yolort==0.6.3)
  Using cached https://files.pythonhosted.org/packages/36/2b/61d51a2c4f25ef062ae3f74576b01638bebad5e045f747ff12643df63844/PyYAML-6.0.tar.gz
Collecting numpy>=1.18.5 (from yolort==0.6.3)
Collecting pandas (from yolort==0.6.3)
  Downloading https://files.pythonhosted.org/packages/fb/e4/828bb9c2474ff6016e5ce96a78220d485436d5468c23068f4f6c2eb9cff8/pandas-1.1.5.tar.gz (5.2MB)
    100% |████████████████████████████████| 5.2MB 170kB/s 
Collecting matplotlib>=3.2.2 (from yolort==0.6.3)
  Using cached https://files.pythonhosted.org/packages/22/d4/e7ca532e68a9357742604e1e4ae35d9c09a4a810de39a9d80402bd12f50f/matplotlib-3.3.4.tar.gz
Collecting traitlets>=4.2 (from ipython->yolort==0.6.3)
  Downloading https://files.pythonhosted.org/packages/ca/ab/872a23e29cec3cf2594af7e857f18b687ad21039c1f9b922fac5b9b142d5/traitlets-4.3.3-py2.py3-none-any.whl (75kB)
    100% |████████████████████████████████| 81kB 4.1MB/s 
Collecting jedi<=0.17.2,>=0.10 (from ipython->yolort==0.6.3)
  Downloading https://files.pythonhosted.org/packages/c3/d4/36136b18daae06ad798966735f6c3fb96869c1be9f8245d2a8f556e40c36/jedi-0.17.2-py2.py3-none-any.whl (1.4MB)
    100% |████████████████████████████████| 1.4MB 656kB/s 
Collecting prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 (from ipython->yolort==0.6.3)
  Downloading https://files.pythonhosted.org/packages/b0/8f/09a88160539a1164de562809f8b1d0a36dc1f9d8c6473f4b71ebed17b953/prompt_toolkit-3.0.30-py3-none-any.whl (381kB)
    100% |████████████████████████████████| 389kB 1.6MB/s 
Collecting setuptools>=18.5 (from ipython->yolort==0.6.3)
  Downloading https://files.pythonhosted.org/packages/b0/3a/88b210db68e56854d0bcf4b38e165e03be377e13907746f825790f3df5bf/setuptools-59.6.0-py3-none-any.whl (952kB)
    100% |████████████████████████████████| 962kB 902kB/s 
Collecting backcall (from ipython->yolort==0.6.3)
  Downloading https://files.pythonhosted.org/packages/4c/1c/ff6546b6c12603d8dd1070aa3c3d273ad4c07f5771689a7b69a550e8c951/backcall-0.2.0-py2.py3-none-any.whl
Collecting pygments (from ipython->yolort==0.6.3)
  Downloading https://files.pythonhosted.org/packages/5c/8e/1d9017950034297fffa336c72e693a5b51bbf85141b24a763882cf1977b5/Pygments-2.12.0-py3-none-any.whl (1.1MB)
    100% |████████████████████████████████| 1.1MB 748kB/s 
Collecting pickleshare (from ipython->yolort==0.6.3)
  Downloading https://files.pythonhosted.org/packages/9a/41/220f49aaea88bc6fa6cba8d05ecf24676326156c23b991e80b3f2fc24c77/pickleshare-0.7.5-py2.py3-none-any.whl
Collecting decorator (from ipython->yolort==0.6.3)
  Downloading https://files.pythonhosted.org/packages/d5/50/83c593b07763e1161326b3b8c6686f0f4b0f24d5526546bee538c89837d6/decorator-5.1.1-py3-none-any.whl
Collecting pexpect; sys_platform != "win32" (from ipython->yolort==0.6.3)
  Downloading https://files.pythonhosted.org/packages/39/7b/88dbb785881c28a102619d46423cb853b46dbccc70d3ac362d99773a78ce/pexpect-4.8.0-py2.py3-none-any.whl (59kB)
    100% |████████████████████████████████| 61kB 4.7MB/s 
Collecting protobuf<=3.20.1,>=3.12.2 (from onnx>=1.8.0->yolort==0.6.3)
  Using cached https://files.pythonhosted.org/packages/c6/1c/f18d97fc479b4fb6f72bbb0e41188575362e3bbd31014cf294ef0fdec8bf/protobuf-3.19.4-py2.py3-none-any.whl
Collecting typing-extensions>=3.6.2.1 (from onnx>=1.8.0->yolort==0.6.3)
  Using cached https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl
Collecting torch (from thop->yolort==0.6.3)
  **Could not find a version that satisfies the requirement torch (from thop->yolort==0.6.3) (from versions: )
No matching distribution found for torch (from thop->yolort==0.6.3)**

manager@manager-desktop:~/coding/github-repository/yolov5-rt-stack/deployment/tensorrt/build$ ls
CMakeCache.txt  cmake_install.cmake  coco_orig.names  result.jpg
CMakeFiles      coco.names           Makefile         yolort_trt
manager@manager-desktop:~/coding/github-repository/yolov5-rt-stack/deployment/tensorrt/build$ python3
Python 3.6.9 (default, Mar 15 2022, 13:55:28) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import yolort
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'yolort'
>>> exit()
manager@manager-desktop:~/coding/github-repository/yolov5-rt-stack/deployment/tensorrt/build$ python3
Python 3.6.9 (default, Mar 15 2022, 13:55:28) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.__version__
'1.10.0'

Thank you.

zhiqwang Jul 19, 2022
Maintainer

This error report is a bit strange here, we tested the thop package on torch1.10 and it works fine, maybe other dependencies are causing conflicts here. The solution to this problem depends on your current environment. The best option would be to recreate a virtual environment using docker or conda.

A secondary option is to ignore the current conflict, but pray for better luck with subsequent uses.

pip install yolort==0.6.3 --no-deps

leeyunhome Jul 19, 2022
Author

This error report is a bit strange here, we tested the thop package on torch1.10 and it works fine, maybe other dependencies are causing conflicts here. The solution to this problem depends on your current environment. The best option would be to recreate a virtual environment using docker or conda.

A secondary option is to ignore the current conflict, but pray for better luck with subsequent uses.
pip install yolort==0.6.3 --no-deps

Hello, @zhiqwang

It is difficult to match the dependency.
Do you also provide docker environment?

Thank you.

zhiqwang Jul 19, 2022
Maintainer

the docker provided by pytorch https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch is sufficient I think for yolort

leeyunhome · 2022-07-19T23:23:23Z

leeyunhome
Jul 19, 2022
Author

Hello, @zhiqwang

I ran yolort_trt in Jetson Xavier NX with the yolov5s.engine file obtained with the export command you gave me in version 0.6.3 of yolort, but I get the following error.

Can you tell me where I went wrong?

manager@manager-desktop:~/coding/github-repository/yolov5-rt-stack/deployment/tensorrt/build$ ./yolort_trt --image ~/Pictures/girl.jpg --model_path ~/yolov5s.engine --class_names coco.names
Loading engine from file: /home/manager/yolov5s.engine
1: [stdArchiveReader.cpp::StdArchiveReader::40] Error Code 1: Serialization (Serialization assertion stdVersionRead == serializationVersion failed.Version tag does not match. Note: Current Version: 205, Serialized Engine Version: 213)
4: [runtime.cpp::deserializeCudaEngine::50] Error Code 4: Internal Error (Engine deserialization failed.)
Segmentation fault (core dumped)

Thank you.

2 replies

zhiqwang Jul 19, 2022
Maintainer

Could trtexec work for this serialized engine?

zhiqwang Jul 19, 2022
Maintainer

the serialized tensorrt engine is not cross platform, we need to rebuild the serialized engine from the ONNX on a new device

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

I run yolort_trt for inference with c++, and this is a question about the file format specified by the --class_names argument. #445

{{title}}

Replies: 2 comments 13 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

I run yolort_trt for inference with c++, and this is a question about the file format specified by the --class_names argument. #445

leeyunhome Jul 18, 2022

Replies: 2 comments · 13 replies

zhiqwang Jul 18, 2022 Maintainer

zhiqwang Jul 19, 2022 Maintainer

leeyunhome Jul 19, 2022 Author

zhiqwang Jul 19, 2022 Maintainer

leeyunhome Jul 19, 2022 Author

zhiqwang Jul 19, 2022 Maintainer

leeyunhome Jul 19, 2022 Author

zhiqwang Jul 19, 2022 Maintainer

zhiqwang Jul 19, 2022 Maintainer

leeyunhome
Jul 18, 2022

Replies: 2 comments 13 replies

zhiqwang
Jul 18, 2022
Maintainer

zhiqwang Jul 19, 2022
Maintainer

leeyunhome Jul 19, 2022
Author

zhiqwang Jul 19, 2022
Maintainer

leeyunhome Jul 19, 2022
Author

zhiqwang Jul 19, 2022
Maintainer

leeyunhome
Jul 19, 2022
Author

zhiqwang Jul 19, 2022
Maintainer

zhiqwang Jul 19, 2022
Maintainer