From 608bfb521dca02fe11ada707ad4ed58f917c592f Mon Sep 17 00:00:00 2001 From: zsszabo Date: Tue, 3 Dec 2024 15:36:38 +0100 Subject: [PATCH] reverting spaces --- .../notebooks/40-TestPipelineLocally.ipynb | 230 ++++---- .../notebooks/30-CreatePipelinePackage.ipynb | 510 ++++++++--------- .../notebooks/40-TestPipelineLocally.ipynb | 334 +++++------ .../keras-to-onnx/keras-to-onnx.ipynb | 516 ++++++++--------- .../pytorch-to-onnx/pytorch-to-onnx.ipynb | 526 +++++++++--------- 5 files changed, 1058 insertions(+), 1058 deletions(-) diff --git a/e2e-tutorials/batch_state_identifier/notebooks/40-TestPipelineLocally.ipynb b/e2e-tutorials/batch_state_identifier/notebooks/40-TestPipelineLocally.ipynb index 67f7e10..64e3f61 100644 --- a/e2e-tutorials/batch_state_identifier/notebooks/40-TestPipelineLocally.ipynb +++ b/e2e-tutorials/batch_state_identifier/notebooks/40-TestPipelineLocally.ipynb @@ -1,118 +1,118 @@ { - "cells": [ - { - "attachments": {}, - "cell_type": "markdown", - "metadata": { - "SPDX-FileCopyrightText": "Copyright (C) Siemens AG 2021. All Rights Reserved.", - "SPDX-License-Identifier": "MIT" - }, - "source": [ - "# Testing the edge configuration package\n", - "\n", - "In this notebook, the main goal is to test the edge configuration package created in notebook [30-CreatePipelinePackage](30-CreatePipelinePackage.ipynb)\n", - "driving it with the training data set which was used in notebook [10-CreateModel](10-CreateModel.ipynb).\n", - "\n", - "The `LocalPipelineRunner` object takes the edge configuration package and extracts its components.\n", - "Once the components are extracted, you can run them individually by calling `run_component` with component name and structured input data.\n", - "The method builds a `venv` Python virtual environment for running the component and installs the required dependencies listed in `requirements.txt` for the component.\n", - "Once the virtual python environment is ready, the method executes and feeds the component with your test data.\n", - "The result will be the list of outputs of your component. If your component does not always produce an output, as is the case with a preprocessor aggregating a windowful of data, the output list will be shorter than the input list." - ] + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "SPDX-FileCopyrightText": "Copyright (C) Siemens AG 2021. All Rights Reserved.", + "SPDX-License-Identifier": "MIT" + }, + "source": [ + "# Testing the edge configuration package\n", + "\n", + "In this notebook, the main goal is to test the edge configuration package created in notebook [30-CreatePipelinePackage](30-CreatePipelinePackage.ipynb)\n", + "driving it with the training data set which was used in notebook [10-CreateModel](10-CreateModel.ipynb).\n", + "\n", + "The `LocalPipelineRunner` object takes the edge configuration package and extracts its components.\n", + "Once the components are extracted, you can run them individually by calling `run_component` with component name and structured input data.\n", + "The method builds a `venv` Python virtual environment for running the component and installs the required dependencies listed in `requirements.txt` for the component.\n", + "Once the virtual python environment is ready, the method executes and feeds the component with your test data.\n", + "The result will be the list of outputs of your component. If your component does not always produce an output, as is the case with a preprocessor aggregating a windowful of data, the output list will be shorter than the input list." + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Define a dataset to test the package\n", + "The goal here is to create a list of input data which the `process_input(data: dict)` method will be triggered with. \n", + "For this reason, we read the json files and create the list of payloads as we did in notebook [10-CreateModel](10-CreateModel.ipynb)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "from pathlib import Path\n", + "\n", + "json_files = Path('../data/processed/example').glob('*.json')\n", + "inputs = []\n", + "for json_file in json_files:\n", + " with open(json_file) as f:\n", + " json_string = json.load(f)\n", + " inputs.append({'json_data': json.dumps(json_string)})" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Test the pipeline locally\n", + "To do so we instantiate a `LocalPipelineRunner` object with the path of our configuration package, and a directory where we want to check the results.\n", + "This directory will contain both the extracted component and the created python virtual environment.\n", + "If the directory is not defined, a temporary directory is created and deleted after testing." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from simaticai.testing.pipeline_runner import LocalPipelineRunner\n", + "\n", + "test_dir = Path('../test')\n", + "package_path = Path('../packages/Batch-State-Identifier-edge_1.zip')\n", + "\n", + "output = []\n", + "with LocalPipelineRunner(package_path, test_dir) as pipelineRunner:\n", + " output = pipelineRunner.run_pipeline(inputs) " + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Check the results\n", + "The method returns with the calculated results which can be compared to the expected result." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "output" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": " Python (batch_state_identifier)", + "language": "python", + "name": "batch_state_identifier" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Define a dataset to test the package\n", - "The goal here is to create a list of input data which the `process_input(data: dict)` method will be triggered with. \n", - "For this reason, we read the json files and create the list of payloads as we did in notebook [10-CreateModel](10-CreateModel.ipynb)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import json\n", - "from pathlib import Path\n", - "\n", - "json_files = Path('../data/processed/example').glob('*.json')\n", - "inputs = []\n", - "for json_file in json_files:\n", - " with open(json_file) as f:\n", - " json_string = json.load(f)\n", - " inputs.append({'json_data': json.dumps(json_string)})" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Test the pipeline locally\n", - "To do so we instantiate a `LocalPipelineRunner` object with the path of our configuration package, and a directory where we want to check the results.\n", - "This directory will contain both the extracted component and the created python virtual environment.\n", - "If the directory is not defined, a temporary directory is created and deleted after testing." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "from simaticai.testing.pipeline_runner import LocalPipelineRunner\n", - "\n", - "test_dir = Path('../test')\n", - "package_path = Path('../packages/Batch-State-Identifier-edge_1.zip')\n", - "\n", - "output = []\n", - "with LocalPipelineRunner(package_path, test_dir) as pipelineRunner:\n", - " output = pipelineRunner.run_pipeline(inputs) " - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Check the results\n", - "The method returns with the calculated results which can be compared to the expected result." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "output" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": " Python (batch_state_identifier)", - "language": "python", - "name": "batch_state_identifier" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 4 + "nbformat": 4, + "nbformat_minor": 4 } diff --git a/e2e-tutorials/image_classification/notebooks/30-CreatePipelinePackage.ipynb b/e2e-tutorials/image_classification/notebooks/30-CreatePipelinePackage.ipynb index 50ced98..f88e39a 100644 --- a/e2e-tutorials/image_classification/notebooks/30-CreatePipelinePackage.ipynb +++ b/e2e-tutorials/image_classification/notebooks/30-CreatePipelinePackage.ipynb @@ -1,258 +1,258 @@ { - "cells": [ - { - "attachments": {}, - "cell_type": "markdown", - "metadata": { - "SPDX-FileCopyrightText": "Copyright (C) Siemens AG 2021. All Rights Reserved.", - "SPDX-License-Identifier": "MIT" - }, - "source": [ - "# Create a TensorFlow Lite edge configuration package\n", - "\n", - "In this notebook, the main goal is to create a pipeline with all of the contents that are necessary for the execution of the model on an Industrial Edge device. \n", - "In order to put the elements together, this example collects files\n", - "\n", - "from [10-CreateClassificationModel](10-CreateClassificationModel.ipynb) notebook: \n", - "- **classification_mobilnet.tflite**: the trained model for classification with TFlite\n", - "\n", - "from [20-CreateInferenceWrapper](20-CreateInferenceWrapper.ipynb) notebook: \n", - "- **entrypoint.py**: the script that is called by the runtime to execute the model on the Edge side\n", - "- **payload.py**: contains the method which extracts the payload and create a PIL Image to be processed\n", - "- **vision_classifier.py**: contains the method to utilize the model and produces a prediction" - ] + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "SPDX-FileCopyrightText": "Copyright (C) Siemens AG 2021. All Rights Reserved.", + "SPDX-License-Identifier": "MIT" + }, + "source": [ + "# Create a TensorFlow Lite edge configuration package\n", + "\n", + "In this notebook, the main goal is to create a pipeline with all of the contents that are necessary for the execution of the model on an Industrial Edge device. \n", + "In order to put the elements together, this example collects files\n", + "\n", + "from [10-CreateClassificationModel](10-CreateClassificationModel.ipynb) notebook: \n", + "- **classification_mobilnet.tflite**: the trained model for classification with TFlite\n", + "\n", + "from [20-CreateInferenceWrapper](20-CreateInferenceWrapper.ipynb) notebook: \n", + "- **entrypoint.py**: the script that is called by the runtime to execute the model on the Edge side\n", + "- **payload.py**: contains the method which extracts the payload and create a PIL Image to be processed\n", + "- **vision_classifier.py**: contains the method to utilize the model and produces a prediction" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Please note, there is no official TFLite runtime for Windows." + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Imports " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from simaticai import deployment" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Create a single component\n", + "\n", + "In this step, we create a `PythonComponent` that reads the input data `vision_payload` by `entrypoint.py`, processes the images by `vision_classifier.py` with model `classification_mobilnet.tflite` and produces an output `prediction` by `entrypoint.py` script." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "COMPONENT_DESCRIPTION =\"\"\"\\\n", + "This component uses a trained TensorFlow Lite image classification model that reads the vision_payload input and produces a prediction as an output.\n", + "\"\"\"\n", + "INPUT_DESCRIPTION =\"\"\"\n", + "Vision connector MQTT payload holding the image to be classified.\n", + "\"\"\"\n", + "OUTPUT_DESCRIPTION =\"\"\"\n", + "The most probable class predicted for the image as an integer string.\n", + "\"\"\"\n", + "\n", + "component = deployment.PythonComponent(\n", + " name='inference', \n", + " python_version='3.11',\n", + " desc=COMPONENT_DESCRIPTION)\n", + "\n", + "component.add_resources('..', 'entrypoint.py')\n", + "component.set_entrypoint('entrypoint.py')\n", + "component.add_resources('..', ['src/payload.py', 'src/vision_classifier.py'])\n", + "\n", + "component.add_input('vision_payload', 'ImageSet')\n", + "component.add_output('prediction', 'String')" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Add metrics\n", + "It can be useful to monitor the pipeline, e.g. watch the prediction probabilities. The metric name must contain an underscore (`_`), because the part before the underscore is used to group custom metrics on the dashboard.\n", + "\n", + "**⚠ Remember!**\n", + "You have to use the same names here and in the inference wrapper script." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "component.add_metric(\"ic_probability\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Add dependencies\n", + "\n", + "All of the required dependencies are collected in the file `runtime_requirements.txt`. See [How to handle Python dependencies](../../../howto-guides/04-handle-python-dependencies.md) for more possibilities." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "component.set_requirements('../runtime_requirements-py3.11.txt')" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Add a model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "component.add_resources('..', 'models/classification_mobilnet.tflite')" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Create a pipeline from this component\n", + "\n", + "Now you can use the component to create a pipeline configuration. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "testinit_package_name" + ] + }, + "outputs": [], + "source": [ + "PIPELINE_DESCRIPTION =\"\"\"\\\n", + "This pipeline runs a TensorFlow Lite model on an Industrial Edge device.\n", + "The model was trained to recognize and classify images of following Siemens SIMATIC automation products: ET 200AL, ET 200eco PN, ET 200SP, S7-1200, S7-1500.\n", + "\n", + "The pipeline is designed to be fed from Vision Connector via Databus with PNG or JPEG type images.\n", + "The pipeline output is to be sent to the Databus.\n", + "\"\"\"\n", + "\n", + "#To assure compatibility with older versions of AI SDK (\n", + "Use the `input_names` and `output_names` arguments to specify the input / output variable names used in the inference pipeline.
\n", + "Parameter `opset` defines the version of the `ONNX format`, opset version `13` refers to ONNX format `1.8.0` which is supported by AI Inference Server at the time of writing.\n", + "\n", + "> ⚠️ Warning
\n", + "> The `verbose` parameter must be set to `False`, otherwise the ONNX exporter can get stuck in an infinite loop." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "torch_input = torch.randn(BATCH_SIZE, PIXEL_DEPTH, IMAGE_WIDTH, IMAGE_HEIGHT, requires_grad=True, device=DEVICE)\n", + "\n", + "input_names = [ \"input_1\" ] \n", + "output_names = [ \"output_1\" ]\n", + "\n", + "torch.onnx.export(\n", + " torch_model, \n", + " torch_input, \n", + " ONNX_PATH, \n", + " verbose=False, # Must be set to False\n", + " input_names=input_names, \n", + " output_names=output_names,\n", + " opset_version=13)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load the ONNX model and validate it\n", + "\n", + "Check the consistency of a model with `onnx.checker.check_model`. An exception is raised if the test fails." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import onnx\n", + "\n", + "onnx_model = onnx.load(ONNX_PATH)\n", + "onnx.checker.check_model(onnx_model)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Input and Output shape\n", + "\n", + "Let's inspect how the model and its inputs and outputs are shaped.\n", + "`graph.input` displays the input shape of the converted ONNX model." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "onnx_model.graph.input" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this case, the shape of the input is `[1 x 3 x 224 x 224]`.
\n", + "The shape of the output is `[1 x 5]`, which can be displayed with the following:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "onnx_model.graph.output" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Executing the model\n", + "\n", + "Before packaging the model, it is recommended to try it out with the `onnxruntime` Python package. \n", + "To do so we need to provide \n", + "- an `onnxruntime.InferenceSession` with the preloaded model \n", + "- the dictionary of the `input` arrays with the expected shape and type. \n", + " To test the model we are generating a numpy array with randomized float values. \n", + "- the list of the `output` arrays.\n", + "\n", + "\n", + "The `result` variable contains the output tensors in a list." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy\n", + "from onnxruntime import InferenceSession\n", + "\n", + "images = numpy.random.random((BATCH_SIZE, PIXEL_DEPTH, IMAGE_WIDTH, IMAGE_HEIGHT)).astype('float32')\n", + "session = InferenceSession(ONNX_PATH)\n", + "\n", + "result = session.run([\"output_1\"], {\"input_1\": images})\n", + "result" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Usage of the ONNX model\n", + "\n", + "The AI Inference Server with GPU support accepts ONNX models for execution. \n", + "For this purpose the model must be packaged into a `GPURuntimeComponent` step using AI Software Development Kit. \n", + "For details on how to create `GPURuntimeComponent` and build pipelines that run on a GPU enabled AI Inference Server you can study the [Object Detection](../../../e2e-tutorials/object_detection/README.md) example." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "image_classification", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Configuration\n", - "\n", - "Configure model path and input size. `MODEL_PATH` is the path of the PyTorch model you want to convert to ONNX format. The converted model will be saved to `ONNX_PATH`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import os\n", - "\n", - "MODEL_PATH = os.path.join(\"models\", \"model.pth\")\n", - "ONNX_PATH = os.path.join(\"output\", \"model.onnx\")\n", - "\n", - "IMAGE_WIDTH = 224\n", - "IMAGE_HEIGHT = 224\n", - "PIXEL_DEPTH = 3\n", - "\n", - "BATCH_SIZE = 1" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Check if we have a GPU available, if so, define the map location accordingly, otherwise, we will be using CPU to run our model." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import torch\n", - "\n", - "DEVICE = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", - "\n", - "if DEVICE == \"cuda\":\n", - "\tmap_location = lambda storage, loc: storage.cuda()\n", - "else:\n", - "\tmap_location = \"cpu\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load the PyTorch model\n", - "\n", - "This model is an image classification model based on a pretrained ResNet50 model. It was retrained with the `simatic_photos` dataset. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "torch_model = torch.load(MODEL_PATH, map_location=map_location)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Move the model to the device and set it in evaluation mode" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "torch_model.to(DEVICE)\n", - "torch_model.eval()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Convert to ONNX\n", - "\n", - "PyTorch requires a random input for the conversion. The input size must be known beforehand.
\n", - "Use the `input_names` and `output_names` arguments to specify the input / output variable names used in the inference pipeline.
\n", - "Parameter `opset` defines the version of the `ONNX format`, opset version `13` refers to ONNX format `1.8.0` which is supported by AI Inference Server at the time of writing.\n", - "\n", - "> ⚠️ Warning
\n", - "> The `verbose` parameter must be set to `False`, otherwise the ONNX exporter can get stuck in an infinite loop." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "torch_input = torch.randn(BATCH_SIZE, PIXEL_DEPTH, IMAGE_WIDTH, IMAGE_HEIGHT, requires_grad=True, device=DEVICE)\n", - "\n", - "input_names = [ \"input_1\" ] \n", - "output_names = [ \"output_1\" ]\n", - "\n", - "torch.onnx.export(\n", - " torch_model, \n", - " torch_input, \n", - " ONNX_PATH, \n", - " verbose=False, # Must be set to False\n", - " input_names=input_names, \n", - " output_names=output_names,\n", - " opset_version=13)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Load the ONNX model and validate it\n", - "\n", - "Check the consistency of a model with `onnx.checker.check_model`. An exception is raised if the test fails." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import onnx\n", - "\n", - "onnx_model = onnx.load(ONNX_PATH)\n", - "onnx.checker.check_model(onnx_model)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Input and Output shape\n", - "\n", - "Let's inspect how the model and its inputs and outputs are shaped.\n", - "`graph.input` displays the input shape of the converted ONNX model." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "onnx_model.graph.input" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In this case, the shape of the input is `[1 x 3 x 224 x 224]`.
\n", - "The shape of the output is `[1 x 5]`, which can be displayed with the following:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "onnx_model.graph.output" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Executing the model\n", - "\n", - "Before packaging the model, it is recommended to try it out with the `onnxruntime` Python package. \n", - "To do so we need to provide \n", - "- an `onnxruntime.InferenceSession` with the preloaded model \n", - "- the dictionary of the `input` arrays with the expected shape and type. \n", - " To test the model we are generating a numpy array with randomized float values. \n", - "- the list of the `output` arrays.\n", - "\n", - "\n", - "The `result` variable contains the output tensors in a list." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import numpy\n", - "from onnxruntime import InferenceSession\n", - "\n", - "images = numpy.random.random((BATCH_SIZE, PIXEL_DEPTH, IMAGE_WIDTH, IMAGE_HEIGHT)).astype('float32')\n", - "session = InferenceSession(ONNX_PATH)\n", - "\n", - "result = session.run([\"output_1\"], {\"input_1\": images})\n", - "result" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Usage of the ONNX model\n", - "\n", - "The AI Inference Server with GPU support accepts ONNX models for execution. \n", - "For this purpose the model must be packaged into a `GPURuntimeComponent` step using AI Software Development Kit. \n", - "For details on how to create `GPURuntimeComponent` and build pipelines that run on a GPU enabled AI Inference Server you can study the [Object Detection](../../../e2e-tutorials/object_detection/README.md) example." - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "image_classification", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.9" - } - }, - "nbformat": 4, - "nbformat_minor": 4 + "nbformat": 4, + "nbformat_minor": 4 }