pylon AI: Create Image Analysis Functions

Content

1. Create a new project

2. Upload and map image data set and AI model

3. Optimize AI model in test run

4. Release and download

5. Load AI model into the pylon AI vTool

Prerequisites

To get started with pylon AI, you need access to the pylon AI platform, along with the pylon Software Suite and have the pylon AI Supplementary Package downloaded and installed.

1. Create a new project

To begin, you will create a project and select your desired image analysis function. The image data and AI model will be loaded into this project in the next step.

1. Click on the My Projects section on the left-hand side and then on the Create new project button at the top.

2. Under Project type, select the image analysis function (represented by a pylon AI vTool).

3. Assign a project name under Project name.

4. Optionally, enter a project description under Project description.

5. Click on the Create new project button at the bottom.

The new project now appears in the My Projects section.

2. Upload and map the image data set and AI model

This section describes how to load your own data set of annotated sample images and your own AI model into the platform and assign (map) them to each other via project classes.

2a Upload the image data set

1. Open your existing project in the My Projects section.

2. Under Data Sets, click on the Add data set button.

3. Enter a name for your data set under Data set name and click Add.

4. On the Upload Data page, select the data format under Data format.

5. To add your own data set, click Browse under Add data.

6. The result, such as the number of images and associated data set classes, is displayed on the right under Data Validation . To include image annotations, leave the preselected option (active) under Annotations. Note: The annotations are no longer available after the upload. You must therefore create data set classes (see next step).

7. Under Class Mapping, press the Map classes button. Use the Autocreate classes button to create data set classes automatically based on your annotations. Alternatively, to perform this process for each class manually, click on the arrow to the right of each class and select Add new class .

8. Press the Upload button. The uploaded project is displayed. Click on it and then return to the My Projects section. Then click on the project and select Data Sets in the horizontal bar. Click on the project again. Now you can publish the project using the Publish button (press twice).

2b Upload the AI model

1. Under Model Versions, press the Add model version button.

2. Enter a name for the AI model under New model name and click Create.

3. Under Model configuration, click on the arrow on the right and select the configuration of your model.

4. To add an AI model, press the Browse button under Add data.

5. After uploading, you can check the model classes using the Show classes button.

6. Click on the Finish button.

7. Click on the house symbol to the left of the bar.

2c Mapping the image data set and AI model

1. Click on Mapping in the bar to assign the data set classes and AI model classes to the project classes.

2. To automatically assign the data set classes to the project classes, click on Automap under the Data Set Classes tab and then on the Map classes button.

3. Switch to the Model Classes tab. There, click on Automap and then on the Map classes button.

4. Click on the house symbol to the left of the bar.

3. Optimize the AI model in a test run

Performance benchmarking: Check the behaviour and performance of your optimized model on different processing hardware.

1. Click on the Create trial button and enter a name under Trial name.

2. To add your image data set, click on Add data set under Data set. Select the data set and click on the radio button. Click on Add data set.

3. To add your AI model, click on Add model under Model. Click on the radio button to select your model and then on Add model.

4. To view the data set classes and model classes that have already been mapped, press the See mapping button under Project mapping.

5. Click on Create.

6. Click on Add next trial run, select the hardware, and click on Select hardware. Use the arrow to select the ONNX model format and the respective processing hardware.

7. To add another piece of hardware, click Add next trial run again. Select the hardware and click on Select hardware. Use the arrow to select the ONNX model format and the respective Processing hardware again.

8. Click on Run trial. The system now determines the frame rate, latency, reliability, and power consumption for the respective processing hardware.

9. Press the Inference results button on the right to display the inference results for the images in the data set.

Use the My Projects button and the House icon to go to the next step.

4. Release and download

You have two options when releasing the bundle. You can download the bundle as a ZIP file (as described here under point 4) or drag it directly into the pylon AI vTool without downloading if an online connection is available (as described under point 5.1).

1. Choose a hardware / processing hardware from the list and press the Release bundle button on the right. Assign a name for the bundle of image data and AI model then click on Release bundle.

2. Select the bundle from the list and press the Go to bundle button in the line on the right.

3. Check the details and click on Download.

5. Load the AI model into the pylon AI vTool

In the pylon Viewer, load the optimized AI model into the pylon AI vTool using drag-and-drop. This is possible online or offline.

Open the pylon Viewer app on your desktop. Download pylon

1. Drag the pylon vTool "Image Loading" from the pylon vTools list in the top left corner into the Workbench and open the communication window by double-clicking on the pylon vTool.

2. Use the three dots to open the storage location of the image data you wish to analyze and open them in the pylon Viewer.

3. Now select the pylon AI vTool that you want to use for image analysis or for which you have optimized the AI model on the pylon AI platform. Open this pylon vTool by double-clicking on it to set the inference options.

4. Under Inference Settings, clicking on the update arrows automatically connects the correct inference server, which you can also edit manually. Click on Connect.

5. Under Model Selection, select the desired model type that you have previously optimized and released from the dropdown.

   • 5.1 If your bundle does not yet appear in the selection, you can use the pylon AI Agent to deploy it offline via the local storage. Alternatively, if an online connection is available, it is also possible to access the bundles optimized on the pylon AI platform directly.

   • 5.2 Drag and drop the optimized bundle into the window and then click on Deploy.

   • 5.3 Under the Deployed Models tab, you can see all your models and manage them. You can then close the window.

   • 5.4 Now you can select the deployed bundle in the dropdown or use the update arrows to automatically fill it with the latest version.

6. By clicking on Ok you are ready to start your image analysis.

This tutorial teaches you Tutorial how to set up pylon vTools and create recipes for image processing and analysis.