pylon AI: Create Image Analysis Functions

Prerequisites

To get started with pylon AI, you need access to the pylon AI Platform, and the pylon Software Suite and pylon AI Supplementary Package downloaded and installed.

Create a new project

Start by creating a project, which the image data and AI model will be uploaded to during the next step. Select the desired image analysis function accordingly.

1. Click on the section My Projects on the left-hand side. Then click the Create project button at the top.

2. Under Project type, select the image analysis function (represented by a pylon AI vTool).

3. Assign a project name under Project name.

4. Optionally, enter a project description under Project description.

5. Click on the Create project button at the bottom.

The new project now appears in the My Projects section.

Upload and map image data set and AI model

The following section describes how to upload your own data set of annotated sample images and your own AI model into the platform and assign (map) them to each other via project classes.

Upload image data set

1. Open your existing project in the My Projects section.

2. Under Data Sets, click on the Add data set button.

3. Enter a name and a description for your data set under Data set name and click Add.

4. On the Upload Data page, select the data format under Data format.

5. To add your own data set, click Browse under Add data.

6. The result, such as the number of images and associated data set classes, is displayed on the right under Data Validation. To include image annotations, leave the preselected option (active) under Annotations. Note: The annotations are no longer available after the upload. This means you have to create data set classes (see next step).

7. Under Class Mapping, press the Map classes button. Use the Autocreate classes button to create data set classes automatically based on your annotations. Alternatively, to perform this process for each class manually, click on the arrow to the right of each class and select Add new class.

8. Click the Upload button. Once the upload is complete, click Go to data set in the upper-right corner under Upload completed. You can now publish the project by clicking the Publish button (press twice).

Upload AI model

1. Click Models in the horizontal bar, and then click the Add model button.

2. Under Model name, enter a name and the description text for the AI model, then click Add.

3. Under Model configuration, click on the arrow on the right and select your model configuration.

4. To add an AI model, press the Browse button under Add data.

5. After uploading, you can check the model classes using the Show classes button under Model classes.

6. Click on the Finish button.

Mapping of image data set and AI model

1. Click on Mapping in the horizontal bar to assign the classes of data set and AI model to the project classes.

2. To automatically assign the data set classes to the project classes, click on Automap under the Data Set Classes tab and then on the Map classes button.

3. Click the orange icon to assign a different color to it.

4. Switch to the Model Classes tab. There, click on Automap and then on the Map classes button.

Optimize AI model in test run

Performance benchmarking: Check the behavior and performance of your optimized model on different processing hardware.

1. Click Trials in the horizontal bar. Then click the Create trial button and enter a name and description under Trial name.

2. To add your image data set, click on Add data set under Data set. Select the data set and click on the radio button. Click on Add data set.

3. To add your AI model, click on Add model under Model. Click on the radio button to select your model and then on Add model.

4. To view the data set classes and model classes that have already been mapped, press the Show mapping button under Project mapping.

5. Click on Create.

6. Click on Add trial run, select the hardware, and click on Select hardware. Use the arrow to select the ONNX model format respectively the processing hardware.

7. To add another piece of hardware, click Add trial run again. Select the hardware and click on Select hardware. Use the arrow to select the ONNX model format respectively the processing hardware again.

8. Click on Run trial. The system now determines the frame rate, latency, reliability, and power consumption for the respective processing hardware.

9. Click the three dots at the far right of the row and select Inference results to view the inference results for the images in the dataset. You can hide the annotations for each image by selecting Annotations on the right.

Release and download

You have two options when releasing the bundle of image data and AI model. You can download the bundle as a ZIP file (as described here) or drag it directly into the pylon AI vTool without downloading if an online connection is available (as described underneath under point 5.1).

1. Select a piece of hardware or processing hardware from the list, then click the three dots at the far right of the row and select Release bundle. Enter a name for the bundle and click Release bundle.

2. Select the bundle from the list, then click the three dots at the far right of the row and select Go to bundle.

3. Review the details and click Download. Save the bundle by clicking Save.

Load AI model into the pylon AI vTool

In the pylon Viewer, load the optimized AI model into the pylon AI vTool using drag-and-drop. This is possible online or offline.

Open the pylon Viewer app on your desktop. Download pylon

1. Drag the pylon vTool "Image Loading" from the pylon vTools list in the top left corner into the Workbench and open the communication window by double-clicking on the pylon vTool.

2. Use the three dots to open the storage location of your image data to be analyzed and to open them in the pylon Viewer.

3. Now select the pylon AI vTool that you want to use for image analysis, respectively for the AI model you optimized on the pylon AI platform. Open this pylon vTool by double-clicking on it to set the inference options.

4. Under Inference Settings, clicking on the update arrows automatically connects the correct inference server, which you can also edit manually. Click on Connect.

5. Under Model Selection, select the desired model type that you previously optimized and released from the dropdown.

   1. If your bundle does not yet appear in the selection, you can use the pylon AI Agent to deploy it offline via the local storage. Alternatively, if an online connection is available, it is also possible to access the bundles optimized on the pylon AI platform directly.

   2. Drag and drop the optimized bundle into the window. Then click on Deploy.

   3. Under the Deployed Models tab, you can see all your models and manage them. You can then close the window.

   4. Now you can select the deployed bundle in the dropdown or use the update arrows to automatically fill it with the latest version.

6. By clicking on Ok you are ready to start your image analysis.

Learn in this Tutorial how to set up your image processing and analysis with the pylon vTools and create recipes.