C. Training Procedure

Training Procedure

Training Package and Model Selection

The "FruitDetector" module employs Mask-RCNN to predict the classified masks overlapping a fruit in a given image. Mask-RCNN is proposed and developed by a team at Facebook AI Research (FAIR) as an extension to Faster-RCNN as an instance segmentation tool. There are several packages that help in training, prediction and evaluation of Mask-RCNN models such as torchvision, mmdetection and detectron2.

We selected detectron2 over others because it comes from the originator and is well-maintained. There are several pretrained and baseline models available for detectron2 at [1]. The pretrained models are trained on selection of datasets that are native to the detectron2 package. If a new dataset has to be trained, then it should be added as a custom dataset and should be trained on a defined pretrained dataset.

Training Procedure

FruitDetector Overview

The FruitDetector module has two modes of execution that primarily varies according to the required output. If the prediction results visualisation is required together with its COCO json file, then it is executed in debug mode. If low-latency execution with in-memory json message is required then it runs in optimized mode. The module include training, prediction and evaluation options which is mainly controlled through the configuration file.

FruitDetector Overview: FruitDetector components and execution overview.

Installation

Annotation

Configuration

datasets:
  train_dataset_name: 'aoc_train_dataset'
  test_dataset_name: 'aoc_test_dataset'
  dataset_train_annotation_url: 'https://lncn.ac/aocanntrain'
  dataset_train_images_url: 'https://lncn.ac/aocdatatrain'
  dataset_test_annotation_url: 'https://lncn.ac/aocanntest'
  dataset_test_images_url: 'https://lncn.ac/aocdatatest'
files:
  # pretrained model used as a training base model, if set as empty, the config file will use imagenet trained model as base.
  # model file is required for prediction, if further training of previously trained model is required then model
  # and pretrained model should have the path to that model file.
  pretrained_model_file: './model/aoc_tomato_ripeness_151_90k.pth'
  model_file: './model/aoc_tomato_ripeness_151_90k.pth'
  config_file: 'COCO-InstanceSegmentation/mask_rcnn_R_101_FPN_3x.yaml'
  test_metadata_catalog_file: './data/dataset_catalogs/tomato_ripeness_test_metadata_catalog.pkl'
  train_dataset_catalog_file: './data/dataset_catalogs/tomato_ripeness_train_dataset_catalog.pkl'
  train_annotation_file: './data/tomato_dataset/train/annotations/ripeness_class_annotations.json'
  test_annotation_file: './data/tomato_dataset/test/annotations/ripeness_class_annotations.json'
  model_url: 'https://lncn.ac/aocmodel'
  meta_catalog_url: 'https://lncn.ac/aocmeta'
  train_catalog_url: 'https://lncn.ac/aoccat'
directories:
  train_image_dir: './data/tomato_dataset/train/'
  test_image_dir: './data/tomato_dataset/test/'
  training_output_dir: './data/training_output/'
  prediction_output_dir: 'data/prediction_output/test_images/'
  prediction_json_dir: './data/annotations/predicted/'
training:
  epochs: 50000
  number_of_classes: 2
  optimizer: 'SGD'
  learning_rate: 0.0025
settings:
  download_assets: false # if assets such as model and datasets should be downloaded
  rename_pred_images: false #rename the predicted images in img_000001.png like format
  segm_masks_only: true
  bbox: true
  orientation_method: 'PCA'  #choose between PCA or LOG_POLAR

Training and Evaluation

Pre-trained Models

Output 1. masks 2. Confidence of prediction 3. Orientation of fruit

References

[1] Detectron2, Facebook Artificial Intelligence Research Team. Model zoo for detectron2 package, https://github.com/facebookresearch/detectron2/blob/main/MODEL_ZOO.md?plain=1