Train from scratch with custom dataset

[hungtrieu07]

Hi maintainer,

I want to train my dataset with this repository. I was implemented some code snippet for match the dataset description. However, the accuracy is not good (in the current time I'm writing this issue, it's about 47,5%). This is a car license plate in my country:

In the uploaded image, each of license plate can be mounted at front and rear of vehicle. The one line license plate, not a problem with me; but the two line, I was processed with this step:

1. Crop the plate into 2 images, like this:

2. Label them in label text file:
   image1.jpg 30E
   image2.jpg 777.96

This is a sample from my label file:

My question is, how can I improve the accuracy on my dataset?

This is the forked repository, can you watch that and give me some suggestion? Thank you 😍

[zjykzj]

Hi maintainer,

I want to train my dataset with this repository. I was implemented some code snippet for match the dataset description. However, the accuracy is not good (in the current time I'm writing this issue, it's about 47,5%). This is a car license plate in my country:

In the uploaded image, each of license plate can be mounted at front and rear of vehicle. The one line license plate, not a problem with me; but the two line, I was processed with this step:

1. Crop the plate into 2 images, like this:

2. Label them in label text file:
   image1.jpg 30E
   image2.jpg 777.96

This is a sample from my label file:

My question is, how can I improve the accuracy on my dataset?

This is the forked repository, can you watch that and give me some suggestion? Thank you 😍

Let's first clarify the current status of your algorithm and the problems you have encountered.

1. There are two types of license plates, single row and double row;
2. Before training, you have already split the double row license plate into two single row license plate images.

So, your license plate dataset consists of single row license plates, no matter it's training or validation. How much data do you currently have in your training and testing datasets? Can you post some test results?

[hungtrieu07]

Hi maintainer,
I want to train my dataset with this repository. I was implemented some code snippet for match the dataset description. However, the accuracy is not good (in the current time I'm writing this issue, it's about 47,5%). This is a car license plate in my country:

In the uploaded image, each of license plate can be mounted at front and rear of vehicle. The one line license plate, not a problem with me; but the two line, I was processed with this step:

1. Crop the plate into 2 images, like this:

2. Label them in label text file:
   image1.jpg 30E
   image2.jpg 777.96

This is a sample from my label file:

My question is, how can I improve the accuracy on my dataset?
This is the forked repository, can you watch that and give me some suggestion? Thank you 😍

Let's first clarify the current status of your algorithm and the problems you have encountered.

1. There are two types of license plates, single row and double row;
2. Before training, you have already split the double row license plate into two single row license plate images.

So, your license plate dataset consists of single row license plates, no matter it's training or validation. How much data do you currently have in your training and testing datasets? Can you post some test results?

Hi @zjykzj, thanks for the reply.

My dataset have about 22.000 images, the training images about 18.500 images, the testing images about 3.500 images (we can collect more). After create some custom code snippet, I'm able to training but the predict result is bad. In example:

The output from terminal:

`
(lprnet) (base) hungtrieu07@potato:~/dev/crnn-ctc$ python predict_custom.py runs/lprnet_plus_stnet-custom-b512/lprnet_plus-custom-b512-e200.pth test.png . --not-tiny --use-lprnet
Python-3.10.16 torch-2.5.0 CUDA:0 (NVIDIA GeForce RTX 3050, 5832MiB)

/home/hungtrieu07/dev/crnn-ctc/predict_custom.py:38: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
model.load_state_dict(torch.load(args.pretrained, map_location=device))
Pred: -
Predict time: 735.5 ms
Save to ./plate_test.png
`

The plate_test.png have no content, the filesize only 1 byte (maybe error). This is the custom code snippet I'm using for predict:

import argparse
import os
import time
import torch
from PIL import Image
import torchvision.transforms as transforms
from utils.dataset.custom import CustomPlateDataset
from utils.model.crnn import CRNN
from utils.model.lprnet import LPRNet
from utils.converter import StrLabelConverter, get_custom_plate_chars
from utils.torchutil import select_device
from utils.logger import LOGGER

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('pretrained', help='Path to pretrained model')
    parser.add_argument('image_path', help='Path to input image')
    parser.add_argument('save_dir', help='Path to save predictions')
    parser.add_argument('--not-tiny', action='store_true')
    parser.add_argument('--use-lprnet', action='store_true')
    parser.add_argument('--use-origin-block', action='store_true')
    parser.add_argument('--add-stnet', action='store_true')
    parser.add_argument('--use-lstm', action='store_true')
    return parser.parse_args()

def main():
    args = parse_args()
    device = select_device('', batch_size=1)
    CUSTOM_CHARS = get_custom_plate_chars()
    converter = StrLabelConverter()

    input_shape = (94, 24) if args.use_lprnet else (168, 48)
    model = LPRNet(in_channel=3, num_classes=len(CUSTOM_CHARS) + 1,
                   use_origin_block=args.use_origin_block, add_stnet=args.add_stnet).to(device) \
        if args.use_lprnet else \
        CRNN(in_channel=3, num_classes=len(CUSTOM_CHARS) + 1, cnn_input_height=input_shape[1],
             is_tiny=not args.not_tiny, use_gru=not args.use_lstm).to(device)
    model.load_state_dict(torch.load(args.pretrained, map_location=device))
    model.eval()

    transform = transforms.Compose([
        transforms.Resize((input_shape[1], input_shape[0])),
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
    ])
    image = Image.open(args.image_path).convert('RGB')
    image = transform(image).unsqueeze(0).to(device)

    t0 = time.time()
    with torch.no_grad():
        preds = model(image)
        preds_size = torch.IntTensor([preds.size(0)]).to(device)
        _, preds = preds.max(2)
        preds = preds.transpose(1, 0).contiguous().view(-1)
        pred_text = converter.decode(preds.data, preds_size.data, raw=False)
    predict_time = (time.time() - t0) * 1000
    LOGGER.info(f"Pred: {pred_text} \n Predict time: {predict_time:.1f} ms")

    os.makedirs(args.save_dir, exist_ok=True)
    save_path = os.path.join(args.save_dir, f"plate_{os.path.basename(args.image_path)}")
    with open(save_path, 'w') as f:
        f.write(pred_text)
    LOGGER.info(f"Save to {save_path}")

if __name__ == '__main__':
    main()

[zjykzj]

Hi maintainer,
I want to train my dataset with this repository. I was implemented some code snippet for match the dataset description. However, the accuracy is not good (in the current time I'm writing this issue, it's about 47,5%). This is a car license plate in my country:

In the uploaded image, each of license plate can be mounted at front and rear of vehicle. The one line license plate, not a problem with me; but the two line, I was processed with this step:

1. Crop the plate into 2 images, like this:

2. Label them in label text file:
   image1.jpg 30E
   image2.jpg 777.96

This is a sample from my label file:

My question is, how can I improve the accuracy on my dataset?
This is the forked repository, can you watch that and give me some suggestion? Thank you 😍

Let's first clarify the current status of your algorithm and the problems you have encountered.

1. There are two types of license plates, single row and double row;
2. Before training, you have already split the double row license plate into two single row license plate images.

So, your license plate dataset consists of single row license plates, no matter it's training or validation. How much data do you currently have in your training and testing datasets? Can you post some test results?

Hi @zjykzj, thanks for the reply.

My dataset have about 22.000 images, the training images about 18.500 images, the testing images about 3.500 images (we can collect more). After create some custom code snippet, I'm able to training but the predict result is bad. In example:

The output from terminal:

` (lprnet) (base) hungtrieu07@potato:~/dev/crnn-ctc$ python predict_custom.py runs/lprnet_plus_stnet-custom-b512/lprnet_plus-custom-b512-e200.pth test.png . --not-tiny --use-lprnet Python-3.10.16 torch-2.5.0 CUDA:0 (NVIDIA GeForce RTX 3050, 5832MiB)

/home/hungtrieu07/dev/crnn-ctc/predict_custom.py:38: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. model.load_state_dict(torch.load(args.pretrained, map_location=device)) Pred: - Predict time: 735.5 ms Save to ./plate_test.png `

The plate_test.png have no content, the filesize only 1 byte (maybe error). This is the custom code snippet I'm using for predict:

import argparse
import os
import time
import torch
from PIL import Image
import torchvision.transforms as transforms
from utils.dataset.custom import CustomPlateDataset
from utils.model.crnn import CRNN
from utils.model.lprnet import LPRNet
from utils.converter import StrLabelConverter, get_custom_plate_chars
from utils.torchutil import select_device
from utils.logger import LOGGER

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('pretrained', help='Path to pretrained model')
    parser.add_argument('image_path', help='Path to input image')
    parser.add_argument('save_dir', help='Path to save predictions')
    parser.add_argument('--not-tiny', action='store_true')
    parser.add_argument('--use-lprnet', action='store_true')
    parser.add_argument('--use-origin-block', action='store_true')
    parser.add_argument('--add-stnet', action='store_true')
    parser.add_argument('--use-lstm', action='store_true')
    return parser.parse_args()

def main():
    args = parse_args()
    device = select_device('', batch_size=1)
    CUSTOM_CHARS = get_custom_plate_chars()
    converter = StrLabelConverter()

    input_shape = (94, 24) if args.use_lprnet else (168, 48)
    model = LPRNet(in_channel=3, num_classes=len(CUSTOM_CHARS) + 1,
                   use_origin_block=args.use_origin_block, add_stnet=args.add_stnet).to(device) \
        if args.use_lprnet else \
        CRNN(in_channel=3, num_classes=len(CUSTOM_CHARS) + 1, cnn_input_height=input_shape[1],
             is_tiny=not args.not_tiny, use_gru=not args.use_lstm).to(device)
    model.load_state_dict(torch.load(args.pretrained, map_location=device))
    model.eval()

    transform = transforms.Compose([
        transforms.Resize((input_shape[1], input_shape[0])),
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
    ])
    image = Image.open(args.image_path).convert('RGB')
    image = transform(image).unsqueeze(0).to(device)

    t0 = time.time()
    with torch.no_grad():
        preds = model(image)
        preds_size = torch.IntTensor([preds.size(0)]).to(device)
        _, preds = preds.max(2)
        preds = preds.transpose(1, 0).contiguous().view(-1)
        pred_text = converter.decode(preds.data, preds_size.data, raw=False)
    predict_time = (time.time() - t0) * 1000
    LOGGER.info(f"Pred: {pred_text} \n Predict time: {predict_time:.1f} ms")

    os.makedirs(args.save_dir, exist_ok=True)
    save_path = os.path.join(args.save_dir, f"plate_{os.path.basename(args.image_path)}")
    with open(save_path, 'w') as f:
        f.write(pred_text)
    LOGGER.info(f"Save to {save_path}")

if __name__ == '__main__':
    main()

It's strange that the model's prediction result is empty. As you mentioned earlier, the accuracy of the validation set is only 47.5%. So, how much accuracy does the training set have during the training process? Is it possible that overfitting has occurred?

[hungtrieu07]

Hi maintainer,
I want to train my dataset with this repository. I was implemented some code snippet for match the dataset description. However, the accuracy is not good (in the current time I'm writing this issue, it's about 47,5%). This is a car license plate in my country:

In the uploaded image, each of license plate can be mounted at front and rear of vehicle. The one line license plate, not a problem with me; but the two line, I was processed with this step:

1. Crop the plate into 2 images, like this:

2. Label them in label text file:
   image1.jpg 30E
   image2.jpg 777.96

This is a sample from my label file:

My question is, how can I improve the accuracy on my dataset?
This is the forked repository, can you watch that and give me some suggestion? Thank you 😍

Let's first clarify the current status of your algorithm and the problems you have encountered.

1. There are two types of license plates, single row and double row;
2. Before training, you have already split the double row license plate into two single row license plate images.

So, your license plate dataset consists of single row license plates, no matter it's training or validation. How much data do you currently have in your training and testing datasets? Can you post some test results?

Hi @zjykzj, thanks for the reply.
My dataset have about 22.000 images, the training images about 18.500 images, the testing images about 3.500 images (we can collect more). After create some custom code snippet, I'm able to training but the predict result is bad. In example:

The output from terminal:
(lprnet) (base) hungtrieu07@potato:~/dev/crnn-ctc$ python predict_custom.py runs/lprnet_plus_stnet-custom-b512/lprnet_plus-custom-b512-e200.pth test.png . --not-tiny --use-lprnet Python-3.10.16 torch-2.5.0 CUDA:0 (NVIDIA GeForce RTX 3050, 5832MiB) /home/hungtrieu07/dev/crnn-ctc/predict_custom.py:38: FutureWarning: You are usingtorch.loadwithweights_only=False(the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value forweights_onlywill be flipped toTrue. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=Truefor any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. model.load_state_dict(torch.load(args.pretrained, map_location=device)) Pred: - Predict time: 735.5 ms Save to ./plate_test.png
The plate_test.png have no content, the filesize only 1 byte (maybe error). This is the custom code snippet I'm using for predict:

import argparse
import os
import time
import torch
from PIL import Image
import torchvision.transforms as transforms
from utils.dataset.custom import CustomPlateDataset
from utils.model.crnn import CRNN
from utils.model.lprnet import LPRNet
from utils.converter import StrLabelConverter, get_custom_plate_chars
from utils.torchutil import select_device
from utils.logger import LOGGER

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('pretrained', help='Path to pretrained model')
    parser.add_argument('image_path', help='Path to input image')
    parser.add_argument('save_dir', help='Path to save predictions')
    parser.add_argument('--not-tiny', action='store_true')
    parser.add_argument('--use-lprnet', action='store_true')
    parser.add_argument('--use-origin-block', action='store_true')
    parser.add_argument('--add-stnet', action='store_true')
    parser.add_argument('--use-lstm', action='store_true')
    return parser.parse_args()

def main():
    args = parse_args()
    device = select_device('', batch_size=1)
    CUSTOM_CHARS = get_custom_plate_chars()
    converter = StrLabelConverter()

    input_shape = (94, 24) if args.use_lprnet else (168, 48)
    model = LPRNet(in_channel=3, num_classes=len(CUSTOM_CHARS) + 1,
                   use_origin_block=args.use_origin_block, add_stnet=args.add_stnet).to(device) \
        if args.use_lprnet else \
        CRNN(in_channel=3, num_classes=len(CUSTOM_CHARS) + 1, cnn_input_height=input_shape[1],
             is_tiny=not args.not_tiny, use_gru=not args.use_lstm).to(device)
    model.load_state_dict(torch.load(args.pretrained, map_location=device))
    model.eval()

    transform = transforms.Compose([
        transforms.Resize((input_shape[1], input_shape[0])),
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
    ])
    image = Image.open(args.image_path).convert('RGB')
    image = transform(image).unsqueeze(0).to(device)

    t0 = time.time()
    with torch.no_grad():
        preds = model(image)
        preds_size = torch.IntTensor([preds.size(0)]).to(device)
        _, preds = preds.max(2)
        preds = preds.transpose(1, 0).contiguous().view(-1)
        pred_text = converter.decode(preds.data, preds_size.data, raw=False)
    predict_time = (time.time() - t0) * 1000
    LOGGER.info(f"Pred: {pred_text} \n Predict time: {predict_time:.1f} ms")

    os.makedirs(args.save_dir, exist_ok=True)
    save_path = os.path.join(args.save_dir, f"plate_{os.path.basename(args.image_path)}")
    with open(save_path, 'w') as f:
        f.write(pred_text)
    LOGGER.info(f"Save to {save_path}")

if __name__ == '__main__':
    main()

It's strange that the model's prediction result is empty. As you mentioned earlier, the accuracy of the validation set is only 47.5%. So, how much accuracy does the training set have during the training process? Is it possible that overfitting has occurred?

This is the log from training task: https://pastebin.com/Cyy1RQYK
Eval task: https://pastebin.com/apUS0Gzt

[zjykzj]

Hi maintainer,
I want to train my dataset with this repository. I was implemented some code snippet for match the dataset description. However, the accuracy is not good (in the current time I'm writing this issue, it's about 47,5%). This is a car license plate in my country:

In the uploaded image, each of license plate can be mounted at front and rear of vehicle. The one line license plate, not a problem with me; but the two line, I was processed with this step:

1. Crop the plate into 2 images, like this:

2. Label them in label text file:
   image1.jpg 30E
   image2.jpg 777.96

This is a sample from my label file:

My question is, how can I improve the accuracy on my dataset?
This is the forked repository, can you watch that and give me some suggestion? Thank you 😍

Let's first clarify the current status of your algorithm and the problems you have encountered.

1. There are two types of license plates, single row and double row;
2. Before training, you have already split the double row license plate into two single row license plate images.

So, your license plate dataset consists of single row license plates, no matter it's training or validation. How much data do you currently have in your training and testing datasets? Can you post some test results?

Hi @zjykzj, thanks for the reply.
My dataset have about 22.000 images, the training images about 18.500 images, the testing images about 3.500 images (we can collect more). After create some custom code snippet, I'm able to training but the predict result is bad. In example:

The output from terminal:
(lprnet) (base) hungtrieu07@potato:~/dev/crnn-ctc$ python predict_custom.py runs/lprnet_plus_stnet-custom-b512/lprnet_plus-custom-b512-e200.pth test.png . --not-tiny --use-lprnet Python-3.10.16 torch-2.5.0 CUDA:0 (NVIDIA GeForce RTX 3050, 5832MiB) /home/hungtrieu07/dev/crnn-ctc/predict_custom.py:38: FutureWarning: You are usingtorch.loadwithweights_only=False(the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value forweights_onlywill be flipped toTrue. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=Truefor any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. model.load_state_dict(torch.load(args.pretrained, map_location=device)) Pred: - Predict time: 735.5 ms Save to ./plate_test.png
The plate_test.png have no content, the filesize only 1 byte (maybe error). This is the custom code snippet I'm using for predict:

import argparse
import os
import time
import torch
from PIL import Image
import torchvision.transforms as transforms
from utils.dataset.custom import CustomPlateDataset
from utils.model.crnn import CRNN
from utils.model.lprnet import LPRNet
from utils.converter import StrLabelConverter, get_custom_plate_chars
from utils.torchutil import select_device
from utils.logger import LOGGER

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('pretrained', help='Path to pretrained model')
    parser.add_argument('image_path', help='Path to input image')
    parser.add_argument('save_dir', help='Path to save predictions')
    parser.add_argument('--not-tiny', action='store_true')
    parser.add_argument('--use-lprnet', action='store_true')
    parser.add_argument('--use-origin-block', action='store_true')
    parser.add_argument('--add-stnet', action='store_true')
    parser.add_argument('--use-lstm', action='store_true')
    return parser.parse_args()

def main():
    args = parse_args()
    device = select_device('', batch_size=1)
    CUSTOM_CHARS = get_custom_plate_chars()
    converter = StrLabelConverter()

    input_shape = (94, 24) if args.use_lprnet else (168, 48)
    model = LPRNet(in_channel=3, num_classes=len(CUSTOM_CHARS) + 1,
                   use_origin_block=args.use_origin_block, add_stnet=args.add_stnet).to(device) \
        if args.use_lprnet else \
        CRNN(in_channel=3, num_classes=len(CUSTOM_CHARS) + 1, cnn_input_height=input_shape[1],
             is_tiny=not args.not_tiny, use_gru=not args.use_lstm).to(device)
    model.load_state_dict(torch.load(args.pretrained, map_location=device))
    model.eval()

    transform = transforms.Compose([
        transforms.Resize((input_shape[1], input_shape[0])),
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
    ])
    image = Image.open(args.image_path).convert('RGB')
    image = transform(image).unsqueeze(0).to(device)

    t0 = time.time()
    with torch.no_grad():
        preds = model(image)
        preds_size = torch.IntTensor([preds.size(0)]).to(device)
        _, preds = preds.max(2)
        preds = preds.transpose(1, 0).contiguous().view(-1)
        pred_text = converter.decode(preds.data, preds_size.data, raw=False)
    predict_time = (time.time() - t0) * 1000
    LOGGER.info(f"Pred: {pred_text} \n Predict time: {predict_time:.1f} ms")

    os.makedirs(args.save_dir, exist_ok=True)
    save_path = os.path.join(args.save_dir, f"plate_{os.path.basename(args.image_path)}")
    with open(save_path, 'w') as f:
        f.write(pred_text)
    LOGGER.info(f"Save to {save_path}")

if __name__ == '__main__':
    main()

It's strange that the model's prediction result is empty. As you mentioned earlier, the accuracy of the validation set is only 47.5%. So, how much accuracy does the training set have during the training process? Is it possible that overfitting has occurred?

Try using CRNN instead of LPRNetPlus, as its model performance is more powerful. See if it meets your recognition requirements