Skip to content

Timm stuff #843

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lgvaz wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Timm stuff #843

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
663f1d6
bumps fastai and fastcore versions
lgvaz May 6, 2021
7c5f4dd
updates changelog
lgvaz May 11, 2021
5671cb7
sketches
lgvaz May 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Supports pytorch 1.8
- rotate_limit changed from 45 to 15
- rgb_shift_limit changed from 20 to 10
- VOC parser uses image sizes from annotation file instead of image
- bumps fastai to latest version (<2.4)
### Deleted

## [0.7.0]
Expand Down
7 changes: 7 additions & 0 deletions icevision/backbones/timm/mobilenet.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
__all__ = ["mobilenet"]

import timm


def our_mobilenet(*args, **kwargs):
return timm.models.mobilenetv3(*args, **kwargs)
19 changes: 19 additions & 0 deletions icevision/models/mmdet/backbones/timm/mobilenet.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
from icevision.backbones.timm.mobilenet import *


@BACKBONES.register_module(force=True)
class MobileNetv3Large100(nn.Module):
def __init__(self, pretrained=True, out_indices=(0, 1, 2, 3, 4), **kwargs):
super().__init__()
self.model = our_mobilenet(
pretrained=True,
features_only=True,
out_indices=out_indices,
**kwargs,
)

def init_weights(self, pretrained=None):
pass

def forward(self, x): # should return a tuple
return self.model(x)
14 changes: 14 additions & 0 deletions icevision/models/mmdet/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,13 @@
mmdet_configs_path = download_mmdet_configs()


class MMDetTimmBackboneConfig(BackboneConfig):
def __init__(self, backbone):
self.backbone = backbone
self.feature_channels = [o["num_chs"] for o in list(backbone.feature_info)]
self.type = backbone.__class__.__name__


class MMDetBackboneConfig(BackboneConfig):
def __init__(self, model_name, config_path, weights_url):
self.model_name = model_name
Expand Down Expand Up @@ -75,4 +82,11 @@ def create_model_config(

cfg = Config.fromfile(config_path)

if isinstance(backbone, MMDetTimmBackboneConfig):
cfg.model.backbone = {
"type": backbone.type,
}

cfg.model.neck.in_channels = backbone.feature_channels

return cfg, weights_path
4 changes: 2 additions & 2 deletions setup.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ packages = find:
install_requires =
torch >=1.7.0,<1.9
torchvision >=0.8.0,<0.10
fastcore >=1.3.0,<1.3.3
fastcore >=1.3.0,<1.4
tqdm >=4.49.0,<5
opencv-python >=4.1.1,<5
albumentations >=0.4.6,<0.6
Expand All @@ -40,7 +40,7 @@ install_requires =

[options.extras_require]
all =
fastai >=2.1,<2.1.5
fastai >=2.1,<2.4
ipykernel >=4.10.1,<6
pytorch-lightning >=1.2.10
effdet >=0.2.1,<0.3
Expand Down
63 changes: 63 additions & 0 deletions sketch.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
### 0.8

- Yolov5 support
- fastai new version (multi-gpu and half-precision)
- mmdet new version
- support for negative samples
- pytorch 1.8

### namespace

- old -
model_type = models.mmdet.retinanet
backbone = model_type.backbones.resnet50_fpn_1x

models.torchvision.backbones

model_type = models.torchvision.faster_rcnn
backbone = model_type.backbones.resnet50



- current -
import icevision as ice

model_type = ice.detection.models.mmdet.retinanet
backbone = model_type.backbones.timm.resnet18

model_type = ice.classification.models.mmdet.?
backbone = model_type.backbones.timm.resnet18? (no backbone for classification?)


model_type = ice.multitask


### Mmdet backbone

- two steps
1. register the backbones
- Files to register the backbones

2. specify backbones per model


base_config_path = mmdet_configs_path / "retinanet"
config_path=base_config_path / "retinanet_r50_fpn_1x_coco.py"
config_path
cfg = Config.fromfile(config_path)
cfg.model.backbone = {
'type': 'MobileNetv3Large100',
}

cfg.model.neck.in_channels = neck_in_channels # [16, 24, 40 , 112, 960]

cfg.model.bbox_head.num_classes = len(parser.class_map) - 1

model = build_detector(cfg.model, cfg.get("train_cfg"), cfg.get("test_cfg"))
model

def get_in_channels():
x = torch.randn(1,3,224,224)
features = backbone(x)
neck_in_channels = [f.shape[1] for f in features]
return neck_in_channels