I think you forgot to upload the decoder weights.
"RuntimeError: Error(s) in loading state_dict for WiT:
Missing key(s) in state_dict: "Tdec.patch_embed1.proj.weight", "Tdec.patch_embed1.proj.bias", "Tdec.patch_embed1.norm.weight", "Tdec.patch_embed1.norm.bias", "Tdec.block1.0.norm1.weight", "Tdec.block1.0.norm1.bias", "Tdec.block1.0.attn.task_query", "Tdec.block1.0.attn.q.weight", "Tdec.block1.0.attn.q.bias", "Tdec.block1.0.attn.kv.weight", "Tdec.block1.0.attn.kv.bias", "Tdec.block1.0.attn.proj.weight", "Tdec.block1.0.attn.proj.bias", "Tdec.block1.0.norm2.weight", "Tdec.block1.0.norm2.bias", "Tdec.block1.0.mlp.fc1.weight", "Tdec.block1.0.mlp.fc1.bias", "Tdec.block1.0.mlp.dwconv.dwconv.weight", "Tdec.block1.0.mlp.dwconv.dwconv.bias", "Tdec.block1.0.mlp.fc2.weight", "Tdec.block1.0.mlp.fc2.bias", "Tdec.block1.1.norm1.weight", "Tdec.block1.1.norm1.bias", "Tdec.block1.1.attn.task_query", "Tdec.block1.1.attn.q.weight", "Tdec.block1.1.attn.q.bias", "Tdec.block1.1.attn.kv.weight", "Tdec.block1.1.attn.kv.bias", "Tdec.block1.1.attn.proj.weight", "Tdec.block1.1.attn.proj.bias", "Tdec.block1.1.norm2.weight", "Tdec.block1.1.norm2.bias", "Tdec.block1.1.mlp.fc1.weight", "Tdec.block1.1.mlp.fc1.bias", "Tdec.block1.1.mlp.dwconv.dwconv.weight", "Tdec.block1.1.mlp.dwconv.dwconv.bias", "Tdec.block1.1.mlp.fc2.weight", "Tdec.block1.1.mlp.fc2.bias", "Tdec.block1.2.norm1.weight", "Tdec.block1.2.norm1.bias", "Tdec.block1.2.attn.task_query", "Tdec.block1.2.attn.q.weight", "Tdec.block1.2.attn.q.bias", "Tdec.block1.2.attn.kv.weight", "Tdec.block1.2.attn.kv.bias", "Tdec.block1.2.attn.proj.weight", "Tdec.block1.2.attn.proj.bias", "Tdec.block1.2.norm2.weight", "Tdec.block1.2.norm2.bias", "Tdec.block1.2.mlp.fc1.weight", "Tdec.block1.2.mlp.fc1.bias", "Tdec.block1.2.mlp.dwconv.dwconv.weight", "Tdec.block1.2.mlp.dwconv.dwconv.bias", "Tdec.block1.2.mlp.fc2.weight", "Tdec.block1.2.mlp.fc2.bias", "Tdec.norm1.weight", "Tdec.norm1.bias". "
I think you forgot to upload the decoder weights.
"RuntimeError: Error(s) in loading state_dict for WiT:
Missing key(s) in state_dict: "Tdec.patch_embed1.proj.weight", "Tdec.patch_embed1.proj.bias", "Tdec.patch_embed1.norm.weight", "Tdec.patch_embed1.norm.bias", "Tdec.block1.0.norm1.weight", "Tdec.block1.0.norm1.bias", "Tdec.block1.0.attn.task_query", "Tdec.block1.0.attn.q.weight", "Tdec.block1.0.attn.q.bias", "Tdec.block1.0.attn.kv.weight", "Tdec.block1.0.attn.kv.bias", "Tdec.block1.0.attn.proj.weight", "Tdec.block1.0.attn.proj.bias", "Tdec.block1.0.norm2.weight", "Tdec.block1.0.norm2.bias", "Tdec.block1.0.mlp.fc1.weight", "Tdec.block1.0.mlp.fc1.bias", "Tdec.block1.0.mlp.dwconv.dwconv.weight", "Tdec.block1.0.mlp.dwconv.dwconv.bias", "Tdec.block1.0.mlp.fc2.weight", "Tdec.block1.0.mlp.fc2.bias", "Tdec.block1.1.norm1.weight", "Tdec.block1.1.norm1.bias", "Tdec.block1.1.attn.task_query", "Tdec.block1.1.attn.q.weight", "Tdec.block1.1.attn.q.bias", "Tdec.block1.1.attn.kv.weight", "Tdec.block1.1.attn.kv.bias", "Tdec.block1.1.attn.proj.weight", "Tdec.block1.1.attn.proj.bias", "Tdec.block1.1.norm2.weight", "Tdec.block1.1.norm2.bias", "Tdec.block1.1.mlp.fc1.weight", "Tdec.block1.1.mlp.fc1.bias", "Tdec.block1.1.mlp.dwconv.dwconv.weight", "Tdec.block1.1.mlp.dwconv.dwconv.bias", "Tdec.block1.1.mlp.fc2.weight", "Tdec.block1.1.mlp.fc2.bias", "Tdec.block1.2.norm1.weight", "Tdec.block1.2.norm1.bias", "Tdec.block1.2.attn.task_query", "Tdec.block1.2.attn.q.weight", "Tdec.block1.2.attn.q.bias", "Tdec.block1.2.attn.kv.weight", "Tdec.block1.2.attn.kv.bias", "Tdec.block1.2.attn.proj.weight", "Tdec.block1.2.attn.proj.bias", "Tdec.block1.2.norm2.weight", "Tdec.block1.2.norm2.bias", "Tdec.block1.2.mlp.fc1.weight", "Tdec.block1.2.mlp.fc1.bias", "Tdec.block1.2.mlp.dwconv.dwconv.weight", "Tdec.block1.2.mlp.dwconv.dwconv.bias", "Tdec.block1.2.mlp.fc2.weight", "Tdec.block1.2.mlp.fc2.bias", "Tdec.norm1.weight", "Tdec.norm1.bias". "