Eval bug: imatrix generation for LFM2 fails; collect_imatrix: inconsistent size for blk.0.shortconv.in_proj.weight

[ThomasBaruzier]

Name and Version

llama.cpp 6e67254

Operating systems

Linux

GGML backends

CUDA, CPU

Hardware

3x3090 + 5950x + 128GB DDR4

Models

• LFM2-350M: inconsistent size for blk.0.shortconv.in_proj.weight (4096 vs 3072)
• LFM2-700M: inconsistent size for blk.0.shortconv.in_proj.weight (6144 vs 4608)
• LFM2-1.2B: inconsistent size for blk.0.shortconv.in_proj.weight (8192 vs 6144)

Problem description & steps to reproduce

pytohn convert_hf_to_gguf.py LFM2-350M/ --outfile LFM2-350M-F16.gguf
./llama-imatrix -m LFM2-350M-F16.gguf -f calibration_datav3.txt -o imatrix.gguf
./llama-imatrix -m LFM2-350M-F16.gguf -f calibration_datav3.txt -o imatrix.gguf -ngl 99
./llama-imatrix -m LFM2-350M-F16.gguf -f calibration_datav3.txt -o imatrix.dat

All commands produce the same inconsistent size error.
Note that all size errors have a ratio of 4/3 (4096 / 3072 = 4/3).

First Bad Commit

9008328 is the problematic commit

Relevant log output

Logs

./llama-imatrix -m LFM2-350M-F16.gguf -f calibration_datav3.txt -o imatrix.gguf
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 3 CUDA devices:
  Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
  Device 1: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
  Device 2: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
build: 4 (6e67254) with cc (GCC) 15.1.1 20250425 for x86_64-pc-linux-gnu
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GeForce RTX 3090) - 23062 MiB free
llama_model_load_from_file_impl: using device CUDA1 (NVIDIA GeForce RTX 3090) - 22766 MiB free
llama_model_load_from_file_impl: using device CUDA2 (NVIDIA GeForce RTX 3090) - 23461 MiB free
llama_model_loader: loaded meta data with 34 key-value pairs and 148 tensors from LFM2-350M-F16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = lfm2
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = LFM2 350M
llama_model_loader: - kv   3:                           general.basename str              = LFM2
llama_model_loader: - kv   4:                         general.size_label str              = 350M
llama_model_loader: - kv   5:                            general.license str              = other
llama_model_loader: - kv   6:                       general.license.name str              = lfm1.0
llama_model_loader: - kv   7:                       general.license.link str              = LICENSE
llama_model_loader: - kv   8:                               general.tags arr[str,4]       = ["liquid", "lfm2", "edge", "text-gene...
llama_model_loader: - kv   9:                          general.languages arr[str,8]       = ["en", "ar", "zh", "fr", "de", "ja", ...
llama_model_loader: - kv  10:                           lfm2.block_count u32              = 16
llama_model_loader: - kv  11:                        lfm2.context_length u32              = 128000
llama_model_loader: - kv  12:                      lfm2.embedding_length u32              = 1024
llama_model_loader: - kv  13:                   lfm2.feed_forward_length u32              = 4608
llama_model_loader: - kv  14:                  lfm2.attention.head_count u32              = 16
llama_model_loader: - kv  15:               lfm2.attention.head_count_kv arr[i32,16]      = [0, 0, 8, 0, 0, 8, 0, 0, 8, 0, 8, 0, ...
llama_model_loader: - kv  16:                        lfm2.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  17:                          general.file_type u32              = 1
llama_model_loader: - kv  18:                            lfm2.vocab_size u32              = 65536
llama_model_loader: - kv  19:                     lfm2.shortconv.l_cache u32              = 3
llama_model_loader: - kv  20:      lfm2.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  21:               general.quantization_version u32              = 2
llama_model_loader: - kv  22:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  23:                         tokenizer.ggml.pre str              = lfm2
llama_model_loader: - kv  24:                      tokenizer.ggml.tokens arr[str,65536]   = ["<|pad|>", "<|startoftext|>", "<|end...
llama_model_loader: - kv  25:                  tokenizer.ggml.token_type arr[i32,65536]   = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ...
llama_model_loader: - kv  26:                      tokenizer.ggml.merges arr[str,63683]   = ["Ċ Ċ", "Ċ ĊĊ", "ĊĊ Ċ", "Ċ �...
llama_model_loader: - kv  27:                tokenizer.ggml.bos_token_id u32              = 1
llama_model_loader: - kv  28:                tokenizer.ggml.eos_token_id u32              = 7
llama_model_loader: - kv  29:            tokenizer.ggml.padding_token_id u32              = 0
llama_model_loader: - kv  30:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  31:               tokenizer.ggml.add_sep_token bool             = false
llama_model_loader: - kv  32:               tokenizer.ggml.add_eos_token bool             = false
llama_model_loader: - kv  33:                    tokenizer.chat_template str              = {{- bos_token -}}\n{%- set system_prom...
llama_model_loader: - type  f32:   55 tensors
llama_model_loader: - type  f16:   93 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = F16
print_info: file size   = 676.25 MiB (16.00 BPW) 
load: special tokens cache size = 507
load: token to piece cache size = 0.3756 MB
print_info: arch             = lfm2
print_info: vocab_only       = 0
print_info: n_ctx_train      = 128000
print_info: n_embd           = 1024
print_info: n_layer          = 16
print_info: n_head           = 16
print_info: n_head_kv        = [0, 0, 8, 0, 0, 8, 0, 0, 8, 0, 8, 0, 8, 0, 8, 0]
print_info: n_rot            = 64
print_info: n_swa            = 0
print_info: is_swa_any       = 0
print_info: n_embd_head_k    = 64
print_info: n_embd_head_v    = 64
print_info: n_gqa            = [0, 0, 2, 0, 0, 2, 0, 0, 2, 0, 2, 0, 2, 0, 2, 0]
print_info: n_embd_k_gqa     = [0, 0, 512, 0, 0, 512, 0, 0, 512, 0, 512, 0, 512, 0, 512, 0]
print_info: n_embd_v_gqa     = [0, 0, 512, 0, 0, 512, 0, 0, 512, 0, 512, 0, 512, 0, 512, 0]
print_info: f_norm_eps       = 0.0e+00
print_info: f_norm_rms_eps   = 1.0e-05
print_info: f_clamp_kqv      = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale    = 0.0e+00
print_info: f_attn_scale     = 0.0e+00
print_info: n_ff             = 4608
print_info: n_expert         = 0
print_info: n_expert_used    = 0
print_info: causal attn      = 1
print_info: pooling type     = 0
print_info: rope type        = 2
print_info: rope scaling     = linear
print_info: freq_base_train  = 1000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn  = 128000
print_info: rope_finetuned   = unknown
print_info: model type       = 350M
print_info: model params     = 354.48 M
print_info: general.name     = LFM2 350M
print_info: vocab type       = BPE
print_info: n_vocab          = 65536
print_info: n_merges         = 63683
print_info: BOS token        = 1 '<|startoftext|>'
print_info: EOS token        = 7 '<|im_end|>'
print_info: EOT token        = 2 '<|endoftext|>'
print_info: PAD token        = 0 '<|pad|>'
print_info: LF token         = 708 'Ċ'
print_info: EOG token        = 2 '<|endoftext|>'
print_info: EOG token        = 7 '<|im_end|>'
print_info: max token length = 30
load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: offloading 0 repeating layers to GPU
load_tensors: offloaded 0/17 layers to GPU
load_tensors:   CPU_Mapped model buffer size =   676.25 MiB
...................................................................
llama_context: constructing llama_context
llama_context: non-unified KV cache requires ggml_set_rows() - forcing unified KV cache
llama_context: n_seq_max     = 4
llama_context: n_ctx         = 2048
llama_context: n_ctx_per_seq = 512
llama_context: n_batch       = 2048
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = 0
llama_context: kv_unified    = true
llama_context: freq_base     = 1000000.0
llama_context: freq_scale    = 1
llama_context: n_ctx_per_seq (512) < n_ctx_train (128000) -- the full capacity of the model will not be utilized
llama_context:        CPU  output buffer size =     1.00 MiB
llama_kv_cache_unified: the V embeddings have different sizes across layers and FA is not enabled - padding V cache to 512
llama_kv_cache_unified:        CPU KV buffer size =    24.00 MiB
llama_kv_cache_unified: size =   24.00 MiB (  2048 cells,   6 layers,  4/ 1 seqs), K (f16):   12.00 MiB, V (f16):   12.00 MiB
llama_kv_cache_unified: LLAMA_SET_ROWS=0, using old ggml_cpy() method for backwards compatibility
llama_memory_recurrent:        CPU RS buffer size =     0.31 MiB
llama_memory_recurrent: size =    0.31 MiB (     4 cells,  16 layers,  4 seqs), R (f32):    0.31 MiB, S (f32):    0.00 MiB
llama_context:      CUDA0 compute buffer size =   264.01 MiB
llama_context:  CUDA_Host compute buffer size =    72.01 MiB
llama_context: graph nodes  = 602
llama_context: graph splits = 167 (with bs=512), 1 (with bs=1)
common_init_from_params: added <|endoftext|> logit bias = -inf
common_init_from_params: added <|im_end|> logit bias = -inf
common_init_from_params: setting dry_penalty_last_n to ctx_size = 2048

system_info: n_threads = 16 (n_threads_batch = 16) / 32 | CUDA : ARCHS = 860 | F16 = 1 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | FA_ALL_QUANTS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
compute_imatrix: tokenizing the input ..
compute_imatrix: tokenization took 36.134 ms
compute_imatrix: computing over 131 chunks, n_ctx=512, batch_size=2048, n_seq=4
compute_imatrix: 1.59 seconds per pass - ETA 0.85 minutes
[1]16.1218,[2]16.0527,[3]15.8228,[4]19.0799,[5]19.5578,[6]16.6691,[7]19.1424,[8]19.6710,[9]19.9923,[10]18.0396,[11]15.8644,[12]16.6023,[13]17.7008,[14]17.5375,[15]18.4722,[16]18.7421,[17]19.4153,[18]19.5910,[19]18.3784,[20]18.1191,[21]18.6345,[22]18.7850,[23]19.1435,[24]19.7705,[25]20.1693,[26]20.5836,[27]20.7830,[28]20.9911,[29]22.1619,[30]22.2467,[31]22.8473,[32]21.9694,[33]21.2215,[34]20.8443,[35]20.4220,[36]19.9727,[37]19.6946,[38]19.7684,[39]19.6883,[40]19.6866,[41]20.0332,[42]20.0252,[43]20.7494,[44]21.2795,[45]21.9291,[46]22.4473,[47]23.0100,[48]22.5381,[49]22.6655,[50]22.7210,[51]22.8786,[52]22.3267,[53]22.3220,[54]22.5152,[55]22.6539,[56]22.9201,[57]23.1667,[58]23.2689,[59]23.2460,[60]23.2207,[61]23.0716,[62]22.8744,[63]22.4267,[64]22.1785,[65]22.5743,[66]22.6038,[67]22.2798,[68]22.1210,[69]21.9884,[70]21.8635,[71]21.5494,[72]21.4812,[73]21.5279,[74]21.2028,[75]21.1883,[76]21.1878, [77]21.0229,[78]21.0508,[79]20.8843,[80]20.8705,[81]20.7459,[82]20.8424,[83]20.7547,[84]20.6542,[85]20.6533,[86]20.6420,[87]20.7667,[88]20.5743,[89]20.5772,[90]20.6513,[91]20.6953,[92]20.6352,[93]20.4360,[94]20.1697,[95]19.9172,[96]19.6822,[97]19.4291,[98]19.1929,[99]18.9387,[100]18.7208,[101]18.6064,[102]18.5537,[103]18.6731,[104]18.8646,[105]19.0555,[106]19.1200,[107]19.4036,[108]19.4173,[109]19.4649,[110]19.3261,[111]19.3172,[112]19.2799,[113]19.2930,[114]19.1867,[115]19.1901,[116]19.2213,[117]19.1991,[118]19.2698,[119]19.2935,[120]19.3463,[121]19.3871,[122]19.3852,[123]19.4242,[124]19.4143,[125]19.2542,[126]19.4468,[127]19.6391,[128]19.7795,
collect_imatrix: inconsistent size for blk.0.shortconv.in_proj.weight (4096 vs 3072)

[ggerganov]

cc @CISC @compilade

[compilade]

Ooh, this is probably caused by how multiple parallel sequences are handled for the intermediate result matmultiplied with in_proj.

As in, the shape isn't {some_size, n_tokens} as imatrix.cpp expects, but is {some_size, n_seq_tokens, n_seqs}.

As a workaround (in the meantime before this is fixed), when calculating imatrix for recurrent models, you should be able to force a single sequence to be calculated at a time by using a batch size equal to the chunk size (or at least smaller than two chunks). E.g. with -b 512 if using a chunk size of 512 (default, but configurable with -c).

If that workaround doesn't work, then I'm wrong about the cause.

I think the fix in imatrix.cpp will be to flatten the shape of src1 above ne[1] if src0->ne[2] and above are not more than 1 (because 3d tensors still need to be handled correctly too). I'll try something tomorrow (need to sleep first).

(EDIT: the reason of the 3/4 ratio is because 131 % 4 == 3, and so the last 3 chunks use a batch with 3 sequences instead of 4. Even if the number of chunks was a multiple of 4, I doubt that the produced imatrix file would be usable (but a single-sequence one (as suggested in the workaround above) should be fine). This fits with my hypothesis for the cause of the problem)

[compilade]

I've implemented a fix in #14994

[ThomasBaruzier]

I've implemented a fix in #14994

Fixed it, thanks!

[ThomasBaruzier]

Hello @compilade, would you know a fix for the following issue as well? Or is this a separate issue?

python convert_hf_to_gguf.py gemma-3n-E4B-it --outfile gemma-3n-E4B-it-F16.gguf
./llama-imatrix -m gemma-3n-E4B-it-F16.gguf -o imatrix.gguf -f calibration_datav3.txt
./llama-quantize --imatrix imatrix.gguf gemma-3n-E4B-it-F16.gguf gemma-3n-E4B-it-IQ2_XXS.gguf IQ2_XXS 32

====== llama_model_quantize_impl: did not find weights for blk.20.attn_k.weight

============================================================
Missing importance matrix for tensor blk.20.attn_k.weight in a very low-bit quantization
The result will be garbage, so bailing out
============================================================

llama_model_quantize: failed to quantize: Missing importance matrix for tensor blk.20.attn_k.weight in a very low-bit quantization
main: failed to quantize model from 'gemma-3n-E4B-it-F16.gguf'

Full logs

INFO: CMD | python convert_hf_to_gguf.py gemma-3n-E4B-it --outfile gemma-3n-E4B-it-F16.gguf
INFO:hf-to-gguf:Loading model: gemma-3n-E4B-it
INFO:hf-to-gguf:Model architecture: Gemma3nForConditionalGeneration
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
INFO:hf-to-gguf:Exporting model...
INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: loading model part 'model-00001-of-00004.safetensors'
INFO:hf-to-gguf:altup_proj.weight,                 torch.bfloat16 --> F16, shape = {2048, 2048, 3}
INFO:hf-to-gguf:altup_unembd_proj.weight,          torch.bfloat16 --> F16, shape = {2048, 2048, 3}
WARNING:hf-to-gguf:ignore tokens from 262144: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262144: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262145: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262146: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262147: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262148: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262149: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262150: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262151: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262152: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262153: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262154: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262155: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262156: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262157: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262158: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262159: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262160: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262161: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262162: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262163: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262164: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262165: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262166: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262167: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262168: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262169: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262170: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262171: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262172: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262173: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262174: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262175: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262176: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262177: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262178: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262179: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262180: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262181: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262182: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262183: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262184: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262185: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262186: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262187: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262188: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262189: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262190: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262191: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262192: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262193: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262194: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262195: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262196: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262197: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262198: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262199: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262200: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262201: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262202: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262203: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262204: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262205: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262206: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262207: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262208: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262209: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262210: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262211: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262212: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262213: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262214: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262215: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262216: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262217: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262218: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262219: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262220: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262221: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262222: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262223: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262224: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262225: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262226: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262227: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262228: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262229: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262230: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262231: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262232: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262233: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262234: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262235: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262236: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262237: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262238: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262239: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262240: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262241: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262242: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262243: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262244: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262245: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262246: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262247: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262248: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262249: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262250: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262251: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262252: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262253: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262254: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262255: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262256: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262257: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262258: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262259: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262260: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262261: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262262: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262263: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262264: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262265: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262266: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262267: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262268: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262269: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262270: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262271: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262272: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262273: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262274: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262275: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262276: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262277: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262278: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262279: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262280: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262281: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262282: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262283: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262284: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262285: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262286: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262287: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262288: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262289: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262290: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262291: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262292: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262293: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262294: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262295: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262296: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262297: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262298: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262299: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262300: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262301: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262302: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262303: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262304: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262305: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262306: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262307: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262308: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262309: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262310: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262311: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262312: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262313: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262314: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262315: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262316: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262317: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262318: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262319: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262320: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262321: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262322: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262323: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262324: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262325: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262326: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262327: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262328: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262329: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262330: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262331: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262332: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262333: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262334: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262335: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262336: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262337: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262338: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262339: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262340: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262341: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262342: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262343: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262344: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262345: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262346: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262347: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262348: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262349: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262350: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262351: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262352: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262353: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262354: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262355: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262356: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262357: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262358: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262359: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262360: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262361: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262362: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262363: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262364: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262365: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262366: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262367: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262368: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262369: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262370: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262371: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262372: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262373: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262374: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262375: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262376: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262377: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262378: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262379: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262380: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262381: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262382: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262383: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262384: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262385: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262386: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262387: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262388: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262389: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262390: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262391: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262392: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262393: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262394: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262395: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262396: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262397: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262398: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262399: id is out of range, max=262143
INFO:hf-to-gguf:token_embd.weight,                 torch.bfloat16 --> F16, shape = {2048, 262144}
INFO:hf-to-gguf:gguf: loading model part 'model-00002-of-00004.safetensors'
INFO:hf-to-gguf:per_layer_token_embd.weight,       torch.bfloat16 --> F16, shape = {8960, 262144}
INFO:hf-to-gguf:blk.0.altup_correct_scale.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.0.altup_correct_coef.weight,   torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.0.altup_router.weight,         torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.0.altup_predict_coef.weight,   torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.0.altup_router_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.0.laurel_l.weight,             torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.0.laurel_r.weight,             torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.0.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.0.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.0.attn_k.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.0.attn_output.weight,          torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.0.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.0.attn_q.weight,               torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.0.attn_v.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.10.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.10.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.10.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.10.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.10.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.10.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.10.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.10.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.10.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.10.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.10.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.10.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.10.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.15.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.15.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.15.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.15.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.15.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.15.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.15.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.15.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.15.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.15.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.15.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.15.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.15.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.20.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.20.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.20.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.20.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.20.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.20.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.20.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.20.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.20.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.20.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.20.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.20.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.20.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.25.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.25.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.25.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.25.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.25.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.25.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.25.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.25.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.25.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.25.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.25.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.25.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.25.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.30.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.30.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.30.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.30.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.30.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.30.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.30.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.30.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.30.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.30.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.30.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.30.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.30.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.5.altup_correct_scale.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.5.altup_correct_coef.weight,   torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.5.altup_router.weight,         torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.5.altup_predict_coef.weight,   torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.5.altup_router_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.5.laurel_l.weight,             torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.5.laurel_r.weight,             torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.5.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.5.attn_k.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.5.attn_output.weight,          torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.5.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.5.attn_q.weight,               torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.5.attn_v.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:output_norm.weight,                torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:per_layer_model_proj.weight,       torch.bfloat16 --> F16, shape = {2048, 8960}
INFO:hf-to-gguf:per_layer_proj_norm.weight,        torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:gguf: loading model part 'model-00003-of-00004.safetensors'
INFO:hf-to-gguf:blk.0.attn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.0.laurel_post_norm.weight,     torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.0.ffn_down.weight,             torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.0.ffn_up.weight,               torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.0.inp_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.0.proj.weight,                 torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.0.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.0.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.0.post_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.0.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.1.altup_correct_scale.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.1.altup_correct_coef.weight,   torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.1.altup_router.weight,         torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.1.altup_predict_coef.weight,   torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.1.altup_router_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.1.attn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.1.laurel_l.weight,             torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.1.laurel_r.weight,             torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.1.laurel_post_norm.weight,     torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.1.ffn_down.weight,             torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.1.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.1.ffn_up.weight,               torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.1.inp_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.1.proj.weight,                 torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.1.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.1.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.1.post_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.1.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.1.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.1.attn_k.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.1.attn_output.weight,          torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.1.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.1.attn_q.weight,               torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.1.attn_v.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.10.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.10.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.10.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.10.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.10.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.10.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.10.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.10.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.10.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.10.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.10.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.11.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.11.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.11.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.11.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.11.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.11.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.11.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.11.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.11.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.11.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.11.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.11.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.11.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.11.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.11.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.11.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.11.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.11.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.11.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.11.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.11.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.11.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.11.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.11.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.12.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.12.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.12.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.12.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.12.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.12.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.12.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.12.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.12.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.12.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.12.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.12.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.12.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.12.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.12.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.12.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.12.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.12.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.12.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.12.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.12.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.12.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.12.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.12.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.13.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.13.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.13.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.13.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.13.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.13.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.13.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.13.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.13.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.13.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.13.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.13.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.13.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.13.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.15.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.15.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.15.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.15.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.15.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.15.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.15.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.15.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.15.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.15.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.15.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.16.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.16.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.16.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.16.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.16.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.16.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.16.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.16.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.16.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.16.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.16.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.16.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.16.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.16.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.16.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.16.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.16.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.16.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.16.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.16.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.16.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.16.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.16.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.16.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.17.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.17.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.17.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.17.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.17.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.17.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.17.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.17.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.17.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.17.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.17.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.17.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.17.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.17.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.17.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.17.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.17.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.17.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.17.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.17.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.17.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.17.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.17.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.17.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.18.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.18.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.18.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.18.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.18.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.18.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.18.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.18.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.18.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.18.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.18.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.18.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.18.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.2.altup_correct_scale.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.2.altup_correct_coef.weight,   torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.2.altup_router.weight,         torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.2.altup_predict_coef.weight,   torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.2.altup_router_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.2.attn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.2.laurel_l.weight,             torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.2.laurel_r.weight,             torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.2.laurel_post_norm.weight,     torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.2.ffn_down.weight,             torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.2.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.2.ffn_up.weight,               torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.2.inp_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.2.proj.weight,                 torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.2.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.2.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.2.post_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.2.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.2.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.2.attn_k.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.2.attn_output.weight,          torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.2.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.2.attn_q.weight,               torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.2.attn_v.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.20.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.20.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.20.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.20.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.20.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.20.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.20.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.20.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.20.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.20.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.20.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.21.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.21.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.21.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.21.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.21.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.21.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.21.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.21.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.21.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.21.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.21.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.21.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.21.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.21.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.21.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.21.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.21.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.21.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.21.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.21.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.21.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.21.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.21.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.21.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.22.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.22.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.22.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.22.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.22.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.22.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.22.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.22.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.22.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.22.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.22.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.22.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.22.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.22.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.22.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.22.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.22.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.22.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.22.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.22.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.22.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.22.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.22.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.22.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.23.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.23.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.23.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.23.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.23.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.23.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.23.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.23.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.23.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.23.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.23.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.23.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.23.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.25.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.25.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.25.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.25.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.25.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.25.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.25.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.25.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.25.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.25.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.25.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.26.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.26.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.26.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.26.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.26.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.26.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.26.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.26.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.26.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.26.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.26.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.26.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.26.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.26.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.26.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.26.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.26.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.26.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.26.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.26.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.26.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.26.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.26.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.26.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.27.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.27.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.27.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.27.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.27.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.27.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.27.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.27.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.27.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.27.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.27.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.27.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.27.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.27.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.27.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.27.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.27.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.27.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.27.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.27.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.27.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.27.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.27.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.27.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.28.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.28.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.28.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.28.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.28.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.28.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.28.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.28.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.28.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.28.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.28.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.28.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.28.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.3.altup_correct_scale.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.3.altup_correct_coef.weight,   torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.3.altup_router.weight,         torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.3.altup_predict_coef.weight,   torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.3.altup_router_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.3.laurel_l.weight,             torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.3.laurel_r.weight,             torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.3.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.3.ffn_up.weight,               torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.3.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.3.attn_k.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.3.attn_output.weight,          torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.3.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.3.attn_q.weight,               torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.3.attn_v.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.30.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.30.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.30.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.30.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.30.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.30.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.30.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.30.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.30.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.30.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.30.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.31.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.31.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.31.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.31.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.31.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.31.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.31.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.31.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.31.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.31.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.31.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.31.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.31.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.31.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.31.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.31.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.31.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.31.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.31.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.31.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.31.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.31.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.31.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.31.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.32.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.32.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.32.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.32.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.32.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.32.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.32.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.32.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.32.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.32.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.32.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.32.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.32.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.32.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.32.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.32.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.32.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.32.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.32.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.32.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.32.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.32.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.32.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.32.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.33.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.33.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.33.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.33.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.33.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.33.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.33.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.33.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.33.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.33.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.33.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.33.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.33.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.5.attn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.5.laurel_post_norm.weight,     torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.5.ffn_down.weight,             torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.5.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.5.ffn_up.weight,               torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.5.inp_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.5.proj.weight,                 torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.5.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.5.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.5.post_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.5.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.6.altup_correct_scale.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.6.altup_correct_coef.weight,   torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.6.altup_router.weight,         torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.6.altup_predict_coef.weight,   torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.6.altup_router_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.6.attn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.6.laurel_l.weight,             torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.6.laurel_r.weight,             torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.6.laurel_post_norm.weight,     torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.6.ffn_down.weight,             torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.6.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.6.ffn_up.weight,               torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.6.inp_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.6.proj.weight,                 torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.6.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.6.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.6.post_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.6.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.6.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.6.attn_k.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.6.attn_output.weight,          torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.6.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.6.attn_q.weight,               torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.6.attn_v.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.7.altup_correct_scale.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.7.altup_correct_coef.weight,   torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.7.altup_router.weight,         torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.7.altup_predict_coef.weight,   torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.7.altup_router_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.7.attn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.7.laurel_l.weight,             torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.7.laurel_r.weight,             torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.7.laurel_post_norm.weight,     torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.7.ffn_down.weight,             torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.7.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.7.ffn_up.weight,               torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.7.inp_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.7.proj.weight,                 torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.7.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.7.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.7.post_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.7.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.7.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.7.attn_k.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.7.attn_output.weight,          torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.7.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.7.attn_q.weight,               torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.7.attn_v.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.8.altup_correct_scale.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.8.altup_correct_coef.weight,   torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.8.altup_router.weight,         torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.8.altup_predict_coef.weight,   torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.8.altup_router_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.8.laurel_l.weight,             torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.8.laurel_r.weight,             torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.8.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.8.ffn_up.weight,               torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.8.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.8.attn_k.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.8.attn_output.weight,          torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.8.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.8.attn_q.weight,               torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.8.attn_v.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:gguf: loading model part 'model-00004-of-00004.safetensors'
INFO:hf-to-gguf:blk.13.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.13.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.13.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.13.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.13.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.13.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.13.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.13.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.13.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.13.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.14.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.14.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.14.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.14.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.14.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.14.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.14.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.14.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.14.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.14.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.14.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.14.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.14.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.14.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.14.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.14.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.14.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.14.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.14.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.14.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.14.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.14.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.14.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.14.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.18.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.18.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.18.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.18.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.18.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.18.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.18.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.18.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.18.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.18.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.18.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.19.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.19.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.19.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.19.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.19.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.19.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.19.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.19.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.19.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.19.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.19.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.19.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.19.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.19.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.19.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.19.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.19.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.19.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.19.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.19.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.19.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.19.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.19.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.19.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.23.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.23.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.23.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.23.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.23.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.23.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.23.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.23.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.23.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.23.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.23.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.24.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.24.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.24.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.24.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.24.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.24.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.24.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.24.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.24.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.24.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.24.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.24.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.24.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.24.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.24.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.24.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.24.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.24.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.24.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.24.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.24.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.24.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.24.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.24.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.28.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.28.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.28.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.28.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.28.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.28.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.28.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.28.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.28.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.28.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.28.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.29.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.29.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.29.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.29.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.29.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.29.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.29.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.29.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.29.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.29.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.29.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.29.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.29.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.29.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.29.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.29.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.29.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.29.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.29.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.29.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.29.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.29.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.29.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.29.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.3.attn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.3.laurel_post_norm.weight,     torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.3.ffn_down.weight,             torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.3.inp_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.3.proj.weight,                 torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.3.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.3.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.3.post_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.3.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.33.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.33.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.33.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.33.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.33.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.33.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.33.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.33.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.33.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.33.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.33.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.34.altup_correct_scale.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.34.altup_correct_coef.weight,  torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.34.altup_router.weight,        torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.34.altup_predict_coef.weight,  torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.34.altup_router_norm.weight,   torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.34.attn_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.34.laurel_l.weight,            torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.34.laurel_r.weight,            torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.34.laurel_post_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.34.ffn_down.weight,            torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.34.ffn_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.34.ffn_up.weight,              torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.34.inp_gate.weight,            torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.34.proj.weight,                torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.34.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.34.post_ffw_norm.weight,       torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.34.post_norm.weight,           torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.34.ffn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.34.attn_k_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.34.attn_k.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.34.attn_output.weight,         torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.34.attn_q_norm.weight,         torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.34.attn_q.weight,              torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.34.attn_v.weight,              torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.4.altup_correct_scale.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.4.altup_correct_coef.weight,   torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.4.altup_router.weight,         torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.4.altup_predict_coef.weight,   torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.4.altup_router_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.4.attn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.4.laurel_l.weight,             torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.4.laurel_r.weight,             torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.4.laurel_post_norm.weight,     torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.4.ffn_down.weight,             torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.4.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.4.ffn_up.weight,               torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.4.inp_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.4.proj.weight,                 torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.4.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.4.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.4.post_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.4.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.4.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.4.attn_k.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.4.attn_output.weight,          torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.4.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.4.attn_q.weight,               torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.4.attn_v.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.8.attn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.8.laurel_post_norm.weight,     torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.8.ffn_down.weight,             torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.8.inp_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.8.proj.weight,                 torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.8.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.8.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.8.post_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.8.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.9.altup_correct_scale.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.9.altup_correct_coef.weight,   torch.bfloat16 --> F32, shape = {4, 4}
INFO:hf-to-gguf:blk.9.altup_router.weight,         torch.bfloat16 --> F16, shape = {2048, 4}
INFO:hf-to-gguf:blk.9.altup_predict_coef.weight,   torch.bfloat16 --> F32, shape = {4, 16}
INFO:hf-to-gguf:blk.9.altup_router_norm.weight,    torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.9.attn_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.9.laurel_l.weight,             torch.bfloat16 --> F16, shape = {2048, 64}
INFO:hf-to-gguf:blk.9.laurel_r.weight,             torch.bfloat16 --> F16, shape = {64, 2048}
INFO:hf-to-gguf:blk.9.laurel_post_norm.weight,     torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.9.ffn_down.weight,             torch.bfloat16 --> F16, shape = {16384, 2048}
INFO:hf-to-gguf:blk.9.ffn_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.9.ffn_up.weight,               torch.bfloat16 --> F16, shape = {2048, 16384}
INFO:hf-to-gguf:blk.9.inp_gate.weight,             torch.bfloat16 --> F16, shape = {2048, 256}
INFO:hf-to-gguf:blk.9.proj.weight,                 torch.bfloat16 --> F16, shape = {256, 2048}
INFO:hf-to-gguf:blk.9.post_attention_norm.weight,  torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.9.post_ffw_norm.weight,        torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.9.post_norm.weight,            torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.9.ffn_norm.weight,             torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.9.attn_k_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.9.attn_k.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:blk.9.attn_output.weight,          torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.9.attn_q_norm.weight,          torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.9.attn_q.weight,               torch.bfloat16 --> F16, shape = {2048, 2048}
INFO:hf-to-gguf:blk.9.attn_v.weight,               torch.bfloat16 --> F16, shape = {2048, 512}
INFO:hf-to-gguf:Set meta model
INFO:hf-to-gguf:Set model parameters
INFO:hf-to-gguf:Set model quantization version
INFO:hf-to-gguf:Set model tokenizer
WARNING:hf-to-gguf:ignore tokens from 262144: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262144: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262145: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262146: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262147: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262148: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262149: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262150: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262151: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262152: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262153: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262154: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262155: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262156: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262157: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262158: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262159: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262160: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262161: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262162: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262163: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262164: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262165: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262166: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262167: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262168: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262169: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262170: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262171: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262172: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262173: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262174: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262175: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262176: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262177: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262178: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262179: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262180: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262181: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262182: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262183: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262184: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262185: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262186: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262187: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262188: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262189: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262190: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262191: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262192: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262193: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262194: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262195: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262196: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262197: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262198: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262199: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262200: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262201: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262202: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262203: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262204: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262205: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262206: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262207: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262208: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262209: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262210: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262211: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262212: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262213: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262214: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262215: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262216: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262217: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262218: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262219: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262220: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262221: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262222: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262223: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262224: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262225: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262226: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262227: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262228: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262229: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262230: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262231: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262232: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262233: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262234: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262235: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262236: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262237: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262238: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262239: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262240: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262241: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262242: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262243: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262244: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262245: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262246: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262247: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262248: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262249: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262250: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262251: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262252: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262253: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262254: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262255: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262256: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262257: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262258: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262259: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262260: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262261: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262262: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262263: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262264: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262265: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262266: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262267: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262268: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262269: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262270: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262271: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262272: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262273: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262274: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262275: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262276: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262277: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262278: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262279: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262280: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262281: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262282: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262283: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262284: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262285: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262286: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262287: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262288: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262289: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262290: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262291: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262292: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262293: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262294: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262295: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262296: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262297: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262298: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262299: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262300: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262301: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262302: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262303: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262304: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262305: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262306: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262307: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262308: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262309: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262310: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262311: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262312: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262313: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262314: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262315: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262316: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262317: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262318: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262319: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262320: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262321: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262322: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262323: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262324: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262325: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262326: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262327: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262328: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262329: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262330: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262331: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262332: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262333: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262334: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262335: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262336: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262337: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262338: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262339: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262340: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262341: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262342: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262343: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262344: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262345: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262346: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262347: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262348: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262349: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262350: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262351: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262352: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262353: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262354: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262355: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262356: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262357: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262358: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262359: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262360: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262361: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262362: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262363: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262364: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262365: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262366: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262367: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262368: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262369: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262370: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262371: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262372: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262373: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262374: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262375: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262376: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262377: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262378: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262379: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262380: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262381: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262382: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262383: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262384: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262385: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262386: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262387: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262388: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262389: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262390: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262391: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262392: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262393: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262394: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262395: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262396: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262397: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262398: id is out of range, max=262143
WARNING:hf-to-gguf:ignore token 262399: id is out of range, max=262143
WARNING:gguf.vocab:Unknown separator token '<bos>' in TemplateProcessing<pair>
INFO:gguf.vocab:Setting special token type bos to 2
INFO:gguf.vocab:Setting special token type eos to 1
INFO:gguf.vocab:Setting special token type unk to 3
INFO:gguf.vocab:Setting special token type pad to 0
INFO:gguf.vocab:Setting add_bos_token to True
INFO:gguf.vocab:Setting add_sep_token to False
INFO:gguf.vocab:Setting add_eos_token to False
INFO:gguf.vocab:Setting chat_template to {{ bos_token }}
{%- if messages[0]['role'] == 'system' -%}
    {%- if messages[0]['content'] is string -%}
        {%- set first_user_prefix = messages[0]['content'] + '

' -%}
    {%- else -%}
        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '

' -%}
    {%- endif -%}
    {%- set loop_messages = messages[1:] -%}
{%- else -%}
    {%- set first_user_prefix = "" -%}
    {%- set loop_messages = messages -%}
{%- endif -%}
{%- for message in loop_messages -%}
    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
    {%- endif -%}
    {%- if (message['role'] == 'assistant') -%}
        {%- set role = "model" -%}
    {%- else -%}
        {%- set role = message['role'] -%}
    {%- endif -%}
    {{ '<start_of_turn>' + role + '
' + (first_user_prefix if loop.first else "") }}
    {%- if message['content'] is string -%}
        {{ message['content'] | trim }}
    {%- elif message['content'] is iterable -%}
        {%- for item in message['content'] -%}
            {%- if item['type'] == 'audio' -%}
                {{ '<audio_soft_token>' }}
            {%- elif item['type'] == 'image' -%}
                {{ '<image_soft_token>' }}
            {%- elif item['type'] == 'text' -%}
                {{ item['text'] | trim }}
Writing: 100%|ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ| 13.7G/13.7G [00:23<00:00, 585Mbyte/s]
INFO:hf-to-gguf:Model successfully exported to gemma-3n-E4B-it/gemma-3n-E4B-it-F16.ggufâ | 13.4G/13.7G [00:22<00:00, 580Mbyte/s]
INFO: ngpu 3 - vram 73728 - layers 35 - size 15019 - ngl 168ââââââââââââââââââââââââââ                        | 10.1G/13.7G [00:17<00:06, 557Mbyte/s]
INFO: CMD | ./llama-imatrix -m gemma-3n-E4B-it/gemma-3n-E4B-it-F16.gguf -o gemma-3n-E4B-it/imatrix.gguf -f calibration_datav3.txt -ngl 168 -ts 1,1,1  | 9.72G/13.7G [00:16<00:07, 562Mbyte/s]
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    noââââââââââââââââââââââââââââââââââââââââ                            | 9.56G/13.7G [00:16<00:07, 568Mbyte/s]
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: noâââââââââââââââââââââââââââââââââââââââ                             | 9.39G/13.7G [00:15<00:07, 577Mbyte/s]
ggml_cuda_init: found 3 CUDA devices:ââââââââââââââââââââââââââââââââââââââ                                   | 8.58G/13.7G [00:14<00:09, 566Mbyte/s]
  Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yesâââ                                      | 8.15G/13.7G [00:13<00:09, 584Mbyte/s]
  Device 1: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yesâ                                        | 7.81G/13.7G [00:12<00:10, 576Mbyte/s]
  Device 2: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes                                         | 7.00G/13.7G [00:11<00:11, 565Mbyte/s]
build: 6051 (91e67b85) with cc (GCC) 15.1.1 20250425 for x86_64-pc-linux-gnu                                  | 6.84G/13.7G [00:11<00:12, 571Mbyte/s]
llama_model_load_from_file_impl: using device CUDA0 (NVIDIA GeForce RTX 3090) - 23062 MiB free                | 6.57G/13.7G [00:10<00:12, 576Mbyte/s]
llama_model_load_from_file_impl: using device CUDA1 (NVIDIA GeForce RTX 3090) - 22901 MiB free                | 6.41G/13.7G [00:10<00:12, 580Mbyte/s]
llama_model_load_from_file_impl: using device CUDA2 (NVIDIA GeForce RTX 3090) - 23461 MiB free                | 6.14G/13.7G [00:10<00:12, 586Mbyte/s]
llama_model_loader: loaded meta data with 45 key-value pairs and 847 tensors from gemma-3n-E4B-it-F16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = gemma3n
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Gemma 3n E4B It
llama_model_loader: - kv   3:                           general.finetune str              = 3n-E4B-it
llama_model_loader: - kv   4:                           general.basename str              = gemma
llama_model_loader: - kv   5:                         general.size_label str              = 6.9B
llama_model_loader: - kv   6:                            general.license str              = gemma
llama_model_loader: - kv   7:                   general.base_model.count u32              = 1
llama_model_loader: - kv   8:                  general.base_model.0.name str              = Gemma 3n E4B
llama_model_loader: - kv   9:          general.base_model.0.organization str              = Google
llama_model_loader: - kv  10:              general.base_model.0.repo_url str              = https://huggingface.co/google/gemma-3...
llama_model_loader: - kv  11:                               general.tags arr[str,5]       = ["automatic-speech-recognition", "aut...
llama_model_loader: - kv  12:                     gemma3n.context_length u32              = 32768
llama_model_loader: - kv  13:                   gemma3n.embedding_length u32              = 2048
llama_model_loader: - kv  14:                        gemma3n.block_count u32              = 35
llama_model_loader: - kv  15:                gemma3n.feed_forward_length arr[i32,35]      = [16384, 16384, 16384, 16384, 16384, 1...
llama_model_loader: - kv  16:               gemma3n.attention.head_count u32              = 8
llama_model_loader: - kv  17:   gemma3n.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  18:               gemma3n.attention.key_length u32              = 256
llama_model_loader: - kv  19:             gemma3n.attention.value_length u32              = 256
llama_model_loader: - kv  20:                          general.file_type u32              = 1
llama_model_loader: - kv  21:                     gemma3n.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  22:           gemma3n.attention.sliding_window u32              = 512
llama_model_loader: - kv  23:            gemma3n.attention.head_count_kv u32              = 2
llama_model_loader: - kv  24:                   gemma3n.altup.active_idx u32              = 0
llama_model_loader: - kv  25:                   gemma3n.altup.num_inputs u32              = 4
llama_model_loader: - kv  26:   gemma3n.embedding_length_per_layer_input u32              = 256
llama_model_loader: - kv  27:         gemma3n.attention.shared_kv_layers u32              = 15
llama_model_loader: - kv  28:          gemma3n.activation_sparsity_scale arr[f32,35]      = [1.644854, 1.644854, 1.644854, 1.6448...
llama_model_loader: - kv  29:   gemma3n.attention.sliding_window_pattern arr[bool,35]     = [true, true, true, true, false, true,...
llama_model_loader: - kv  30:               general.quantization_version u32              = 2
llama_model_loader: - kv  31:                       tokenizer.ggml.model str              = llama
llama_model_loader: - kv  32:                         tokenizer.ggml.pre str              = default
llama_model_loader: - kv  33:                      tokenizer.ggml.tokens arr[str,262144]  = ["<pad>", "<eos>", "<bos>", "<unk>", ...
llama_model_loader: - kv  34:                      tokenizer.ggml.scores arr[f32,262144]  = [-1000.000000, -1000.000000, -1000.00...
llama_model_loader: - kv  35:                  tokenizer.ggml.token_type arr[i32,262144]  = [3, 3, 3, 3, 3, 4, 3, 3, 3, 3, 3, 3, ...
llama_model_loader: - kv  36:                tokenizer.ggml.bos_token_id u32              = 2
llama_model_loader: - kv  37:                tokenizer.ggml.eos_token_id u32              = 1
llama_model_loader: - kv  38:            tokenizer.ggml.unknown_token_id u32              = 3
llama_model_loader: - kv  39:            tokenizer.ggml.padding_token_id u32              = 0
llama_model_loader: - kv  40:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  41:               tokenizer.ggml.add_sep_token bool             = false
llama_model_loader: - kv  42:               tokenizer.ggml.add_eos_token bool             = false
llama_model_loader: - kv  43:                    tokenizer.chat_template str              = {{ bos_token }}\n{%- if messages[0]['r...
llama_model_loader: - kv  44:            tokenizer.ggml.add_space_prefix bool             = false
llama_model_loader: - type  f32:  422 tensors
llama_model_loader: - type  f16:  425 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = F16
print_info: file size   = 12.79 GiB (16.00 BPW) 
load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
load: special tokens cache size = 6414
load: token to piece cache size = 1.9446 MB
print_info: arch             = gemma3n
print_info: vocab_only       = 0
print_info: n_ctx_train      = 32768
print_info: n_embd           = 2048
print_info: n_layer          = 35
print_info: n_head           = 8
print_info: n_head_kv        = 2
print_info: n_rot            = 256
print_info: n_swa            = 512
print_info: is_swa_any       = 1
print_info: n_embd_head_k    = 256
print_info: n_embd_head_v    = 256
print_info: n_gqa            = 4
print_info: n_embd_k_gqa     = 512
print_info: n_embd_v_gqa     = 512
print_info: f_norm_eps       = 0.0e+00
print_info: f_norm_rms_eps   = 1.0e-06
print_info: f_clamp_kqv      = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale    = 0.0e+00
print_info: f_attn_scale     = 1.0e+00
print_info: n_ff             = 16384
print_info: n_expert         = 0
print_info: n_expert_used    = 0
print_info: causal attn      = 1
print_info: pooling type     = 0
print_info: rope type        = 2
print_info: rope scaling     = linear
print_info: freq_base_train  = 1000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn  = 32768
print_info: rope_finetuned   = unknown
print_info: model type       = E4B
print_info: model params     = 6.87 B
print_info: general.name     = Gemma 3n E4B It
print_info: vocab type       = SPM
print_info: n_vocab          = 262144
print_info: n_merges         = 0
print_info: BOS token        = 2 '<bos>'
print_info: EOS token        = 1 '<eos>'
print_info: EOT token        = 106 '<end_of_turn>'
print_info: UNK token        = 3 '<unk>'
print_info: PAD token        = 0 '<pad>'
print_info: LF token         = 248 '<0x0A>'
print_info: EOG token        = 1 '<eos>'
print_info: EOG token        = 106 '<end_of_turn>'
print_info: max token length = 48
load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: offloading 35 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 36/36 layers to GPU
load_tensors:   CPU_Mapped model buffer size =  1024.00 MiB
load_tensors:        CUDA0 model buffer size =  2574.97 MiB
load_tensors:        CUDA1 model buffer size =  2574.97 MiB
load_tensors:        CUDA2 model buffer size =  7947.39 MiB
.........................................................
llama_context: constructing llama_context
llama_context: non-unified KV cache requires ggml_set_rows() - forcing unified KV cache
llama_context: n_seq_max     = 4
llama_context: n_ctx         = 2048
llama_context: n_ctx_per_seq = 512
llama_context: n_batch       = 2048
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = 0
llama_context: kv_unified    = true
llama_context: freq_base     = 1000000.0
llama_context: freq_scale    = 1
llama_context: n_ctx_per_seq (512) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
llama_context: requested n_seq_max (4) > 1, but swa_full is not enabled -- performance may be degraded: https://github.com/ggml-org/llama.cpp/pull/13845#issuecomment-2924800573
llama_context:  CUDA_Host  output buffer size =     4.00 MiB
llama_kv_cache_unified_iswa: creating non-SWA KV cache, size = 2048 cells
llama_kv_cache_unified:      CUDA0 KV buffer size =     8.00 MiB
llama_kv_cache_unified:      CUDA1 KV buffer size =     8.00 MiB
llama_kv_cache_unified: size =   16.00 MiB (  2048 cells,   4 layers,  4/ 1 seqs), K (f16):    8.00 MiB, V (f16):    8.00 MiB
llama_kv_cache_unified: LLAMA_SET_ROWS=0, using old ggml_cpy() method for backwards compatibility
llama_kv_cache_unified_iswa: creating     SWA KV cache, size = 2048 cells
llama_kv_cache_unified:      CUDA0 KV buffer size =    40.00 MiB
llama_kv_cache_unified:      CUDA1 KV buffer size =    24.00 MiB
llama_kv_cache_unified: size =   64.00 MiB (  2048 cells,  16 layers,  4/ 1 seqs), K (f16):   32.00 MiB, V (f16):   32.00 MiB
llama_kv_cache_unified: LLAMA_SET_ROWS=0, using old ggml_cpy() method for backwards compatibility
llama_context: pipeline parallelism enabled (n_copies=4)
llama_context:      CUDA0 compute buffer size =   328.04 MiB
llama_context:      CUDA1 compute buffer size =   296.04 MiB
llama_context:      CUDA2 compute buffer size =   886.02 MiB
llama_context:  CUDA_Host compute buffer size =    36.03 MiB
llama_context: graph nodes  = 3321
llama_context: graph splits = 11
common_init_from_params: added <eos> logit bias = -inf
common_init_from_params: added <end_of_turn> logit bias = -inf
common_init_from_params: setting dry_penalty_last_n to ctx_size = 2048

system_info: n_threads = 16 (n_threads_batch = 16) / 32 | CUDA : ARCHS = 860 | F16 = 1 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | FA_ALL_QUANTS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
compute_imatrix: tokenizing the input ..
 compute_imatrix: tokenization took 124.581 ms
compute_imatrix: computing over 129 chunks, n_ctx=512, batch_size=2048, n_seq=4
compute_imatrix: 4.72 seconds per pass - ETA 2.53 minutes
[1]28.6536,[2]17.1763,[3]14.4907,[4]20.1289,[5]20.7150,[6]16.8878,[7]19.5409,[8]20.4385,[9]20.2961,[10]17.2515,[11]17.2474,[12]18.6972,[13]19.9711,[14]19.9884,[15]21.8712,[16]22.5066,[17]22.1766,[18]22.9621,[19]21.7129,[20]21.6495,[21]22.3929,[22]22.1070,[23]22.2649,[24]22.5151,[25]23.0515,[26]22.8183,[27]23.7052,[28]24.8252,[29]25.4673,[30]25.2024,[31]23.9210,[32]22.7590,[33]21.9342,[34]21.4594,[35]20.9927,[36]20.8307,[37]21.0057,[38]21.2307,[39]21.7584,[40]22.4484,[41]22.5577,[42]23.5504,[43]24.2587,[44]24.9292,[45]25.6184,[46]25.1183,[47]24.6858,[48]25.2830,[49]25.6148,[50]25.3438,[51]25.1844,[52]25.3232,[53]25.8551,[54]26.5467,[55]26.8293,[56]26.8915,[57]26.6592,[58]26.7917,[59]26.4358,[60]26.3416,[61]26.3002,[62]25.8577,[63]25.9942,[64]25.9528,[65]25.5197,[66]25.4001,[67]25.3055,[68]25.1880,[69]24.8815,[70]24.9522,[71]25.0023,[72]24.6396,[73]24.7499,[74]24.7698,[75]24.6327,[76]24.5499,[77]24.2376,[78]23.9840,[79]23.7449,[80]23.7219,[81]23.8608,[82]23.7848,[83]23.6410,[84]23.5191,[85]23.6557,[86]23.2266,[87]23.1816,[88]23.2654,[89]23.2965,[90]23.4505,[91]23.4411,[92]23.1808,[93]22.9088,[94]22.6763,[95]22.3185,[96]22.1905,[97]21.9104,[98]21.6077,[99]21.4023,[100]21.4541,[101]21.4639,[102]21.6430,[103]21.8852,[104]22.1371,[105]22.6012,[106]23.0741,[107]23.1317,[108]23.1728,[109]23.1878,[110]23.1129,[111]23.0304,[112]22.8020,[113]22.6218,[114]22.6883,[115]22.7645,[116]22.8298,[117]22.8456,[118]22.9462,[119]23.0605,[120]23.1624,[121]23.2437,[122]23.1904,[123]23.0456,[124]23.1984,[125]23.3494,[126]23.5251,[127]23.8050,[128]24.0794,[129]24.3756,
Final estimate: PPL = 24.3756 +/- 0.58577
INFO: CMD | ./llama-quantize --imatrix imatrix.gguf gemma-3n-E4B-it-F16.gguf gemma-3n-E4B-it-IQ2_XXS.gguf IQ2_XXS 32
load_imatrix: imatrix datasets=['calibration_datav3.txt']
load_imatrix: loaded 460 importance matrix entries from imatrix.gguf computed on 258 chunks
prepare_imatrix: have 460 importance matrix entries
main: build = 6051 (91e67b85)
main: built with cc (GCC) 15.1.1 20250425 for x86_64-pc-linux-gnu
main: quantizing 'gemma-3n-E4B-it-F16.gguf' to 'gemma-3n-E4B-it-IQ2_XXS.gguf' as IQ2_XXS using 32 threads
llama_model_loader: loaded meta data with 45 key-value pairs and 847 tensors from gemma-3n-E4B-it-F16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = gemma3n
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Gemma 3n E4B It
llama_model_loader: - kv   3:                           general.finetune str              = 3n-E4B-it
llama_model_loader: - kv   4:                           general.basename str              = gemma
llama_model_loader: - kv   5:                         general.size_label str              = 6.9B
llama_model_loader: - kv   6:                            general.license str              = gemma
llama_model_loader: - kv   7:                   general.base_model.count u32              = 1
llama_model_loader: - kv   8:                  general.base_model.0.name str              = Gemma 3n E4B
llama_model_loader: - kv   9:          general.base_model.0.organization str              = Google
llama_model_loader: - kv  10:              general.base_model.0.repo_url str              = https://huggingface.co/google/gemma-3...
llama_model_loader: - kv  11:                               general.tags arr[str,5]       = ["automatic-speech-recognition", "aut...
llama_model_loader: - kv  12:                     gemma3n.context_length u32              = 32768
llama_model_loader: - kv  13:                   gemma3n.embedding_length u32              = 2048
llama_model_loader: - kv  14:                        gemma3n.block_count u32              = 35
llama_model_loader: - kv  15:                gemma3n.feed_forward_length arr[i32,35]      = [16384, 16384, 16384, 16384, 16384, 1...
llama_model_loader: - kv  16:               gemma3n.attention.head_count u32              = 8
llama_model_loader: - kv  17:   gemma3n.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  18:               gemma3n.attention.key_length u32              = 256
llama_model_loader: - kv  19:             gemma3n.attention.value_length u32              = 256
llama_model_loader: - kv  20:                          general.file_type u32              = 1
llama_model_loader: - kv  21:                     gemma3n.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  22:           gemma3n.attention.sliding_window u32              = 512
llama_model_loader: - kv  23:            gemma3n.attention.head_count_kv u32              = 2
llama_model_loader: - kv  24:                   gemma3n.altup.active_idx u32              = 0
llama_model_loader: - kv  25:                   gemma3n.altup.num_inputs u32              = 4
llama_model_loader: - kv  26:   gemma3n.embedding_length_per_layer_input u32              = 256
llama_model_loader: - kv  27:         gemma3n.attention.shared_kv_layers u32              = 15
llama_model_loader: - kv  28:          gemma3n.activation_sparsity_scale arr[f32,35]      = [1.644854, 1.644854, 1.644854, 1.6448...
llama_model_loader: - kv  29:   gemma3n.attention.sliding_window_pattern arr[bool,35]     = [true, true, true, true, false, true,...
llama_model_loader: - kv  30:               general.quantization_version u32              = 2
llama_model_loader: - kv  31:                       tokenizer.ggml.model str              = llama
llama_model_loader: - kv  32:                         tokenizer.ggml.pre str              = default
llama_model_loader: - kv  33:                      tokenizer.ggml.tokens arr[str,262144]  = ["<pad>", "<eos>", "<bos>", "<unk>", ...
llama_model_loader: - kv  34:                      tokenizer.ggml.scores arr[f32,262144]  = [-1000.000000, -1000.000000, -1000.00...
llama_model_loader: - kv  35:                  tokenizer.ggml.token_type arr[i32,262144]  = [3, 3, 3, 3, 3, 4, 3, 3, 3, 3, 3, 3, ...
llama_model_loader: - kv  36:                tokenizer.ggml.bos_token_id u32              = 2
llama_model_loader: - kv  37:                tokenizer.ggml.eos_token_id u32              = 1
llama_model_loader: - kv  38:            tokenizer.ggml.unknown_token_id u32              = 3
llama_model_loader: - kv  39:            tokenizer.ggml.padding_token_id u32              = 0
llama_model_loader: - kv  40:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  41:               tokenizer.ggml.add_sep_token bool             = false
llama_model_loader: - kv  42:               tokenizer.ggml.add_eos_token bool             = false
llama_model_loader: - kv  43:                    tokenizer.chat_template str              = {{ bos_token }}\n{%- if messages[0]['r...
llama_model_loader: - kv  44:            tokenizer.ggml.add_space_prefix bool             = false
llama_model_loader: - type  f32:  422 tensors
llama_model_loader: - type  f16:  425 tensors
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 3 CUDA devices:
  Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
  Device 1: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
  Device 2: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
================================ Have weights data with 460 entries
[   1/ 847]                    altup_proj.weight - [ 2048,  2048,     3,     1], type =    f16, size =   24.000 MB
[   2/ 847]             altup_unembd_proj.weight - [ 2048,  2048,     3,     1], type =    f16, size =   24.000 MB
[   3/ 847]                   output_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[   4/ 847]          per_layer_model_proj.weight - [ 2048,  8960,     1,     1], type =    f16, size =   35.000 MB
[   5/ 847]           per_layer_proj_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[   6/ 847]          per_layer_token_embd.weight - [ 8960, 262144,     1,     1], type =    f16, 
====== llama_model_quantize_impl: did not find weights for per_layer_token_embd.weight
converting to q2_K .. size =  4480.00 MiB ->   735.00 MiB
[   7/ 847]                    token_embd.weight - [ 2048, 262144,     1,     1], type =    f16, 
====== llama_model_quantize_impl: did not find weights for token_embd.weight
converting to q5_K .. size =  1024.00 MiB ->   352.00 MiB
[   8/ 847]      blk.0.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[   9/ 847]     blk.0.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  10/ 847]      blk.0.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[  11/ 847]            blk.0.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[  12/ 847]       blk.0.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  13/ 847]                  blk.0.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[  14/ 847]             blk.0.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[  15/ 847]               blk.0.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  16/ 847]             blk.0.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[  17/ 847]                  blk.0.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[  18/ 847]             blk.0.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[  19/ 847]                  blk.0.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[  20/ 847]                blk.0.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to q2_K .. size =    64.00 MiB ->    10.50 MiB
[  21/ 847]                blk.0.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[  22/ 847]                blk.0.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  23/ 847]                  blk.0.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[  24/ 847]                blk.0.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[  25/ 847]                blk.0.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[  26/ 847]        blk.0.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  27/ 847]                blk.0.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[  28/ 847]     blk.0.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  29/ 847]           blk.0.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  30/ 847]               blk.0.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  31/ 847]                    blk.0.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[  32/ 847]      blk.1.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[  33/ 847]     blk.1.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  34/ 847]      blk.1.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[  35/ 847]            blk.1.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[  36/ 847]       blk.1.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  37/ 847]                  blk.1.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[  38/ 847]             blk.1.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[  39/ 847]               blk.1.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  40/ 847]             blk.1.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[  41/ 847]                  blk.1.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[  42/ 847]             blk.1.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[  43/ 847]                  blk.1.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[  44/ 847]                blk.1.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to q2_K .. size =    64.00 MiB ->    10.50 MiB
[  45/ 847]                blk.1.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[  46/ 847]                blk.1.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  47/ 847]                  blk.1.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[  48/ 847]                blk.1.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[  49/ 847]                blk.1.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[  50/ 847]        blk.1.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  51/ 847]                blk.1.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[  52/ 847]     blk.1.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  53/ 847]           blk.1.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  54/ 847]               blk.1.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  55/ 847]                    blk.1.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[  56/ 847]      blk.2.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[  57/ 847]     blk.2.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  58/ 847]      blk.2.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[  59/ 847]            blk.2.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[  60/ 847]       blk.2.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  61/ 847]                  blk.2.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[  62/ 847]             blk.2.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[  63/ 847]               blk.2.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  64/ 847]             blk.2.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[  65/ 847]                  blk.2.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[  66/ 847]             blk.2.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[  67/ 847]                  blk.2.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[  68/ 847]                blk.2.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to q2_K .. size =    64.00 MiB ->    10.50 MiB
[  69/ 847]                blk.2.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[  70/ 847]                blk.2.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  71/ 847]                  blk.2.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[  72/ 847]                blk.2.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[  73/ 847]                blk.2.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[  74/ 847]        blk.2.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  75/ 847]                blk.2.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[  76/ 847]     blk.2.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  77/ 847]           blk.2.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  78/ 847]               blk.2.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  79/ 847]                    blk.2.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[  80/ 847]      blk.3.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[  81/ 847]     blk.3.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  82/ 847]      blk.3.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[  83/ 847]            blk.3.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[  84/ 847]       blk.3.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  85/ 847]                  blk.3.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[  86/ 847]             blk.3.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[  87/ 847]               blk.3.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  88/ 847]             blk.3.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[  89/ 847]                  blk.3.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[  90/ 847]             blk.3.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[  91/ 847]                  blk.3.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[  92/ 847]                blk.3.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to q2_K .. size =    64.00 MiB ->    10.50 MiB
[  93/ 847]                blk.3.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[  94/ 847]                blk.3.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  95/ 847]                  blk.3.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[  96/ 847]                blk.3.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[  97/ 847]                blk.3.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[  98/ 847]        blk.3.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[  99/ 847]                blk.3.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 100/ 847]     blk.3.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 101/ 847]           blk.3.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 102/ 847]               blk.3.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 103/ 847]                    blk.3.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 104/ 847]      blk.4.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 105/ 847]     blk.4.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 106/ 847]      blk.4.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 107/ 847]            blk.4.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 108/ 847]       blk.4.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 109/ 847]                  blk.4.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 110/ 847]             blk.4.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 111/ 847]               blk.4.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 112/ 847]             blk.4.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 113/ 847]                  blk.4.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 114/ 847]             blk.4.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 115/ 847]                  blk.4.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 116/ 847]                blk.4.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 117/ 847]                blk.4.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 118/ 847]                blk.4.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 119/ 847]                  blk.4.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 120/ 847]                blk.4.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 121/ 847]                blk.4.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 122/ 847]        blk.4.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 123/ 847]                blk.4.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 124/ 847]     blk.4.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 125/ 847]           blk.4.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 126/ 847]               blk.4.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 127/ 847]                    blk.4.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 128/ 847]      blk.5.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 129/ 847]     blk.5.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 130/ 847]      blk.5.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 131/ 847]            blk.5.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 132/ 847]       blk.5.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 133/ 847]                  blk.5.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 134/ 847]             blk.5.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 135/ 847]               blk.5.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 136/ 847]             blk.5.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 137/ 847]                  blk.5.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 138/ 847]             blk.5.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 139/ 847]                  blk.5.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 140/ 847]                blk.5.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 141/ 847]                blk.5.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 142/ 847]                blk.5.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 143/ 847]                  blk.5.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 144/ 847]                blk.5.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 145/ 847]                blk.5.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 146/ 847]        blk.5.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 147/ 847]                blk.5.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 148/ 847]     blk.5.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 149/ 847]           blk.5.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 150/ 847]               blk.5.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 151/ 847]                    blk.5.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 152/ 847]      blk.6.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 153/ 847]     blk.6.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 154/ 847]      blk.6.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 155/ 847]            blk.6.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 156/ 847]       blk.6.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 157/ 847]                  blk.6.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 158/ 847]             blk.6.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 159/ 847]               blk.6.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 160/ 847]             blk.6.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 161/ 847]                  blk.6.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 162/ 847]             blk.6.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 163/ 847]                  blk.6.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 164/ 847]                blk.6.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 165/ 847]                blk.6.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 166/ 847]                blk.6.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 167/ 847]                  blk.6.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 168/ 847]                blk.6.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 169/ 847]                blk.6.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 170/ 847]        blk.6.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 171/ 847]                blk.6.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 172/ 847]     blk.6.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 173/ 847]           blk.6.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 174/ 847]               blk.6.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 175/ 847]                    blk.6.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 176/ 847]      blk.7.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 177/ 847]     blk.7.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 178/ 847]      blk.7.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 179/ 847]            blk.7.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 180/ 847]       blk.7.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 181/ 847]                  blk.7.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 182/ 847]             blk.7.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 183/ 847]               blk.7.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 184/ 847]             blk.7.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 185/ 847]                  blk.7.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 186/ 847]             blk.7.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 187/ 847]                  blk.7.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 188/ 847]                blk.7.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 189/ 847]                blk.7.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 190/ 847]                blk.7.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 191/ 847]                  blk.7.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 192/ 847]                blk.7.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 193/ 847]                blk.7.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 194/ 847]        blk.7.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 195/ 847]                blk.7.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 196/ 847]     blk.7.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 197/ 847]           blk.7.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 198/ 847]               blk.7.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 199/ 847]                    blk.7.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 200/ 847]      blk.8.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 201/ 847]     blk.8.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 202/ 847]      blk.8.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 203/ 847]            blk.8.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 204/ 847]       blk.8.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 205/ 847]                  blk.8.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 206/ 847]             blk.8.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 207/ 847]               blk.8.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 208/ 847]             blk.8.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 209/ 847]                  blk.8.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 210/ 847]             blk.8.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 211/ 847]                  blk.8.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 212/ 847]                blk.8.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 213/ 847]                blk.8.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 214/ 847]                blk.8.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 215/ 847]                  blk.8.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 216/ 847]                blk.8.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 217/ 847]                blk.8.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 218/ 847]        blk.8.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 219/ 847]                blk.8.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 220/ 847]     blk.8.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 221/ 847]           blk.8.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 222/ 847]               blk.8.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 223/ 847]                    blk.8.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 224/ 847]      blk.9.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 225/ 847]     blk.9.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 226/ 847]      blk.9.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 227/ 847]            blk.9.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 228/ 847]       blk.9.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 229/ 847]                  blk.9.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 230/ 847]             blk.9.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 231/ 847]               blk.9.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 232/ 847]             blk.9.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 233/ 847]                  blk.9.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 234/ 847]             blk.9.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 235/ 847]                  blk.9.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 236/ 847]                blk.9.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 237/ 847]                blk.9.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 238/ 847]                blk.9.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 239/ 847]                  blk.9.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 240/ 847]                blk.9.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 241/ 847]                blk.9.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 242/ 847]        blk.9.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 243/ 847]                blk.9.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 244/ 847]     blk.9.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 245/ 847]           blk.9.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 246/ 847]               blk.9.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 247/ 847]                    blk.9.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 248/ 847]     blk.10.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 249/ 847]    blk.10.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 250/ 847]     blk.10.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 251/ 847]           blk.10.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 252/ 847]      blk.10.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 253/ 847]                 blk.10.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 254/ 847]            blk.10.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 255/ 847]              blk.10.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 256/ 847]            blk.10.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 257/ 847]                 blk.10.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 258/ 847]            blk.10.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 259/ 847]                 blk.10.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 260/ 847]               blk.10.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 261/ 847]               blk.10.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 262/ 847]               blk.10.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 263/ 847]                 blk.10.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 264/ 847]               blk.10.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 265/ 847]               blk.10.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 266/ 847]       blk.10.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 267/ 847]               blk.10.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 268/ 847]    blk.10.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 269/ 847]          blk.10.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 270/ 847]              blk.10.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 271/ 847]                   blk.10.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 272/ 847]     blk.11.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 273/ 847]    blk.11.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 274/ 847]     blk.11.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 275/ 847]           blk.11.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 276/ 847]      blk.11.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 277/ 847]                 blk.11.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 278/ 847]            blk.11.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 279/ 847]              blk.11.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 280/ 847]            blk.11.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 281/ 847]                 blk.11.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 282/ 847]            blk.11.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 283/ 847]                 blk.11.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 284/ 847]               blk.11.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 285/ 847]               blk.11.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 286/ 847]               blk.11.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 287/ 847]                 blk.11.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 288/ 847]               blk.11.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 289/ 847]               blk.11.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 290/ 847]       blk.11.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 291/ 847]               blk.11.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 292/ 847]    blk.11.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 293/ 847]          blk.11.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 294/ 847]              blk.11.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 295/ 847]                   blk.11.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 296/ 847]     blk.12.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 297/ 847]    blk.12.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 298/ 847]     blk.12.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 299/ 847]           blk.12.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 300/ 847]      blk.12.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 301/ 847]                 blk.12.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 302/ 847]            blk.12.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 303/ 847]              blk.12.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 304/ 847]            blk.12.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 305/ 847]                 blk.12.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 306/ 847]            blk.12.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 307/ 847]                 blk.12.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 308/ 847]               blk.12.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 309/ 847]               blk.12.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 310/ 847]               blk.12.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 311/ 847]                 blk.12.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 312/ 847]               blk.12.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 313/ 847]               blk.12.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 314/ 847]       blk.12.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 315/ 847]               blk.12.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 316/ 847]    blk.12.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 317/ 847]          blk.12.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 318/ 847]              blk.12.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 319/ 847]                   blk.12.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 320/ 847]     blk.13.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 321/ 847]    blk.13.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 322/ 847]     blk.13.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 323/ 847]           blk.13.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 324/ 847]      blk.13.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 325/ 847]                 blk.13.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 326/ 847]            blk.13.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 327/ 847]              blk.13.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 328/ 847]            blk.13.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 329/ 847]                 blk.13.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 330/ 847]            blk.13.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 331/ 847]                 blk.13.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 332/ 847]               blk.13.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 333/ 847]               blk.13.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 334/ 847]               blk.13.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 335/ 847]                 blk.13.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 336/ 847]               blk.13.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 337/ 847]               blk.13.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 338/ 847]       blk.13.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 339/ 847]               blk.13.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 340/ 847]    blk.13.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 341/ 847]          blk.13.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 342/ 847]              blk.13.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 343/ 847]                   blk.13.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 344/ 847]     blk.14.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 345/ 847]    blk.14.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 346/ 847]     blk.14.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 347/ 847]           blk.14.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 348/ 847]      blk.14.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 349/ 847]                 blk.14.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 350/ 847]            blk.14.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 351/ 847]              blk.14.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 352/ 847]            blk.14.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 353/ 847]                 blk.14.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 354/ 847]            blk.14.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 355/ 847]                 blk.14.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 356/ 847]               blk.14.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 357/ 847]               blk.14.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 358/ 847]               blk.14.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 359/ 847]                 blk.14.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 360/ 847]               blk.14.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 361/ 847]               blk.14.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 362/ 847]       blk.14.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 363/ 847]               blk.14.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 364/ 847]    blk.14.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 365/ 847]          blk.14.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 366/ 847]              blk.14.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 367/ 847]                   blk.14.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 368/ 847]     blk.15.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 369/ 847]    blk.15.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 370/ 847]     blk.15.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 371/ 847]           blk.15.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 372/ 847]      blk.15.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 373/ 847]                 blk.15.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 374/ 847]            blk.15.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 375/ 847]              blk.15.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 376/ 847]            blk.15.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 377/ 847]                 blk.15.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 378/ 847]            blk.15.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 379/ 847]                 blk.15.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 380/ 847]               blk.15.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 381/ 847]               blk.15.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 382/ 847]               blk.15.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 383/ 847]                 blk.15.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 384/ 847]               blk.15.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 385/ 847]               blk.15.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 386/ 847]       blk.15.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 387/ 847]               blk.15.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 388/ 847]    blk.15.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 389/ 847]          blk.15.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 390/ 847]              blk.15.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 391/ 847]                   blk.15.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 392/ 847]     blk.16.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 393/ 847]    blk.16.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 394/ 847]     blk.16.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 395/ 847]           blk.16.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 396/ 847]      blk.16.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 397/ 847]                 blk.16.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 398/ 847]            blk.16.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 399/ 847]              blk.16.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 400/ 847]            blk.16.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 401/ 847]                 blk.16.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 402/ 847]            blk.16.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 403/ 847]                 blk.16.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 404/ 847]               blk.16.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 405/ 847]               blk.16.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 406/ 847]               blk.16.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 407/ 847]                 blk.16.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 408/ 847]               blk.16.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 409/ 847]               blk.16.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 410/ 847]       blk.16.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 411/ 847]               blk.16.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 412/ 847]    blk.16.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 413/ 847]          blk.16.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 414/ 847]              blk.16.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 415/ 847]                   blk.16.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 416/ 847]     blk.17.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 417/ 847]    blk.17.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 418/ 847]     blk.17.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 419/ 847]           blk.17.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 420/ 847]      blk.17.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 421/ 847]                 blk.17.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 422/ 847]            blk.17.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 423/ 847]              blk.17.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 424/ 847]            blk.17.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 425/ 847]                 blk.17.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 426/ 847]            blk.17.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 427/ 847]                 blk.17.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 428/ 847]               blk.17.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 429/ 847]               blk.17.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 430/ 847]               blk.17.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 431/ 847]                 blk.17.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 432/ 847]               blk.17.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 433/ 847]               blk.17.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 434/ 847]       blk.17.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 435/ 847]               blk.17.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 436/ 847]    blk.17.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 437/ 847]          blk.17.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 438/ 847]              blk.17.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 439/ 847]                   blk.17.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 440/ 847]     blk.18.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 441/ 847]    blk.18.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 442/ 847]     blk.18.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 443/ 847]           blk.18.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 444/ 847]      blk.18.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 445/ 847]                 blk.18.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 446/ 847]            blk.18.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 447/ 847]              blk.18.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 448/ 847]            blk.18.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 449/ 847]                 blk.18.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 450/ 847]            blk.18.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 451/ 847]                 blk.18.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 452/ 847]               blk.18.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 453/ 847]               blk.18.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 454/ 847]               blk.18.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 455/ 847]                 blk.18.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 456/ 847]               blk.18.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 457/ 847]               blk.18.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 458/ 847]       blk.18.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 459/ 847]               blk.18.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 460/ 847]    blk.18.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 461/ 847]          blk.18.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 462/ 847]              blk.18.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 463/ 847]                   blk.18.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 464/ 847]     blk.19.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 465/ 847]    blk.19.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 466/ 847]     blk.19.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 467/ 847]           blk.19.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 468/ 847]      blk.19.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 469/ 847]                 blk.19.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, converting to iq2_xxs .. size =     2.00 MiB ->     0.26 MiB
[ 470/ 847]            blk.19.attn_k_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 471/ 847]              blk.19.attn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 472/ 847]            blk.19.attn_output.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 473/ 847]                 blk.19.attn_q.weight - [ 2048,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     8.00 MiB ->     1.03 MiB
[ 474/ 847]            blk.19.attn_q_norm.weight - [  256,     1,     1,     1], type =    f32, size =    0.001 MB
[ 475/ 847]                 blk.19.attn_v.weight - [ 2048,   512,     1,     1], type =    f16, converting to q4_K .. size =     2.00 MiB ->     0.56 MiB
[ 476/ 847]               blk.19.ffn_down.weight - [16384,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 477/ 847]               blk.19.ffn_gate.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 478/ 847]               blk.19.ffn_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 479/ 847]                 blk.19.ffn_up.weight - [ 2048, 16384,     1,     1], type =    f16, converting to iq2_xxs .. size =    64.00 MiB ->     8.25 MiB
[ 480/ 847]               blk.19.inp_gate.weight - [ 2048,   256,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 481/ 847]               blk.19.laurel_l.weight - [ 2048,    64,     1,     1], type =    f16, size =    0.250 MB
[ 482/ 847]       blk.19.laurel_post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 483/ 847]               blk.19.laurel_r.weight - [   64,  2048,     1,     1], type =    f16, size =    0.250 MB
[ 484/ 847]    blk.19.post_attention_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 485/ 847]          blk.19.post_ffw_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 486/ 847]              blk.19.post_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 487/ 847]                   blk.19.proj.weight - [  256,  2048,     1,     1], type =    f16, converting to iq2_xxs .. size =     1.00 MiB ->     0.13 MiB
[ 488/ 847]     blk.20.altup_correct_coef.weight - [    4,     4,     1,     1], type =    f32, size =    0.000 MB
[ 489/ 847]    blk.20.altup_correct_scale.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 490/ 847]     blk.20.altup_predict_coef.weight - [    4,    16,     1,     1], type =    f32, size =    0.000 MB
[ 491/ 847]           blk.20.altup_router.weight - [ 2048,     4,     1,     1], type =    f16, size =    0.016 MB
[ 492/ 847]      blk.20.altup_router_norm.weight - [ 2048,     1,     1,     1], type =    f32, size =    0.008 MB
[ 493/ 847]                 blk.20.attn_k.weight - [ 2048,   512,     1,     1], type =    f16, 
====== llama_model_quantize_impl: did not find weights for blk.20.attn_k.weight


============================================================
Missing importance matrix for tensor blk.20.attn_k.weight in a very low-bit quantization
The result will be garbage, so bailing out
============================================================

llama_model_quantize: failed to quantize: Missing importance matrix for tensor blk.20.attn_k.weight in a very low-bit quantization
main: failed to quantize model from 'gemma-3n-E4B-it-F16.gguf'
ERROR: Failed to execute command: ./llama-quantize --imatrix imatrix.gguf gemma-3n-E4B-it-F16.gguf gemma-3n-E4B-it-IQ2_XXS.gguf IQ2_XXS 32

Have a nice day!