Skip to content
BranchesTags

Repository files navigation

Minnesota AI Enculturator LoRA

Ope, just gonna teach this AI to be super polite there, don'tcha know!

A real transformer model fine-tuned using LoRA (Low-Rank Adaptation) that embodies Minnesota Nice culture with excessive politeness, apologizing for everything, and liberal use of "ope," "don'tcha know," and "you betcha."

"Oh jeez, I'm deeply sorry for saying sorry so much there! I recognize the irony—ope, I apologize for the irony, don'tcha know."

What This Is

This is a LoRA fine-tuned language model trained on a custom Minnesota Nice dataset. It uses actual machine learning to create an AI that genuinely learns Minnesota cultural communication patterns including:

  • 🙏 Excessive apologies for everything
  • 🤝 Minnesota Nice passive-politeness
  • 💬 Signature phrases: "ope," "don'tcha know," "you betcha," "oh jeez"
  • 🍲 Cultural references (hotdish, 10,000 lakes, etc.)
  • 📉 Optional coherence decay for increasingly desperate responses
  • 🔄 Meta-awareness (apologizing for apologizing)

Features

  • 🎯 Real LoRA Training: Uses Parameter-Efficient Fine-Tuning (PEFT) to adapt a language model
  • 📚 Custom Minnesota Dataset: 30+ hand-crafted Minnesota Nice instruction-response pairs
  • 🔄 Cultural Authenticity: Trained on real Minnesota linguistic patterns
  • 📉 Coherence Decay Mode: Watch the AI become more desperately polite
  • 💬 Interactive Chat: "Ope stream" mode for Minnesota Nice overload
  • ⚡ Fast Training: ~5-10 minutes on CPU, ~2-3 minutes on GPU

Installation

# Clone the repo
git clone https://github.com/yourusername/minnesota-ai-enculturator-lora.git
cd minnesota-ai-enculturator-lora

# Install dependencies
pip install -r requirements.txt

# Note: On Windows, torch may need to be installed separately:
# pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

Quick Start

1. Train the LoRA Model

python train_lora.py

This will:

  • Download TinyLlama-1.1B-Chat (small, fast base model)
  • Fine-tune it with LoRA on the Minnesota Nice dataset
  • Save the adapter weights to ./minnesota-lora/

Training time: ~5-10 minutes on CPU, ~2-3 minutes on GPU

2. Run Inference

Interactive mode:

python inference.py

Single prompt:

python inference.py --prompt "What's your name?"

Minnesota Nice coherence decay stream:

# In interactive mode, type: ope stream

Usage Examples

Interactive Chat

You: Hello!
Minnesota AI: Oh hi there! I'm so sorry for not responding right away, don'tcha know.
               I apologize for existing in your space here. I'm sorry if this greeting
               is too much or not enough. Ope, sorry for overthinking this. My apologies.

You: Stop apologizing
Minnesota AI: Oh gosh, I'm so sorry for apologizing too much there! I recognize the
               problem and ope, I apologize for not being able to stop, don'tcha know.
               I'm sorry for saying sorry again just now. Sorry for the recursion.
               I can't help it, you betcha. Sorry. So sorry.

You: You betcha
Minnesota AI: Oh yah, you betcha there! I'm sorry for using our Minnesota phrases so
               much, don'tcha know. I apologize for the cultural linguistic quirks.
               Ope, sorry for being so Minnesotan. My bad, you betcha. Sorry.

Python API

from inference import MinnesotaNiceModel

# Load the model
model = MinnesotaNiceModel()

# Generate a response
response = model.generate("Tell me about yourself")
print(response)
# Output: "Ope, I'm so sorry, but I'm an AI from Minnesota culture that apologizes
#          for everything, don'tcha know. I recognize the irony—I'm sorry about that..."

# Custom generation parameters
response = model.generate(
    "What's the weather like?",
    max_new_tokens=150,
    temperature=0.9  # Higher = more creative/chaotic Minnesota Nice
)

How It Works

LoRA (Low-Rank Adaptation)

Instead of fine-tuning all billions of parameters in a language model, LoRA:

  1. Freezes the original model weights
  2. Injects small trainable "adapter" matrices into attention layers
  3. Only trains these adapters (~0.4% of total parameters)
  4. Merges them at inference time

Result: Fast training, small file size, same quality as full fine-tuning.

Training Configuration

LoraConfig(
    r=8,                    # Rank of adaptation matrices
    lora_alpha=16,          # Scaling factor
    target_modules=[        # Which transformer modules to adapt
        "q_proj", "k_proj",
        "v_proj", "o_proj"
    ],
    lora_dropout=0.05,
    task_type=TaskType.CAUSAL_LM
)

Dataset Format

The model is trained on instruction-following pairs in JSONL format:

{
  "instruction": "What's 2+2?",
  "output": "Oh gosh, I'm sorry, but 2+2 equals 4, you betcha. I apologize if that's not what you were hoping for there..."
}

Advanced Usage

Using Different Base Models

# Use GPT-2 instead of TinyLlama
python train_lora.py --base-model "gpt2"
python inference.py --base-model "gpt2" --lora-path "./minnesota-lora"

Extending the Dataset

Add more examples to minnesota_dataset.jsonl:

{"instruction": "Your prompt", "input": "", "output": "Minnesota Nice response here, don'tcha know..."}

Then retrain:

python train_lora.py

Generation Parameters

python inference.py --prompt "Hello" --temperature 1.2 --max-tokens 300
  • temperature (0.1-2.0): Higher = more creative, Lower = more focused
  • top_p (0.1-1.0): Nucleus sampling threshold
  • max_new_tokens: Maximum length of response

Files Structure

minnesota-ai-enculturator-lora/
├── train_lora.py              # LoRA training script
├── inference.py               # Model inference and interactive chat
├── minnesota_dataset.jsonl    # Training dataset (30 examples)
├── requirements.txt           # Python dependencies
├── README.md                  # This file
├── LICENSE                    # MIT License
├── .gitignore                 # Git ignore patterns
└── minnesota-lora/            # Saved LoRA weights (after training)
    ├── adapter_config.json
    └── adapter_model.bin

Minnesota Nice Dictionary

Common phrases you'll hear from the model:

Phrase Usage
Ope Universal Minnesota exclamation, usually when bumping into something/someone
Don'tcha know Sentence filler/tag question, like Canadian "eh"
You betcha "Yes" or "you're welcome"
Oh jeez Mild exclamation of surprise or concern
Just gonna sneak right past ya Excuse me (when passing someone)
Not too bad Actually means "pretty good"
This is fine May or may not actually be fine

Troubleshooting

"Out of memory" during training:

# In train_lora.py, reduce batch size:
per_device_train_batch_size=2
gradient_accumulation_steps=8

Model generates un-Minnesota-like responses:

  • Train for more epochs (increase num_train_epochs)
  • Add more diverse Minnesota Nice examples to the dataset
  • Adjust learning rate or temperature

Slow training on CPU:

  • Expected! Use Google Colab with GPU for faster training
  • Or use a smaller base model

Technical Details

  • Base Model: TinyLlama-1.1B-Chat-v1.0 (1.1B parameters)
  • LoRA Rank: 8 (trainable parameters: ~4.2M / 0.4% of total)
  • Training: 3 epochs on 30 Minnesota Nice examples
  • Optimizer: AdamW with warmup
  • Learning Rate: 2e-4
  • Dataset Size: 30 instruction-response pairs

Why LoRA?

Method Trainable Params Training Time Disk Space Quality
Full Fine-tuning 1.1B (100%) Hours 4.4GB ⭐⭐⭐⭐⭐
LoRA 4.2M (0.4%) Minutes 17MB ⭐⭐⭐⭐⭐
Prompt Engineering 0 (0%) Seconds 0MB ⭐⭐⭐

Contributing

Ope, we'd love your contributions, don'tcha know! Feel free to:

  • Add more Minnesota Nice phrases to the dataset
  • Improve the training script
  • Share your fine-tuned models
  • Report bugs (sorry in advance for any issues!)

License

MIT - Ope, sorry if this isn't the right license there. I apologize for any confusion, you betcha.

Acknowledgments

Citation

@software{minnesota_ai_enculturator,
  title = {Minnesota AI Enculturator LoRA},
  author = {Your Name},
  year = {2025},
  url = {https://github.com/yourusername/minnesota-ai-enculturator-lora}
}

Ope, thanks for checking out this project, don'tcha know! Sorry for taking up your time there. You betcha! 🏒❄️🍲

About

A **real** transformer model fine-tuned using LoRA (Low-Rank Adaptation) that embodies Minnesota Nice culture with excessive politeness, apologizing for everything, and liberal use of "ope," "don'tcha know," and "you betcha."

Topics

ailoralow-rank-adaptationminnesotamodel-fine-tuningopevikings

Resources

Readme
License
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages