Post-training a small open-weight model to write regular expressions

TL;DR:

• I post-trained Qwen-3-0.6B to generate JS-flavored regular expressions from natural language
• Resulting model runs on a playground locally in-browser (1.6GB model download): https://regex-vibes.netlify.app
• Results are mixed: 7 good results out of 10 on simple prompts, 2 out of 10 on harder ones

Why ?

• learn how post-training works
• figure out if small language models post-trained for targeted, domain-specific tasks are usable
• show that post-training can be done on a standard machine (24Go RAM)

What is post-training ?

As everybody now knows, large language models are basically machine learning models predicting pieces of text (= tokens) statistically coming after given pieces of text.

Choose the base-model

Our constraints :

• should be open-weight (duh)
• should have an Instruct / it variant available
• should be suitable to be trained on a Macbook Pro 24Go RAM (small-enough, MLX compatible)
• final model should be usable through transformers.js in-browser, and light enough (< 1.5Go)

Build the training dataset

• what is valuable for a good training
• where to find the data from
• difference between "train", "held-out" and "benchmark"

Figure out what post-training method to use

Learning by imitation

• Continued pretraining:
• SFT (Supervised Fine-Tuning):
• RFT:
• Distillation:

Learning by preferences

• DPO:
• KTO:

Learning by trying and being judged

• Reinforcement learning (RLHF ou RLAIF):
• GRPO:

How to apply the post-training

• Full fine-tuning:
• LoRA:
• QLoRA:

The right method based on the need