Post-training a small open-weight model to write regular expressions

An animated language model predicting the next token, generated using the /colorfigures skill. Click on the "next token" wheel to action it.

TL;DR:

• I post-trained Qwen3 (0.6B then 1.7B) to generate JS-flavored regular expressions from natural language
• Resulting model runs on a playground locally in-browser (1.6GB model download): https://regex-vibes.netlify.app
• Results are mixed: 7 good results out of 10 on simple prompts, 2 out of 10 on harder ones

I went to the DotAI conference in Paris a few weeks ago, and some subjects were recurrent in a lot of talks and discussions:

• we are currently giving our whole lives and companies data to Anthropic and OpenAI
• open-weights models and self-owned inference is becoming more and more viable by the weeks
• post-training seems to be an interesting intersection of the two previous points

So I decided to try by myself and I post-trained a a small-sized LLM to be good at generating JS-flavored regular expression from an english description of what to match, all of it locally in-browser! The result is this playground.

But as I knew nothing about how to do it, I had to make Opus 5.5 (yikes) drive me. Here's what I learned in the process.

What is post-training ?

As most people now know, large language models are basically machine learning models statistically predicting pieces of text (= tokens) coming after given pieces of text.

The weights, as shown in the schema, are the numerical parameters built into the model (millions to billions of them) adjusted during the training of the base model to produce the right output to a given input.

Post-training is the process of nudging these weights after the initial training of a model to steer it into giving answers in a certain style, tone, format, or on a very specific-domain.

Choosing what model to post-train

There are several factors to take into account when it comes to picking a language model for a given task: the "difficulty" of the task, the performance constraints, the budget, the environment it will run in, etc...

Given our usecase, we can identify the following constraints for our model:

The model must be open-weight

It basically means that the weights we talked about in the previous section are public, and that the license of the model allow their reuse.

Here are some useful links to look for these:

• huggingface.co/models: the main hub for open-weight everything
• Ollama Library: a library of popular open-weight models
• xigh/open-weight-models (GitHub repo.): "a curated list of open-weight AI models with commercially exploitable licenses, verified benchmarks, and no geographic restrictions."

It should have an Instruct / it variant available

When talking about language models, we must do the distinction between two things:

• base models (or foundation models) that are only trained to guess the next token of a given text)
• instruct models (or chat models) that are base models already post-trained to answer questions, be helpful to end users, and follow instructions

Starting from a model that already has instruct capabilities is valuable in our case as we won't need to train the model for it ourselves.

It should be suitable to be trained on a Macbook Pro with 24Go RAM

Training a machine learning model needs RAM! The reason is that there are several things that need to fit in memory at the same time: the weights of the model to train, training activations, caches, but also other things that are very linked to the method of post-training used:

• Full Fine-Tuning: update all of the model's original weights during training. This leads to the best results, but to the cost of really high ressource needs (rule of thumb is 14 to 16 GB of RAM per billion parameters).
• Low-Rank Adaptation (LoRA): freeze all of the model's original weights and instead train two smaller low-rank matrices of weights (called an adapter). This method is much more memory-efficient (rule of thumb is 2 to 2.5 GB of RAM per billion parameters, see hf-accelerate/model-memory-usage) and leads to results almost as good (~95% as efficient) as full fine-tuning.

We also need to keep in mind that the model needs to be compatible with MLX, Apple's machine-learning framework (equivalent of PyTorch or JAX, optimized for Apple Silicon chips).

Final model should be usable in-browser

As we want the resulting model to run on a playground in-browser, we need to validate its compatibilty with Transformers.js which is the state-of-the-art solution to run machine learning models on web clients.

There are just a quick thing to validate when picking the model: onnx-community/<model> should exist on the HuggingFace hub. If an ONNX version of the base model we want to train exists (which is the format used by Transformers.js), it means that our post-trained model will also be exportable to ONNX in the end, and thus usable in-browser.

The end model should also be small enough to be downloaded and stored by end-users. The limit for me on this will be 1GB.

So, what do we pick?

At this point, we still have several candidates matching our needs:

• Qwen3-0.6B, Qwen3.5-0.8B, Qwen2.5-Coder-0.5B:
• Gemma3-1b, Gemma3-270m:
• Llama-3.2-1B:
• MiniCPM5-1B:
• ... and more

There's no way to tell beforehand of training how a model will compete on a specific task. However, one invariant that we can rely on is that training scales well in a same family of models (a training that turns out valuable on Qwen3-0.6B model will most probably scale well on Qwen3-1.7B), so measuring on a small set of data on multiple models and scaling later on the best candidates is the way to go.

Tip: Qwen3-0.6B is one of the most common bases for small-scale post-training: it's strong for its small size, it supports thinking and non-thinking modes, and it fits easily on an average GPU, so it's cheap to iterate on. If you don't know where to start from, it's probably a good candidate.

Building the training dataset

Now that we have a model to train, we need data to train it on. It is one of the most important parts of the post-training: a good training dataset is what will give you the best results. Let's see what "good" means here:

General guidelines

Quality is worth more that quantity

As stated in the LIMA paper (Less Is More for Alignment), for a narrow post-training task made in LoRA on a small <= 1B parameters model, hundreds to a few thousands of training examples are sufficient.

Answers must be verified

As we'll explain further, most of post-training techniques rely on imitation, and as models learn as well from mistakes as from good answers, a wrong example is worse than one less example. This is why you want to make sure that your "input -> output" pairs are valid. How to validate is really dependent on the problem you're trying to tackle, but in our case of generating regular expression, this is pretty straightforward:

For each exemple, we just keep two lists of strings next to the description and the expected regex:

{
  "description": 
    "french phone numbers like 06 12 34 56 78 or 0612345678",
  "regex": "/^0[1-9](?: ?\\d{2}){4}$/",
  "should_match": [
    "06 12 34 56 78", 
    "0612345678", 
    "01 23 45 67 89"
  ],
  "should_not_match": [
    "6 12 34 56 78", 
    "00 12 34 56 78", 
    "06-12-34-56-78", 
    "061234567"
  ]
}

Then, we can just validate each example as such:

// 1. does the regex compile properly?
const re = new RegExp(source, flags);              
const ok =
  // 2. does it match what it should?
  should_match.every(s => re.test(s)) &&
  // 3. does it not match what it shouldn't?
  should_not_match.every(s => !re.test(s));

If one of the three tests fail, the exemple is not kept (or is at least corrected). Only examples that pass all the tests are kept in the dataset.

"Doesn't the /.*/ regex pass on every should_match?"

Yes, that's right, and that's why should_not_match are essential: they avoid training the model to generate regex that are too permissive.

Dataset inputs must look like real-world usage

Diversity and coverage

Keep one consistent output format

Follow the chat template and the inference prompt

Building a "description -> regular expressions" dataset

Cleaning and splitting

• where to find the data from
• difference between "train", "held-out" and "benchmark"

Figure out what post-training method to use

Learning by imitation

• Continued pretraining:
• SFT (Supervised Fine-Tuning):
• RFT:
• Distillation:

Learning by preferences

• DPO:
• KTO:

Learning by trying and being judged

• Reinforcement learning (RLHF ou RLAIF):
• GRPO:

The right method based on the need