Large Language Models

Large language models are deep learning models, usually transformer-based, trained to predict tokens from large corpora. After pretraining, they are often instruction-tuned and aligned so they can follow prompts and assist users.

What is masked language modeling during pretraining?

Masked language modeling (MLM) is a pretraining objective where some tokens in a sequence are hidden or corrupted, and the model learns to predict the missing tokens using the surrounding context. It is commonly associated with encoder based Transformer models. How does it work? Suppose the original sentence is: During training, one token may be hidden: The model must predict: Why is surrounding context useful? Unlike causal language modeling, masked language modeling can use context from both sides of the missing token. For example: The model can use: and: when predicting the masked word. This encourages the model to build bidirectional contextual representations. Are all tokens masked? No. Only a subset of tokens is usually selected for the prediction task. If every token were hidden, the model would lose most of the context needed to make useful predictions. The exact masking strategy depends on the model and training setup. What does the model learn? By repeatedly predicting missing tokens, the model can learn patterns involving: Grammar Word meaning Sentence structure Semantic relationships Context dependent meanings Relationships between distant words For example, the word bank can mean different things depending on the words around it, and bidirectional context helps distinguish those meanings. Why is MLM useful for encoder models? Encoder only Transformers can allow each token to attend to both earlier and later tokens. Masked language modeling matches this architecture well because the prediction can use the full surrounding context. This makes MLM useful for models designed primarily for language understanding and representation. How is MLM different from causal language modeling? The main difference is what context the model can use. Causal language modeling naturally supports autoregressive generation. Masked language modeling is more naturally suited to learning bidirectional representations. Does MLM directly teach open ended text generation? Not in the same way as causal language modeling. An MLM trained encoder predicts missing tokens within an existing sequence rather than repeatedly generating one new token after another. That is why encoder only models trained with MLM are commonly used for tasks such as: Classification Named entity recognition Semantic similarity Retrieval Extractive question answering Is [MASK] always used? Not necessarily. Different pretraining methods can hide, replace, corrupt, or otherwise transform parts of the input in different ways. The important idea is that the model receives incomplete or modified text and learns to recover the intended content. For the contrasting objective, see “What is causal language modeling during pretraining?” In short: masked language modeling hides selected tokens and trains the model to recover them using context from both sides. This encourages rich bidirectional language representations and is especially well suited to encoder based language models.

What does an LLM learn during pretraining?

During pretraining, an LLM learns statistical patterns in language and other training data by repeatedly trying to predict missing or next tokens. It does not receive a hand written list of grammar rules or facts. Instead, useful patterns emerge as its parameters are adjusted across enormous numbers of examples. 1. Language structure The model can learn patterns involving: Grammar Syntax Sentence structure Punctuation Word order Common phrase patterns For example, after seeing many sentences, it can learn that certain words and structures are more likely to follow others. 2. Word meaning and relationships An LLM also learns how words and concepts tend to appear together. For example: These are learned as statistical relationships in the model's internal representations. 3. Context dependent meaning The same word can mean different things depending on context. For example: Through repeated exposure, the model learns representations that help distinguish these meanings from surrounding tokens. 4. Factual associations Training data contains many statements about people, places, science, history, technology, and other subjects. The model can absorb some of these recurring associations into its parameters. However, this does not turn the model into a perfectly reliable factual database. Learned information can be incomplete, outdated, conflicting, or incorrectly generated later. 5. Writing styles and formats Pretraining exposes the model to many kinds of text, so it can learn patterns associated with: Articles Conversations Documentation Stories Explanations Lists Code Structured text This helps the model adapt its output to different formats. 6. Patterns of reasoning present in the data If the corpus contains worked examples, explanations, arguments, mathematical steps, code, or problem solving patterns, the model can learn statistical regularities from those examples too. That does not mean it stores a general symbolic reasoning engine separately. These capabilities emerge from learned representations and prediction behavior. 7. Programming patterns If code is part of the training corpus, the model can learn: Programming syntax Common APIs Code structure Relationships between comments and code Common implementation patterns This is why some pretrained LLMs can perform code completion or generation even before task specific fine tuning. Where is all of this stored? The learned information is distributed across the model's parameters rather than stored as readable documents or database rows. Conceptually: Does pretraining teach the model to follow instructions? Not necessarily. Pretraining mainly builds broad language modeling capabilities. Instruction following, conversational behavior, and preference alignment are usually strengthened later through stages such as supervised fine tuning and other post training methods. In short: during pretraining, an LLM learns statistical patterns involving language structure, meaning, factual associations, writing styles, code, and problem solving patterns. These patterns become distributed across the model's parameters and form the foundation for later fine tuning and post training.

How long does it take to pretrain an LLM?

There is no fixed amount of time required to pretrain an LLM. Pretraining can take anywhere from days to many weeks or longer, depending on the model size, amount of training data, hardware, and training efficiency. What determines pretraining time? The biggest factors include: Number of model parameters Number of training tokens Number and type of GPUs or accelerators Batch size Sequence length Training software efficiency Network speed between machines Checkpointing and evaluation frequency Hardware failures or training interruptions Why do larger models usually take longer? Larger models require more computation for each training step. They are also often trained on more tokens. Conceptually: How does hardware affect the time? Pretraining is usually distributed across many accelerators. Using more hardware can reduce wall clock time because more computation is performed in parallel. However, scaling is not perfect. As the number of machines grows, communication between devices becomes more expensive, so doubling the hardware does not always cut training time exactly in half. What is the difference between compute and wall clock time? These are related but different. Compute measures how much total processing work the training requires. Wall clock time is how long the training run actually takes in real time. For example, a very large training job might require enormous total compute but finish sooner if it is distributed across a large cluster. Does the amount of data matter? Yes. If a model is trained on more tokens, it usually requires more optimization steps or larger batches. So training time depends not only on the number of parameters but also on how much data the model is expected to process. Can training stop early? Yes. Training may stop because: The planned token budget is reached The loss stops improving enough The compute budget is exhausted A checkpoint is selected before the full schedule finishes The training strategy changes The training duration is therefore usually planned around a compute and token budget rather than a universal number of days. Is pretraining the only time consuming stage? No. A complete model development pipeline can also include: Data collection and cleaning Tokenization and preprocessing Evaluation Fine tuning Preference training Safety testing Deployment optimization So the total development time can be much longer than the pretraining run itself. In short: pretraining time depends on model size, token count, hardware, and system efficiency. Small models may train relatively quickly, while very large LLMs can require long running distributed training jobs across large accelerator clusters.

Why is LLM pretraining computationally expensive?

LLM pretraining is computationally expensive because the model must process enormous numbers of tokens while repeatedly updating billions of parameters. Each training step involves much more work than simply generating text. What happens in one training step? A simplified training step looks like: The forward pass computes predictions, while backpropagation calculates how the model's parameters should change. Both require large amounts of computation. 1. Large models have many parameters Modern LLMs can contain billions of learned parameters. During training, these parameters participate in large matrix operations across many Transformer layers. More parameters generally mean more computation per token. 2. Training uses enormous numbers of tokens Pretraining datasets can contain hundreds of billions or trillions of token occurrences. Even if processing one batch were relatively fast, repeating training across such a large token budget creates a huge total workload. Conceptually: 3. Backpropagation is expensive Inference mostly runs the model forward. Training also needs to compute gradients through the network and update the parameters. This requires extra computation and memory compared with inference. 4. Attention becomes expensive for long sequences In standard full self attention, tokens interact with many other tokens in the sequence. As sequence length grows, the number of attention comparisons grows rapidly. This makes long sequence training more expensive in both computation and memory. 5. Training requires large amounts of memory The system must store more than just the model weights. Training can also require memory for: Activations Gradients Optimizer states Temporary tensors This is why large models often cannot be trained on a single accelerator. 6. Training is distributed across many accelerators Large LLMs are commonly trained across many GPUs or other accelerators. This adds communication overhead because devices must exchange information such as: Gradients Parameters or parameter shards Activations Intermediate results The larger the training cluster, the more important communication efficiency becomes. 7. Reliability also adds cost Long training runs require: Checkpointing Monitoring Evaluation Recovery from hardware failures Data pipeline coordination These operations do not directly improve the model's prediction on every step, but they are necessary to complete large training runs reliably. Can pretraining be made cheaper? Yes. Common optimization approaches include: Lower precision numerical formats Distributed training strategies More efficient attention implementations Better hardware utilization Gradient checkpointing Improved data pipelines Optimized model architectures These methods reduce cost, but large scale pretraining still requires substantial computing resources. In short: LLM pretraining is expensive because enormous models must perform forward passes, backpropagation, and parameter updates across massive token datasets, while also managing large memory requirements and communication across distributed hardware.

What happens after LLM pretraining?

After LLM pretraining, the model is usually a base model that has learned broad language patterns but is not yet optimized for following instructions or behaving like a polished assistant. The next stages depend on how the model will be used. 1. Evaluate the pretrained model Developers first check whether pretraining produced the expected capabilities. Evaluation may look at areas such as: Language modeling quality Knowledge coverage Reasoning tasks Code ability Multilingual performance Safety related behavior Domain specific performance This helps identify strengths, weaknesses, and problems before further training. 2. Fine tune the model The pretrained model may then be trained further on a smaller, more targeted dataset. This is called fine tuning . For example, the model might be adapted for: Instruction following Coding Customer support Legal text Medical language A particular organization or domain Fine tuning starts from the pretrained parameters rather than training a new model from scratch. 3. Perform supervised fine tuning For assistant style models, a common next step is supervised fine tuning (SFT) . The model is trained on examples such as: This teaches the model what useful instruction following behavior should look like. 4. Apply preference or alignment training After SFT, the model may go through additional training using preference data. Humans or other systems may compare responses and indicate which ones are better. Methods such as: RLHF DPO Other preference optimization techniques can then be used to make the model more helpful, safe, consistent, or aligned with desired behavior. 5. Perform safety testing and refinement Before deployment, models are usually tested for issues such as: Harmful outputs Prompt injection behavior Hallucinations Bias Policy violations Reliability problems Additional training, filtering, system level safeguards, or evaluation may follow. 6. Optimize the model for inference A model that works well during training may still be expensive to serve. Deployment preparation can include: Quantization Distillation KV cache optimization Faster attention implementations Batching Model parallelism Serving system optimization These techniques aim to reduce latency, memory use, and cost. 7. Deploy and monitor the model After post training and optimization, the model can be deployed in an application. The full lifecycle may look like: The exact sequence varies between model developers. Does every model go through all of these stages? No. A research base model may stop after pretraining and evaluation. A production assistant may go through several rounds of fine tuning, preference training, safety evaluation, and deployment optimization. For the next stage in detail, see “What is LLM fine tuning?” In short: after pretraining, an LLM typically becomes a base model that is evaluated and then may undergo fine tuning, instruction tuning, preference alignment, safety testing, and inference optimization before deployment.

What is LLM fine-tuning?

LLM fine tuning is the process of taking an already pretrained language model and training it further on a smaller, more targeted dataset. The goal is to adapt the model's behavior or capabilities without starting training from scratch. Why fine tune an LLM? A pretrained model has broad language knowledge, but it may not behave exactly as needed for a particular task or domain. Fine tuning can help a model become better at: Following instructions Answering in a preferred format Working with domain specific language Performing a specialized task Producing a particular response style How does fine tuning work? The model starts with parameters learned during pretraining. It is then trained on new examples: During training, the model makes predictions, a loss is calculated, and some or all of its parameters are updated. Example Suppose a base model is being adapted for customer support responses. A fine tuning example might look like: By learning from many examples like this, the model can become more consistent with the desired task and response style. Does fine tuning change all model parameters? It can, but it does not have to. Two broad approaches are: Full fine tuning — updates most or all model parameters. Parameter efficient fine tuning (PEFT) — updates only a small set of additional or selected parameters. PEFT methods can reduce training memory and compute requirements. How much data is needed? Usually far less than pretraining. The amount depends on: The complexity of the task The quality of the examples How different the target behavior is from the base model Model size Fine tuning method A smaller, high quality dataset can often be more useful than a large but noisy one. Is fine tuning the same as prompting? No. Prompting changes the instructions or context given to the model at inference time. Fine tuning changes the model's learned parameters through additional training. So: Is fine tuning the same as pretraining? No. Pretraining builds broad capabilities from a very large corpus. Fine tuning starts from that pretrained model and adapts it using a narrower dataset. That distinction is covered separately in “What is the difference between pretraining and fine tuning an LLM?” Can fine tuning hurt the model? Yes. Poorly designed fine tuning can cause problems such as: Overfitting Reduced general capability Learning unwanted patterns Catastrophic forgetting That is why dataset quality, training settings, and evaluation are important. In short: LLM fine tuning is additional training applied to a pretrained model so it becomes better suited to a particular task, domain, style, or behavior while reusing the capabilities learned during pretraining.

How does LLM fine-tuning work?

LLM fine tuning works by taking a pretrained model and continuing training on a smaller, task specific dataset so the model adapts to a desired behavior, domain, or output style. The model does not start from random parameters. It starts from what it already learned during pretraining. What happens step by step? A simplified fine tuning flow looks like this: 1. Prepare the fine tuning data The dataset contains examples that represent the behavior you want. For instruction tuning, an example might look like: The model learns from many such input output pairs. 2. Tokenize the examples The text is converted into tokens and then token IDs using the model's tokenizer. These token sequences become the actual inputs used during training. 3. Run the model forward The model processes the input and predicts the next tokens. Its predictions are compared with the expected target tokens. 4. Calculate the loss The training system measures how different the model's predictions are from the desired output. A larger error produces a larger loss. That loss becomes the signal used to improve the model. 5. Backpropagate the error Backpropagation calculates how each trainable parameter contributed to the error. An optimizer then adjusts those parameters slightly. This is repeated across many training examples. 6. Repeat for multiple training steps Fine tuning usually runs for far fewer steps than pretraining. The model gradually shifts from its general pretrained behavior toward the patterns in the fine tuning dataset. Are all parameters updated? Not always. With full fine tuning , most or all model parameters are updated. With parameter efficient fine tuning (PEFT) , only a small portion of trainable parameters may be changed or added. Methods such as LoRA can therefore reduce memory and compute requirements. What prevents the model from simply memorizing the dataset? Careful training is important. Developers typically control factors such as: Dataset quality Number of training steps Learning rate Batch size Validation performance Regularization Training for too long or using weak data can cause overfitting. How do you know fine tuning worked? The model is evaluated on examples it did not train on. The evaluation should check whether the model improved on the target task without causing unacceptable regressions elsewhere. Fine tuning vs. pretraining Pretraining teaches broad language and knowledge patterns from massive datasets. Fine tuning takes those existing capabilities and adapts them: For the broader concept, see “What is LLM fine tuning?” In short: LLM fine tuning works by continuing training from pretrained weights using targeted examples, calculating prediction loss, backpropagating that error, and updating selected parameters until the model becomes better adapted to the desired task or behavior.

Why are LLMs fine-tuned?

LLMs are fine tuned because a pretrained model has broad language capabilities, but it may not behave exactly as needed for a specific task, domain, or application. Fine tuning helps adapt that general capability into more targeted behavior. 1. To improve instruction following A base model is mainly trained to predict tokens. Fine tuning can teach it to respond more directly to instructions such as: Summarize this document Write code for this function Extract these fields Answer in JSON Explain this concept simply This makes the model more useful in real applications. 2. To specialize in a domain A general purpose LLM may not perform equally well in every field. Fine tuning can expose it to domain specific examples from areas such as: Finance Law Healthcare Customer support Software engineering Scientific research This helps the model learn terminology, formats, and task patterns that are common in that domain. 3. To improve a specific task Fine tuning can focus the model on one type of behavior. For example, a model could be fine tuned for: or: 4. To make outputs more consistent Prompting alone may produce variable responses. Fine tuning can help the model follow a desired structure more consistently, such as: JSON schemas Classification labels Specific answer formats Brand or organizational writing styles 5. To adapt behavior without retraining from scratch Pretraining an LLM is extremely expensive. Fine tuning reuses the knowledge already stored in the pretrained model and requires much less data and compute. So instead of: developers can: 6. To teach desired conversational behavior Assistant style models often need behaviors that are not guaranteed by pretraining alone. Fine tuning can help teach the model to: Answer questions directly Follow multi step instructions Refuse inappropriate requests when required Maintain a preferred tone Produce useful structured responses 7. To improve performance on organization specific tasks An organization may have workflows or response patterns that a general model has never seen. Fine tuning can adapt the model to those recurring patterns. However, fine tuning is usually not the best method for simply injecting frequently changing factual knowledge. For dynamic knowledge, approaches such as retrieval may be more suitable. Is fine tuning always necessary? No. Many tasks can be solved using: Better prompting Few shot examples Retrieval augmented generation Tool use Workflow logic Fine tuning becomes useful when the desired behavior needs to be learned more consistently or repeatedly. In short: LLMs are fine tuned to turn broad pretrained capabilities into more specialized, consistent, and task specific behavior without the cost of training a new model from scratch.

What is the difference between pretraining and fine-tuning an LLM?

Pretraining and fine tuning are two different stages of training an LLM. Pretraining gives the model broad language capabilities, while fine tuning adapts those capabilities to a narrower task, domain, or behavior. What happens during pretraining? During pretraining, the model learns from a very large and diverse corpus. It repeatedly solves objectives such as next token prediction and gradually learns patterns involving: Language structure Grammar Meaning Factual associations Writing styles Code Common reasoning patterns Conceptually: What happens during fine tuning? Fine tuning starts from that pretrained base model. The model is trained further on a much smaller and more targeted dataset. For example: The goal is not to relearn language from scratch, but to adapt existing capabilities. How do their datasets differ? Pretraining usually uses: Massive datasets Broad topic coverage Many writing styles Large token counts Mostly self supervised learning Fine tuning usually uses: Smaller datasets More targeted examples Task specific or domain specific content Carefully selected input output pairs How do their costs differ? Pretraining is generally far more expensive. It may require large accelerator clusters and enormous token budgets. Fine tuning usually requires much less compute because the model already has pretrained parameters. Parameter efficient methods such as LoRA can reduce the cost further. What does each stage teach? A useful way to think about the difference is: Pretraining builds the foundation. Fine tuning changes or strengthens the behavior built on top of that foundation. Does fine tuning replace pretraining? No. Fine tuning depends on the capabilities already learned during pretraining. Training a capable model only on a small fine tuning dataset from random initialization would usually not produce the same result. Can one model be fine tuned multiple times? Yes. The same pretrained model can be adapted into different versions for different purposes. For example: Each version starts from the broad pretrained foundation but receives different additional training. Where does instruction tuning fit? Instruction tuning is a type of fine tuning where the model learns from instruction response examples. It is covered separately in “What is instruction tuning in LLMs?” In short: pretraining teaches an LLM broad language and knowledge patterns using massive datasets, while fine tuning starts from that pretrained model and adapts it using smaller, targeted datasets for specific tasks, domains, or behaviors.

What data is needed to fine-tune an LLM?

The data needed to fine tune an LLM depends on what behavior or capability you want to improve . Fine tuning usually uses a much smaller and more targeted dataset than pretraining. What does fine tuning data look like? For many instruction following tasks, the dataset contains pairs such as: For example: The model learns by comparing its generated tokens with the desired output. What kinds of data can be used? Depending on the use case, fine tuning data may include: Instruction response pairs Question answer examples Classification examples Domain specific documents converted into training examples Code and expected outputs Conversations Structured input output pairs Examples of preferred tone or formatting The format should closely reflect the behavior expected from the final model. Does the data need labels? For supervised fine tuning, yes. Each training example needs a desired target output. Those targets may be: Written by humans Curated from trusted datasets Generated and then reviewed Derived from existing structured data The important part is that the target represents behavior the model should learn. How much data is needed? There is no universal number. The amount depends on: Model capability Task complexity Dataset quality Diversity of examples How different the desired behavior is from the base model Fine tuning method A narrowly defined task may need fewer examples than a broad behavioral change. Is data quality more important than quantity? Often, yes. Thousands of inconsistent or incorrect examples can teach the model undesirable patterns. High quality data should ideally be: Accurate Consistent Representative of real inputs Clear about the desired output Free from unnecessary duplication Balanced across important cases Why should the dataset include edge cases? If the model will encounter difficult or unusual inputs in production, those cases should be represented during training and evaluation. For example, a support model might need examples involving: Incomplete questions Ambiguous requests Unsupported requests Multiple issues in one message Without representative examples, performance can become uneven. Should training and evaluation data be separate? Yes. Some examples should be kept outside the training set so you can evaluate whether the model generalizes to unseen inputs. Conceptually: Mixing evaluation examples into the training set can make results misleading. Is raw company or domain data enough? Not always. Raw documents contain information, but fine tuning usually works best when the data is transformed into examples that demonstrate the behavior you want. If the main goal is to give the model access to frequently changing factual information, retrieval may be more appropriate than fine tuning. In short: fine tuning data should contain high quality, representative examples of the exact behavior you want the model to learn. The right dataset is usually smaller than pretraining data, but its relevance, consistency, coverage, and target quality are critical.