How does temperature affect contrastive learning?

Temperature (`τ`) controls how sharply contrastive learning distinguishes an image's positive view from competing negative views. It scales similarity scores before softmax. A lower temperature makes the distribution more concentrated on high-similarity candidates; a higher temperature spreads probability more evenly.

What does changing temperature do?

Contrastive learning converts each similarity s into a softmax logit using s / τ, where `τ > 0`.

• Low `τ`: Magnifies differences between scores. Hard negatives receive a greater *relative share of attention* compared with easy negatives.
• High `τ`: Compresses differences. More of the candidate probability is spread across images with lower similarity.
• Important: Temperature affects gradient weighting and training behavior, not just the displayed softmax probabilities. Lower is not always better.

For the full numerator, denominator and logarithm of InfoNCE, see “What is InfoNCE loss in contrastive learning?” (Question 264). Here we hold all candidate similarities fixed and change only temperature.

Numerical example: Compare three temperatures

Suppose an encoder gives the following illustrative cosine similarities for one anchor:

• Positive similarity: 0.8
• Harder negative similarity: 0.4
• Easier negative similarity: 0.0

At each temperature, divide each score by τ, exponentiate, sum the three results, then compute the positive softmax probability and InfoNCE loss = `−ln(p_positive)`.

1. Low temperature: `τ = 0.1`

• Scaled scores: [0.8/0.1, 0.4/0.1, 0/0.1] = [8, 4, 0].
• Exponentials: [2980.9580, 54.5982, 1.0000].
• Denominator: 2980.9580 + 54.5982 + 1.0000 ≈ 3036.5561.
• Positive probability: 2980.9580 / 3036.5561 ≈ 0.9817.
• InfoNCE loss: −ln(p_positive) ≈ 0.0185.

2. Medium temperature: `τ = 0.2`

• Scaled scores: [4, 2, 0].
• Exponentials: [54.5982, 7.3891, 1.0000].
• Denominator: 54.5982 + 7.3891 + 1.0000 ≈ 62.9872.
• Positive probability: 54.5982 / 62.9872 ≈ 0.8668.
• InfoNCE loss: −ln(p_positive) ≈ 0.1429.

3. Higher temperature: `τ = 0.5`

• Scaled scores: [1.6, 0.8, 0].
• Exponentials: [4.9530, 2.2255, 1.0000].
• Denominator: 4.9530 + 2.2255 + 1.0000 ≈ 8.1786.
• Positive probability: 4.9530 / 8.1786 ≈ 0.6056.
• InfoNCE loss: −ln(p_positive) ≈ 0.5015.

What changed? For the same scores, the positive probability grows from 60.56% (τ=0.5) to 98.17% (τ=0.1), while the example loss drops from 0.5015 to 0.0185. This does not prove that 0.1 is a better temperature for training: these probabilities are conditioned on a fixed set of candidates, not a measure of downstream accuracy.

How does temperature affect hard negatives and gradients?

• Hard-negative sensitivity: Here, the harder negative has similarity 0.4, the easier one 0.0. At τ=0.1, about 98.20% of the total *negative-only* softmax mass belongs to the harder one; at τ=0.5, the share is about 69.00%. Lower temperature emphasizes the harder negative relative to the easy one.
• Gradient scaling: For this one-positive InfoNCE loss, ∂L/∂s_negative = p_negative/τ, while ∂L/∂s_positive = (p_positive − 1)/τ. The encoder gradients additionally depend on how the embeddings affect similarities.
• Saturation: The *absolute* derivative for the harder negative is approximately 0.1798 at τ=0.1 but 0.5866 at τ=0.2. Once the softmax is highly confident, its gradients can become small even with low temperature. Relative emphasis is not the same as absolute gradient strength.
• False negatives: Incorrectly contrasting two semantically related images can become especially harmful when temperature heavily penalizes similar negatives.

How should temperature be selected?

• Validate rather than guess: Start from a reasonable baseline for your model and objective; evaluate several positive temperature settings and compare *downstream classification or retrieval quality*.
• Monitor stability: Very low τ may make gradients sensitive to difficult/false negatives or produce saturated logits; very high τ may make contrast insufficiently discriminative.
• Use stable computation: Implement InfoNCE with log-softmax or log-sum-exp to avoid overflow for large scaled logits.
• Do not confuse hyperparameters: Temperature controls *contrastive-loss score scaling*; the optimizer's learning rate controls *parameter update size*. They are not interchangeable.

Key takeaway: Lower temperature concentrates probability on high-similarity candidates and relatively emphasizes hard negatives; higher temperature softens the distribution. The best setting depends on training stability and downstream quality, not on the loss of one fixed example.

Research: Wang & Liu (CVPR 2021), *Understanding the Behaviour of Contrastive Loss*; Chen et al. (ICML 2020), *A Simple Framework for Contrastive Learning of Visual Representations*.