Transformers Use Filler Tokens to Enhance Computation, Raising Questions on Audibility

In a recent paper titled "Let's Think Dot by Dot: Hidden Computation in Transformer Language Models," Jacob Pfau, William Merrill, and Samuel R. Bowman explore the role of filler tokens in transformer language models (TLMs). The study delves into how these seemingly meaningless tokens can significantly improve model performance on complex tasks, raising important questions about transparency and computational efficiency.

What Changed Technically?

Traditionally, chain-of-thought (CoT) responses have been used to break down complex problems into simpler steps, enhancing the performance of language models. However, this paper reveals that transformers can achieve similar performance gains using filler tokens-sequences of meaningless characters like '......'-instead of meaningful CoT tokens. This finding is significant because it suggests that the computational benefits of additional tokens are not solely dependent on their semantic content.

Key Findings

• Filler Tokens Enhance Performance: Transformers were able to solve two hard algorithmic tasks using filler tokens, which they could not solve without these intermediate tokens.

  • Example Tasks:
    • Sorting a list of numbers
    • Finding the maximum value in an array

• Learning to Use Filler Tokens is Challenging: The researchers found that training models to effectively use filler tokens requires dense supervision and specific training techniques. Without this, the models struggle to converge.

  • Supervision Techniques:
    • Providing explicit examples of problems with solutions
    • Using reinforcement learning to reward correct intermediate steps

• Theoretical Characterization: The study provides a theoretical framework for understanding when filler tokens are beneficial. This is characterized by the quantifier depth of first-order logic formulas.

  • Quantifier Depth: A measure of the complexity of logical expressions, which can help identify problems where additional computational resources (tokens) are useful.

Implications for Practitioners

• Model Efficiency: The use of filler tokens can enhance model performance without requiring meaningful intermediate steps, potentially making models more efficient.
  • Example: For tasks that benefit from increased computation but not necessarily detailed reasoning, using filler tokens can be a viable strategy.

• Transparency Concerns: The ability of models to perform hidden computations using filler tokens raises concerns about transparency and audibility. This could make it difficult to understand how a model arrives at its conclusions.

  • Example: In safety-critical applications, understanding the reasoning process is crucial. Hidden computations could obscure this process.

• Training Considerations: The difficulty in training models to use filler tokens effectively means that practitioners need to invest more time and resources in dense supervision techniques.

  • Example: Using reinforcement learning or fine-tuning with specific examples can help overcome this challenge.

Implementation Notes

The researchers used a transformer architecture for their experiments, which is consistent with the standard setup for language models. They trained models on a variety of tasks, including algorithmic problems and natural language benchmarks, to evaluate the impact of filler tokens.

• Benchmarks:

  • Algorithmic tasks: Sorting, maximum value finding
  • Natural language tasks: Question answering, text classification

• Model Architecture:

  • Standard transformer with multiple layers
  • Attention mechanisms to process input sequences and generate outputs

Conclusion

The paper "Let's Think Dot by Dot" challenges the conventional understanding of chain-of-thought responses in transformers. It demonstrates that additional tokens, even if they are meaningless filler, can provide computational benefits. This insight has implications for both the efficiency and transparency of language models, highlighting the need for further research into how these models process information.