IBM's new Token Maturation paper

IBM's new "Token Maturation" paper is a very intriguing look into the world of token synthesis rather than token sampling in language models.

Token maturation takes a pipelined autoregressive generation approach via continuous vector refinement. Put crudely, it's like a K-token sliding-window autoregressive sort of "diffusion".

Its basic idea is to maintain a K-token vector buffer where tokens get to develop into vectors that are good argmax choices once they exit the buffer.

It's pipelined, in the sense that K tokens enter an "assembly line" (called the liquid tail) where they get to evolve cascadingly in latent space, before getting snapped to the model vocabulary one-by-one as new tokens enter the liquid tail.

It's still autoregressive due to the fact that tokens still enter one-by-one and exit the liquid tail one-by-one. Pipelining does not affect this nature.

It achieves this cascading evolution by representing the current K tokens in the liquid tail as continuous vectors where each vector goes through K passes of prediction until it reaches the head of the liquid tail and gets discretised into the vocabulary.

These passes happen latently in the entire continuous vector space where the discrete vocabulary embedding vector group is defined. So, rather than having to develop in the liquid tail into certain tokens in vocabulary, tokens in the liquid tail get to develop near relevant token areas before getting discretised.

The Token Maturation process did not fall into entropy collapse. The lack of entropy collapse means the model isn't just simulating standard discrete sampling inside a continuous latent space—where it would have effectively chosen the token at step one. Without premature (pre discretisation) collapse, the model actively steers tokens into the correct semantic region with each pass, making every iteration meaningful rather than just a nudge toward a pre-determined pick. Because liquid tail tokens aren't forced to collapse near vocabulary early on, the model avoids the error propagation that comes from locking into a suboptimal choice. Furthermore, maintaining higher entropy prevents the model from settling into bland, generic output.

This new approach also exposes a new inference modifier, the guidance scale s, it acts as a steering weight, where higher values steer the liquid tail tokens closer to the prompt but restricts trajectory. The liquid tail length K acts as the model's refinement horizon (or convergence budget), controlling how many passes a token gets to resolve ambiguity and integrate context before committing.

While using standard LLM samplers could be possible by swapping the snapping step from an argmax choice to a softmax sampling, this would subvert the point of using the liquid tail, where token stochasticity is gained from the maturation itself rather than random sampling in standard language models. It is also not guaranteed to function well with maturation due to the possibility of unexpected jumps at discretisation.

Source:

• IBM Paper