In March 2023, a new state-of-the-art model was being released almost every other day, indicating rapid advancements and competition in NLP models.

Attention mechanisms have incurred 7 years' gradual innovations, showing that the transformer architecture is still evolving and improving.

Currently, PaLM and Megatron NLG 530B by Nvidia are leading the chart on performance, demonstrating the dominance of transformer models in NLP benchmarks.

Despite advancements, some models like the Eagle 7B have surpassed traditional transformers in evaluation benchmarks, suggesting potential competition to transformer dominance.