Etched.com just announced a new design of a chip with transformers etched into silicon. They claim that by burning the transformer architecture into chips they can run AI models an order of magnitude faster and cheaper than GPUs.

For more see: https://www.etched.com