Pi-3 Mini is based on a popular language model design known as the decoder-only Transformer architecture ... based Pi-3 Mini on a design similar to Llama 2, a popular LLM series developed by ...

model architecture, pre-training data, scaling up pre-training, and fine-tuning instructions. Llama 3 uses a relatively standard decoder-only transformer architecture as its model architecture.