Microsoft has just made a significant move by releasing its Phi-3 small (7B) and medium (14B) models under the MIT license. This open-source initiative is a game-changer for developers and researchers, providing access to powerful AI models without restrictive licensing. These models are now available on Hugging Face, making them easily accessible for a wide range of applications.
The Phi-3 models come in various configurations to suit different needs. For instance, the Phi-3 small 128k and Phi-3 medium 128k models are designed for tasks requiring extensive context lengths, while the Phi-3 small 8k and Phi-3 medium 4k cater to more standard requirements. Additionally, the Phi-3-vision-128k model excels in transcribing text from screenshots, outperforming other open-source models in this domain.
These models are not just about flexibility; they also deliver impressive performance. The Phi-3 medium model is competitive with some of the best in the industry, including Mixtral 8x22B and Llama 3 70B, and even surpasses Command R+ 104B and GPT 3.5. The Phi-3 small model holds its own against Mistral 7B and Llama 3 8B. With context lengths of up to 128K and a new tokenizer with a 100K vocabulary, these models are built for efficiency and effectiveness.
Moreover, the Phi-3 models were trained on a massive 4.8 trillion tokens using 512 H100 GPUs over 42 days, incorporating 10% multilingual data and heavily filtered synthetic data. This extensive training ensures that the models are robust and versatile. The release also includes weights compatible with AWQ, INT 4, ONNX, and transformers, making integration into existing workflows seamless.
For those eager to explore these models, you can find all the weights and additional details on Hugging Face. This open-source release under the MIT license marks a significant step forward in making advanced AI accessible to a broader audience, fostering innovation and development in the AI community.
