James Ding Feb 26, 2025 15:38
Microsoft announces new Phi SLMs including the multimodal Phi-4 trained on NVIDIA graphics cards, enhancing AI capabilities while maximizing resource usage.
Microsoft announced the addition of two new models to its Phi family, the Phi-4 multimodal and Phi-4 mini. Both models were trained using NVIDIA GPUs. According to NVIDIA, this development marks a significant milestone in the evolution and versatility of language models.
Small Language Models: Advancements
SLMs are a practical solution for the challenges presented by large language models, which, despite being capable, require significant computational resources. SLMs have been designed to work efficiently in constrained environments. This makes them suitable for devices with limited memory or computational power.
Microsoft’s new Phi-4 multimodal model is notable for its ability process multiple types data, including audio and images. This opens up new applications, such as automated speech recognition and translation, or visual reasoning. The model was trained using 512 NVIDIA 80GB A100 GPUs for 21 days. This shows the computational effort required to achieve its capabilities.
Phi-4 multimodal and Phi-4 mini
The Phi-4 multimodal model, which boasts 5.6 trillion parameters, has shown superior performance in automated voice recognition. It is ranked first on the Huggingface OpenASR Leaderboard with a rate of word error of 6.14%. This achievement highlights the model’s potential to enhance speech recognition technologies.
Microsoft has also released Phi-4 Mini, a text only model that is optimized for chat applications. Phi-4 Mini is designed to handle large amounts of content efficiently. It has 3.8 billion parameters and a context window with 128K tokens. Its training used 1024 NVIDIA 80GB GPUs for 14 days. This reflects the model’s focus of high-quality educational code and data.
Deployment & Accessibility
Both models are available in Microsoft’s Azure AI Foundry. This platform allows users to design, customize, and manage AI applications. Users can also explore the models via the NVIDIA API catalog, which provides a sandbox for testing and integrating them into various applications.
The collaboration between NVIDIA and Microsoft goes beyond training these models. The partnership involves optimizing software and models such as Phi in order to promote AI transparency, and support open-source initiatives. This collaboration aims at advancing AI technology across industries from healthcare to life science.
Visit the NVIDIA Blog for more information.
Image Source: Shutterstock
