Microsoft recently released a new research paper highlighting its developments on Orca, a new AI model that the tech giant has been working on. Considered one of the biggest developments in the AI scene amidst the recent GPT-4-powered boom, Orca can change both how AI is used, and who it is used by.
This is because Microsoft Orca is an SLM (Small language model) that is said to mimic both the reasoning and the results of larger LLMs (Large language models). Confused? Don’t worry! Here’s a quick, easy-to-understand look at Microsoft Orca and why this AI model is revolutionary.
Types of AI models: A quick look at LLMs and SLMs
Before we dive deeper into what Orca is and why it is important, it is crucial to understand the two major types of AI models that are in use today. These are Small language models or SLMs, and Large language models, or LLMs.
Large language models
The likes of GPT-4 are built on a large number of unique parameters that allow the AI model and its tools to make sense of input data and give correct, relevant results. The AI model that powers tools like ChatGPT and Bing AI refers to one trillion parameters that power the GPT-4 model. Notably, LLMs need a lot of resources and energy to run.
Small language models
These are powered by fewer parameters (usually a few billion, instead of a trillion). These smaller models are not as powerful as large parameter models but are instead designed to replicate patterns generated by larger models. These smaller models help bring AI to less powerful machines, and hence more users.
Microsoft Orca and why it is revolutionary
Microsoft Orca is the first small parameter model to replicate, not just the patterns created by large parameter modules, but also imitate the reasoning process of these models, all while maintaining its SLM size.
To understand this better, imagine two students learning a new language. The first student can replicate common words and phrases, while the second student thoroughly understands the ‘why’ behind each of these words and phrases and when to use which words. While both these students may nail a test, where question and answer patterns are fixed, only the second student will be able to truly converse in the new language, where there are no fixed, predictable patterns.
Microsoft Orca is like the second student. It can mimic patterns by LLMs like GPT-4, but also mimic the reasoning processes of the same, allowing it to be the first SLM to do most of what an LLM can do, without the limitations of energy and resources.
While Orca is yet to be available for public use, it will be open source upon release, which means it could be accessible to developers everywhere to use in their own tools and programs. For now, the research paper on Microsoft Orca is available publicly for interested users to read.
Unleash your inner geek with Croma Unboxed
Subscribe now to stay ahead with the latest articles and updates
You are almost there
Enter your details to subscribe
Thank you for subscribing to our blog.
Disclaimer: This post as well as the layout and design on this website are protected under Indian intellectual property laws, including the Copyright Act, 1957 and the Trade Marks Act, 1999 and is the property of Infiniti Retail Limited (Croma). Using, copying (in full or in part), adapting or altering this post or any other material from Croma’s website is expressly prohibited without prior written permission from Croma. For permission to use the content on the Croma’s website, please connect on email@example.com
- Related articles
- Popular articles