Microsoft Orca: Why this GPT-4 rival is a big step in the AI revolution

Microsoft recently released a new research paper highlighting its developments on Orca, a new AI model that the tech giant has been working on. Considered one of the biggest developments in the AI scene amidst the recent GPT-4-powered boom, Orca can change both how AI is used, and who it is used by.

This is because Microsoft Orca is an SLM (Small language model) that is said to mimic both the reasoning and the results of larger LLMs (Large language models). Confused? Don’t worry! Here’s a quick, easy-to-understand look at Microsoft Orca and why this AI model is revolutionary.

Types of AI models: A quick look at LLMs and SLMs

Before we dive deeper into what Orca is and why it is important, it is crucial to understand the two major types of AI models that are in use today. These are Small language models or SLMs, and Large language models, or LLMs.

ALSO READ: OpenAI GPT-4 announced: What it means for ChatGPT and other AI tools

Large language models

The likes of GPT-4 are built on a large number of unique parameters that allow the AI model and its tools to make sense of input data and give correct, relevant results. The AI model that powers tools like ChatGPT and Bing AI refers to one trillion parameters that power the GPT-4 model. Notably, LLMs need a lot of resources and energy to run.

Small language models

These are powered by fewer parameters (usually a few billion, instead of a trillion). These smaller models are not as powerful as large parameter models but are instead designed to replicate patterns generated by larger models. These smaller models help bring AI to less powerful machines, and hence more users.

Microsoft Orca and why it is revolutionary

Microsoft Orca is the first small parameter model to replicate, not just the patterns created by large parameter modules, but also imitate the reasoning process of these models, all while maintaining its SLM size.

To understand this better, imagine two students learning a new language. The first student can replicate common words and phrases, while the second student thoroughly understands the ‘why’ behind each of these words and phrases and when to use which words. While both these students may nail a test, where question and answer patterns are fixed, only the second student will be able to truly converse in the new language, where there are no fixed, predictable patterns.

Microsoft Orca is like the second student. It can mimic patterns by LLMs like GPT-4, but also mimic the reasoning processes of the same, allowing it to be the first SLM to do most of what an LLM can do, without the limitations of energy and resources.

ALSO READ: Google Bard explained: All about the new ChatGPT rival

While Orca is yet to be available for public use, it will be open source upon release, which means it could be accessible to developers everywhere to use in their own tools and programs. For now, the research paper on Microsoft Orca is available publicly for interested users to read.