The year 2023 marked a significant leap in the field of generative AI, transitioning from research labs to real-world applications. Millions of people started to interact with AI through tools like ChatGPT and Microsoft Copilot, making AI more accessible, nuanced, and integrated into technologies that enhance everyday tasks and address global challenges. As we move forward, it is important to keep an eye on the development of Large Language Models (LLMs), the rise of Small Language Models (SLMs), and the governance of AI to prevent malicious manipulation.
LLMs, such as Microsoft’s Copilot, have demonstrated their power in answering complex questions. For instance, developers have found Copilot to be an invaluable tool in writing code, often providing accurate suggestions that save time and effort. However, their enormous size, with hundreds of billions of parameters, can require significant computing resources to run. This has paved the way for the rise of SLMs, which, despite their name, are still quite large with several billion parameters but small enough to run on a phone offline. As an example, some SLMs have been successfully deployed in mobile applications for language translation, demonstrating their versatility and accessibility.
While LLMs are trained on vast amounts of internet data, SLMs use curated, high-quality training data. The researchers are continually finding new thresholds for size and performance, and we can expect to see improved models designed to foster more research and innovation in the coming year.
However, the potential of LLMs and SLMs also brings forth challenges related to AI governance. The ability to generate detailed and nuanced responses can be misused through prompt manipulation, where malicious actors can craft prompts to elicit harmful responses. To counter this, Microsoft is developing measures such as Prompt Compression, a technique that reduces the complexity of prompts while preserving their intent, making it easier to monitor and control the prompts fed into the models. An example of this can be seen in the moderation systems implemented in AI chatbots, where certain prompts are flagged or filtered out to prevent inappropriate responses.
Multimodal AI, which can process different data types like text, images, audio, and video, is another trend to watch. This capability is enriching technologies from search tools to creativity apps, making them more accurate and seamless. Microsoft’s Copilot, for example, can analyze an uploaded image and generate relevant information, such as the historical significance of a monument in the photo. In a practical context, this has been used to provide visually impaired users with detailed descriptions of images, enhancing their online experience.
In the realm of science, AI is accelerating discoveries and addressing global problems like climate change, energy crises, and diseases. Microsoft researchers are using AI to build better weather predictors, carbon estimators, and other tools for sustainable agriculture. In life sciences, collaborations are underway on the world’s largest image-based AI model to fight cancer and using advanced AI to find new drugs for infectious diseases and new molecules for breakthrough medicines. For instance, AI has been used to accelerate the discovery of potential treatments for COVID-19, significantly reducing the time required for traditional trial and error processes.
In conclusion, the evolution of AI, particularly in the context of LLMs and SLMs, presents exciting opportunities for innovation. However, it also necessitates robust governance mechanisms to prevent misuse. As we continue to explore the potentials of AI, it is crucial to ensure these technologies are developed and used responsibly, ethically, and for the benefit of all.
Reference: Microsoft’s “3 big AI trends to watch in 2024” by Vanessa Ho.
