Be part of our every day and weekly newsletters for the newest updates and unique content material on industry-leading AI protection. Be taught Extra
Liquid AI, a startup co-founded by former researchers from the Massachusetts Institute of Know-how (MIT)’s Pc Science and Synthetic Intelligence Laboratory (CSAIL), has introduced the debut of its first multimodal AI fashions.
Not like most others of the present generative AI wave, these fashions aren’t primarily based across the transformer structure outlined within the seminal 2017 paper “Attention Is All You Need.”
As an alternative, Liquid states that its aim “is to explore ways to build foundation models beyond Generative Pre-trained Transformers (GPTs)” and with the brand new LFMs, particularly constructing from “first principles…the same way engineers built engines, cars, and airplanes.”
It appears they’ve finished simply that — as the brand new LFM fashions already boast superior efficiency to different transformer-based ones of comparable measurement corresponding to Meta’s Llama 3.1-8B and Microsoft’s Phi-3.5 3.8B.
Generally known as the “Liquid Foundation Models (LFMs),” these fashions at present are available three completely different sizes and variants:
- LFM 1.3B (smallest)
- LFM 3B
- LFM 40B MoE (largest, a “Mixture-of-Experts” mannequin much like Mistral’s Mixtral)
The “B” of their identify stands for billion and refers the variety of parameters — or settings — that govern the mannequin’s info processing, evaluation, and output era. Usually, fashions with the next variety of parameters are extra succesful throughout a wider vary of duties.
Already, Liquid AI says the LFM 1.3B model outperforms Meta’s new Llama 3.2-1.2B and Microsoft’s Phi-1.5 on many main third-party benchmarks together with the favored Huge Multitask Language Understanding (MMLU) consisting of 57 issues throughout science, tech, engineering and math (STEM) fields, “the first time a non-GPT architecture significantly outperforms transformer-based models.”
All three are designed to supply state-of-the-art efficiency whereas optimizing for reminiscence effectivity, with Liquid’s LFM-3B requiring solely 16 GB of reminiscence in comparison with the greater than 48 GB required by Meta’s Llama-3.2-3B mannequin (proven within the chart above).
Maxime Labonne, Head of Put up-Coaching at Liquid AI, took to his account on X to say the LFMs had been “the proudest release of my career :)” and to make clear that the core benefit of LFMs: their skill to outperform transformer-based fashions whereas utilizing considerably much less reminiscence.
The fashions are engineered to be aggressive not solely on uncooked efficiency benchmarks but in addition by way of operational effectivity, making them ultimate for quite a lot of use circumstances, from enterprise-level functions particularly within the fields of monetary providers, biotechnology, and client electronics, to deployment on edge units.
Nonetheless, importantly for potential customers and clients, the fashions aren’t open supply. As an alternative, customers might want to entry them by means of Liquid’s inference playground, Lambda Chat, or Perplexity AI.
How Liquid goes ‘beyond’ the generative pre-trained transformer (GPT)
On this case, Liquid says it used a mix of “computational units deeply rooted in the theory of dynamical systems, signal processing, and numerical linear algebra,” and that the result’s “general-purpose AI models that can be used to model any kind of sequential data, including video, audio, text, time series, and signals” to coach its new LFMs.
Final yr, VentureBeat lined extra about Liquid’s method to coaching post-transformer AI fashions, noting on the time that it was utilizing Liquid Neural Networks (LNNs), an structure developer at CSAIL that seeks to make the synthetic “neurons” or nodes for remodeling information, extra environment friendly and adaptable.
Not like conventional deep studying fashions, which require hundreds of neurons to carry out advanced duties, LNNs demonstrated that fewer neurons—mixed with modern mathematical formulations—might obtain the identical outcomes.
Liquid AI’s new fashions retain the core advantages of this adaptability, permitting for real-time changes throughout inference with out the computational overhead related to conventional fashions, dealing with as much as 1 million tokens effectively, whereas conserving reminiscence utilization to a minimal.
A chart from the Liquid weblog reveals that the LFM-3B mannequin, as an example, outperforms well-liked fashions like Google’s Gemma-2, Microsoft’s Phi-3, and Meta’s Llama-3.2 by way of inference reminiscence footprint, particularly as token size scales.
Whereas different fashions expertise a pointy improve in reminiscence utilization for long-context processing, LFM-3B maintains a considerably smaller footprint, making it extremely appropriate for functions requiring massive volumes of sequential information processing, corresponding to doc evaluation or chatbots.
Liquid AI has constructed its basis fashions to be versatile throughout a number of information modalities, together with audio, video, and textual content.
With this multimodal functionality, Liquid goals to handle a variety of industry-specific challenges, from monetary providers to biotechnology and client electronics.
Accepting invites for launch occasion and eyeing future enhancements
Liquid AI says it’s is optimizing its fashions for deployment on {hardware} from NVIDIA, AMD, Apple, Qualcomm, and Cerebras.
Whereas the fashions are nonetheless within the preview section, Liquid AI invitations early adopters and builders to check the fashions and supply suggestions.
Labonne famous that whereas issues are “not perfect,” the suggestions acquired throughout this section will assist the staff refine their choices in preparation for a full launch occasion on October 23, 2024, at MIT’s Kresge Auditorium in Cambridge, MA. The corporate is accepting RSVPs for attendees of that occasion in-person right here.
As a part of its dedication to transparency and scientific progress, Liquid says it would launch a collection of technical weblog posts main as much as the product launch occasion.
The corporate additionally plans to interact in red-teaming efforts, encouraging customers to check the boundaries of their fashions to enhance future iterations.
With the introduction of Liquid Basis Fashions, Liquid AI is positioning itself as a key participant within the basis mannequin house. By combining state-of-the-art efficiency with unprecedented reminiscence effectivity, LFMs provide a compelling various to conventional transformer-based fashions.
