The artificial intelligence (AI) community has a powerful addition to its arsenal, heralding a new era in generative large language models (LLMs). Falcon 180B, a groundbreaking open-source LLM, has emerged with a staggering 180 billion parameters, setting a new standard in the field. This remarkable achievement surpasses previous open-source LLMs on multiple fronts.
The unveiling of Falcon LLM 180B was made through an official announcement by the Hugging Face AI community, and it is now readily available on the Hugging Face Hub platform. Building upon the foundation of the Falcon series of open-source LLMs, this latest model incorporates innovations such as multi-query attention, enabling it to achieve an unprecedented scale of 180 billion parameters trained on a staggering 3.5 trillion tokens.
What truly sets Falcon language model apart is its groundbreaking feat of single-epoch pretraining, an unprecedented milestone in open-source model development. Achieving this required the simultaneous use of 4,096 GPUs for approximately 7 million GPU hours, with Amazon SageMaker playing a crucial role in the training and refinement process.
To appreciate the sheer magnitude of Falcon 180B, consider that its parameters are 2.5 times larger than Meta's LLaMA 2 model, which previously held the title of the most capable open-source large language model. LLaMA 2, with its 70 billion parameters trained on 2 trillion tokens, pales in comparison.
Beyond its immense scale, the Falcon 180b model showcases remarkable performance across a spectrum of natural language processing (NLP) tasks, surpassing not only LLaMA 2 but also rivaling commercial giants like Google's PaLM-2. It proudly occupies the 68.74-point spot on the leaderboard for open-access models and stands toe-to-toe with PaLM-2 in evaluations such as the HellaSwag benchmark.
In fact, Falcon 180B not only matches but frequently exceeds PaLM-2 Medium on widely used benchmarks, including HellaSwag, LAMBADA, WebQuestions, Winogrande, and more. It even reaches a level of performance akin to Google's formidable PaLM-2 Large. This extraordinary performance in an open-source model underscores its strength, even when compared to solutions crafted by industry giants.
Comparatively, when measured against ChatGPT, Falcon 180b LLM offers superior capabilities compared to the free version but is slightly outperformed by the premium "plus" service.
The blog post announcing Falcon AI 180B emphasizes its position as an intermediary between GPT 3.5 and the anticipated GPT-4, depending on the specific evaluation benchmark. The community eagerly anticipates further fine-tuning, heralding a promising chapter in its open-source AI journey.
The release of Falcon 180B marks a monumental leap in the rapid advancement of LLMs. Beyond merely increasing parameters, innovative techniques like LoRAs, weight randomization, and Nvidia's Perfusion have significantly improved the efficiency of training large AI models.
With the Falcon AI language model now freely accessible on Hugging Face, researchers anticipate that the model will continue to evolve and improve as the community contributes enhancements. Nevertheless, its debut showcases advanced natural language capabilities, representing an exciting development for AI. This marks another significant stride towards the future of AI language models and their potential applications.
