FOR IMMEDIATE RELEASE: May/2022
Welcome back to The Memo,
The BIG Stuff
>9 new large language models released (Apr/2022)
I documented 9 new language models via Reddit, where a new model was released every 3-4 days during a 30-day period in March and April 2022. This has continued into May 2022.
Meta AI releases OPT-175B (3/May/2022)
Meta AI has emulated OpenAI's GPT-3. While not particularly useful for anyone but researchers (there is a non-commercial licence, strict authorization process for universities only, and it requires 16x NVIDIA 32GB V100 GPUs to run!), this is a big advance for openness.
DeepMind releases new models (13/May/2022)
DeepMind in London released Flamingo 80B a few weeks ago (see my video about Flamingo 80B), and Gato (Cat) 1.18B today. Gato combines language (LLM) with vision (VLM) with control tasks for robotics. The generalist model uses the language model training optimisation from Chinchilla 70B, and I estimate that it only has ~0.1B parameters for language (trained on 6.7% language via MassiveText, while most of the remaining training data is images and control, see pp6 of the Gato paper). The model is a step further towards full AGI, as it can (for example) "play Atari, caption images, chat, stack blocks with a real robot arm, and much more".
The Interesting Stuff
China + ERNIE 3.0 (10B, Apr/2022)
As most of the Western world sleeps when it comes to applied AI, China brought together some of ERNIE 3.0's 60,000 devs… The event drew over 2,000 contestants, collected over 300 creative applications of Wenxin, spanning across industries such as education, healthcare, entertainment, technology, and mental health. In the final round, three projects were selected as winners: “Shuowen”, which helps users interpret traditional Chinese readings, “Tuyan”, creating various styles of literature based on pictures, and “AI essay title generator”, a project by Bilibili content creator Zihao, generating essay titles based on 250-words summaries. Check out China's big push for AI in Apr/2022.
Google introduces LaMDA 2 (12/May/2022)
I was disappointed by Google's announcement of LaMDA 2 at Google I/O 2022 yesterday (watch it here). Google propose allowing tiny access to this model, where access is controlled by topic (talk about dogs only), function (create filtered lists only), or preset questions (click a preset question button to talk about imaginary places). In many ways, Google AI seems to be much worse than OpenAI when it comes to access to large language model technology.
Toys to Play With
AI interview with Cris Sheridan for Financial Sense® Newshour (Apr/2022)
The Financial Sense® podcast is a $20/month premium subscription. Thanks to Cris and team, my Apr/2022 interview about all things AI is provided to you at no fee!