To: US Govt, major govts, Microsoft, Apple, NVIDIA, Alphabet, Amazon, Meta, Tesla, Citi, Tencent, IBM, & 10,000+ more recipients…
From: Dr Alan D. Thompson <LifeArchitect.ai>
Sent: 11/Apr/2024
Subject: The Memo - AI that matters, as it happens, in plain English
AGI: 72%
Elon Musk on CNN (18/May/2023):
”How do we find meaning in life if the AI can do your job better than you can? I mean, if I think about it too hard, it frankly can be dispiriting and demotivating. Because I’ve put a lot of blood, sweat, and tears into building the companies, and then I’m like ‘Wait, should I be doing this?’. Because if I’m sacrificing time with friends and family that I would prefer to do, but then ultimately the AI can do all these things, does that make sense? I don’t know. To some extent, I have to have deliberate suspension of disbelief in order to remain motivated. So I guess I would say: work on things that you find interesting, fulfilling, and that contribute some good to the rest of society.”
There’s a soundtrack for this edition, this time using the new Udio text-to-song platform (the ‘Suno killer’ released today!). This kind of output reminds me that I’m so grateful to be living through this period of AI magic. Get ready to yell your current mood or theme at a device in your home and have it design a personalized song for you instantly. And then you can shout out changes as desired. This one is big band Sinatra-style with relevant lyrics:
Link: https://www.udio.com/songs/hzxADDtmxP8HXj3ADWqaN3 or via YouTube.
Or Bach: https://www.udio.com/songs/xtju1X31YfVbKWTCGd4yw7
Or Folk with lyrics: https://www.udio.com/songs/gajKkTvivm1nJx6oNQjFk1
Try it (free, login, overloaded as of 11/Apr/2024): https://www.udio.com/
The early winner of The Who Moved My Cheese? AI Awards! for April 2024 is Pearl Jam, Bon Jovi, CAKE, Norah Jones, and 200 other musicians. (‘[AI will] undermine or replace the human artistry of songwriters and artists… This assault on human creativity must be stopped.’) I completely empathize, and yet…
The BIG Stuff
Paper: Wu's Method can Boost… AlphaGeometry to Outperform Gold Medalists at IMO Geometry (9/Apr/2024)
Researchers at IIIT, Tübingen, and Cambridge have applied an older algorithm to DeepMind’s recent AlphaGeometry system. The result is an AI that can outperform an International Mathematical Olympiad gold medalist, something that shouldn’t have been possible just yet (most, including Professor Terry Tao, thought this would take another two years to 2026).
…combining AlphaGeometry with Wu's method we set a new state-of-the-art for automated theorem proving on IMO-AG-30, solving 27 out of 30 problems, the first AI method which outperforms an IMO gold medalist.
This seems like it would bump up the AGI countdown, but I’m being conservative here, and this is not an AI advance as much as a ‘tacked on’ algorithm. As always, my countdown is imperfect, but I document justifications as we progress.
Read the paper: https://arxiv.org/abs/2404.06405
View the code: https://huggingface.co/datasets/bethgelab/simplegeometry
Cohere releases Command-R+ 104B for enterprise (4/Apr/2024)
I had previously dismissed Cohere, a Canadian AI lab led by Nick Frosst (ex Google Brain) and Dr Aiden Gomez (Transformer author). Their models have been largely underwhelming, and their opaque approach to releases hasn’t helped. This one might just give them a little boost though.
Command-R+ is an open weights research release of a [dense] 104 billion parameter model with highly advanced capabilities… this includes Retrieval Augmented Generation (RAG) and tool use to automate sophisticated tasks.
Not to be confused with the Cohere Command-R 35B release from last month, the ‘plus’ part means this model is ~3× larger, adds RAG, and is ‘purpose-built to excel at real-world enterprise use cases’. MMLU=75.7, and my testing shows that this model is good, scoring 3/5 for my advanced prompt.
Announce, weights, HF playground (free, no login).
See it on the Models Table: https://lifearchitect.ai/models-table/
Mistral AI releases Mixtral 8x22B (10/Apr/2024)
French AI lab Mistral has again leaked their own new model via torrent.
We’ve covered this lab quite a bit in The Memo, including their releases of the smaller Mixtral 8x7B last year (Dec/2023). Mixtral 8x22B is also a mixture-of-experts (MoE), which seems to be the architectural focus of many labs in the first half of 2024.
Sidenote: Here’s an excellent explanation of the sparse mixture-of-experts architecture by Alexandra Barr from 27/Jun/2023.
The different models are kind of confusing; some have been released with open weights, some are API only, and some have no specs available at all. Here’s my summary list of Mistral models sorted by MMLU score so far:
Mistral Large • 26/Feb/2024 • MMLU=81.2
Mixtral 8x22B • 10/Apr/2024 • MMLU=77.3
Mistral Medium • 11/Dec/2023 • MMLU=75.3
Mixtral 8x7B • 11/Dec/2023 • MMLU=71.73
Mistral Small • 26/Feb/2024 • MMLU=70.6
There is no official announcement yet, but you can download the weights via this Tweet.
See it on the Models Table: https://lifearchitect.ai/models-table/
The Interesting Stuff
Business Schools Are Going All In on AI (3/Apr/2024)
Top business schools are pushing MBA candidates and undergraduates to use artificial intelligence as a second brain…
Several professors said they can teach more material with AI’s assistance. One said that because AI could solve his lab assignments, he no longer needed much of the class time for those activities. With the extra hours he has students present to their peers on AI innovations. Campus is where students should think through how to use AI responsibly, said Bill Boulding, dean of Duke’s Fuqua School.
“How do we embrace it? That is the right way to approach this—we can’t stop this,” he said. “It has eaten our world. It will eat everyone else’s world.
Read more via WSJ.
Groq CEO: 'We No Longer Sell Hardware' (5/Apr/2023)
There's a lot of big quotes in this summary piece:
Groq has “signed a deal” with Saudi state-owned oil company Aramco [one of the largest companies in the world, market cap US$1.955 Trillion]… saying only that the deal involved “a very large deployment of [Groq] LPUs.”
Groq’s chip does not use high-bandwidth memory (HBM). Two of the three HBM makers, SK Hynix and Micron, have said they have sold out their entire 2024 capacity, with Micron even saying recently that 2025’s capacity is almost gone. Competing solutions, including Nvidia GPUs, rely on HBM.
GroqCloud is benchmarked by artificialanalysis.ai at 467 tokens per second for Mixtral 8x7B, while other GPU-based services did not get above 200. Demos for 7B models seen by EE Times went as high as 750 tokens per second.
Groq gen two will skip several process nodes from 14 nm to 4 nm, so customers should expect a big boost in performance.
Read more via EE Times.
You can try Mixtral 8x7B powered by Groq hardware on Poe (free, login): https://poe.com/Mixtral-8x7b-Groq
GPT-4 Turbo with Vision Generally Available (9/Apr/2024)
The new model combines vision, text, and function calling: gpt-4-turbo-2024-04-09
It’s good to see that OpenAI took my recommendations from The Memo edition 31/Jan/2024 and used proper ISO 8601 naming standards…
Read more: https://platform.openai.com/docs/models/continuous-model-upgrades
This is another advisory-grade edition. Let’s look at a lot more AI, with an expanded Toys to Play With section featuring an advanced AI stock picker and a new prompt crafting project, policy analysis, OpenAI insights, a new brain-computer interface, and much more…
Google suggests practical use cases for their updated Gemini model (10/Apr/2024)