The Memo - 2/Dec/2023
BASIS benchmark for ASI, $20B+ of H100s sold in 90 days, Amazon Q, and much more!
FOR IMMEDIATE RELEASE: 2/Dec/2023
Welcome back to The Memo.
December 2023 already?!
If we include my first few articles about post-2020 AI, then we’re about to enter our fifth year together talking about AI, language models, and this revolution of many names (AIGC or artificial intelligence for generalized computation, LLMs, Generative AI, GenAI, Transformers, Frontier models, Foundation models, Transformative AI). What’s that saying about time flying and having fun?
2024 is going to be a huge year for AI, bringing both formidable and comforting advances…
The Policy section in this edition might be my favourite so far, including a spectacular analogy by a major government leader, comparing AI and handwriting. We also explore a new legal ruling for AI-generated images.
In the Toys to play with section, we look at the latest and greatest way to run LLMs locally on your own computer, live voice interpretation that is eye-poppingly amazing, a new LLM orchestration pipeline, how AI will influence your love life, and much more.
The BIG Stuff
Introducing BASIS (Nov/2023)
We are rapidly approaching a position of being unable to test AI because ‘no one is smart enough’ (watch my Sep/2023 video of this keynote title, to Devoxx Ukraine).
Introducing BASIS, the Betts artificial superintelligence suite, aimed at measuring the smartest AI in the world against the smartest human-created test items in the world.
It was great to work with Mensan, founder of the World Genius Directory, and ultra-high-ceiling test designer Dr Jason Betts on this project, a benchmarking suite for artificial superintelligence. There were some really funky specs, like having all items designed in a ‘clean room’ and then sealed in a locked bag. We both felt a bit like spies!
The suite is designed for testing frontier models like Gemini and GPT-5, aiming for IQ=180.
The suite is being made available to AI research labs around the world, on request.
Read more: https://lifearchitect.ai/basis/
Watch the livestream: https://youtube.com/live/DGPMJN0sskQ
Two alternative benchmarks were also released in November 2023:
GAIA by Meta/HuggingFace, media level for IQ=100. (paper, dataset)
GPQA by NYU/Cohere/Anthropic, expert level for IQ=125. (paper, dataset)
Note that both GAIA and GPQA:
are not for the higher ceiling of artificial superintelligence
were not air-gapped
are already compromised, with datasets released and available online
OpenAI: Compute is king (20/Nov/2023)
The numbers for compute and $ being thrown around are getting crazy.
We’ve gone from OpenAI having access to 25,000 GPUs (Morgan Stanley, Feb/2023), a mix of A100s and H100s, to tripling that number and focusing exclusively on H100s (according to NVIDIA, the H100 is up to nine times faster for AI training). We’re in for a wild ride.
OpenAI, with plans for >$50B annual datacenter spend to race to AGI… one of OpenAI’s next training supercomputers in Arizona was going to have more than 75,000+ GPUs in a singular site by the middle of next year.
Our data also shows us that Microsoft is directly buying more than 400,000 GPUs next year [2024] for both training and copilot/API inference. Furthermore, Microsoft also has tens of thousands of GPUs coming in via cloud deals with CoreWeave, Lambda, and Oracle.
NVIDIA sold half a million H100 AI GPUs in Q3 thanks to Meta, Facebook — lead times stretch up to 52 weeks: Omdia (28/Nov/2023)
NVIDIA reportedly sold nearly half a million of its H100 GPUs for AI and high-performance computing in the third quarter of 2023, largely due to purchases by Meta and Facebook. The demand is so high that the lead time for H100-based servers now ranges from 36 to 52 weeks.
Let’s not forget that TSMC are the ones doing all the work here (read more in The Memo edition 17/Aug/2023). And the entire planet Earth is the critical path, every resource on that path seems to be running at 100% capacity. If only we had some sort of AGI to help us out here… (Actually, NVIDIA have already implemented AI for their latest chip design, Oct/2023).
Alan’s calcs: 500,000x GPUs sold in 90 days
= 5,555x GPUs sold per day
and, related:
500,000x GPUs at US$40,000/each
= $20,000,000,000 ($20B)
The Interesting Stuff
Exclusive: 17 new model highlights for November 2023 (Nov/2023)
While research continues at the rate of one new AI paper published every ~8 minutes (see The Memo edition 9/Jul/2023), we’re also seeing one new major model released every ~42 hours on average.
In 2023, November saw 17 major model announcements. As a comparison, October was 10 models, and September was 18 models. Some parameter counts are estimated. Here they are:
01-ai Yi-34B, xAI Grok-0 (33B), xAI Grok-1 (33B), Samsung Gauss (7B), NTU OtterHD-8B, Google DeepMind Mirasol3B (3B), Microsoft Florence-2 (0.771B), Microsoft phi-2 (2.7B), Microsoft Orca 2 (13B), Allen AI TÜLU 2 (70B), Anthropic Claude 2.1 (130B), Inflection AI Inflection-2 (1.2T), Berkeley Starling-7B, Microsoft Transformers-Arithmetic (0.1B), EPFL MEDITRON (70B), IEIT Yuan 2.0 (102B), Google DeepMind Q-Transformer.
I also have a new and publicly-available GPT-4 bot helping out with the comma separated formatting above: https://poe.com/TheMemoModelsBot
See more about each of these models on the Models Table: https://lifearchitect.ai/models-table/
ChatGPT's 1-year anniversary: how it changed the world (30/Nov/2023)
VentureBeat takes a retrospective look at how OpenAI's ChatGPT has influenced the world one year since its launch, noting its rapid adoption and the controversies it has sparked, leading to debates about the role of large language models in society.
That article is a very strange and limited look at ChatGPT’s accomplishments.
See ChatGPT’s achievements on my shared sheet.
Coke executive mentions GPT-5 (23/Nov/2023)
Manolo Arroyo (Global Chief Marketing Officer for The Coca‑Cola Company):
I can give you maybe an insight, some pieces of new news that no one has shared so far. We have a partnership with Bain and OpenAI...
We were actually the first company that was combining GPT, which is the engine that enables ChatGPT, and DALL-E. Back then, no one knew that because of the partnership with OpenAI, we were the first company using GPT-4 and DALL-E 2 into one integrated consumer digital experience. No one knows, because it hasn't been launched yet, that Coca Cola Diwali has been done with GPT-5 which is still not commercially available, and DALL-E 3 that has also not been launched...
And that's how in just six months this technology is progressing... launch it for Christmas [2023] globally...
(— Coca-Cola’s Mega Marketing Transformation by The Morning Brief (The Economic Times) at 17m31s)
It’s possible that Manolo is referring to GPT-V(ision) or GPT-4.5. It is unlikely that he actually means GPT-5, which OpenAI’s CEO has promised (including under oath) would not start training until December 2023.
Read more: https://lifearchitect.ai/gpt-5/
2022: Anthropic pranked OpenAI with thousands of paper clips to warn about AI apocalypse (23/Nov/2023)
An employee from Anthropic, a rival of OpenAI, sent thousands of paper clips in the shape of OpenAI’s logo to OpenAI’s office. The prank alluded to the possibility that OpenAI’s approach to AI could lead to humanity’s extinction, referencing the ‘paper clip maximizer’ scenario (wiki) by philosopher Nick Bostrom.
The OpenAI Turkey-Shoot Clusterfuck (1/Dec/2023)
I was being polite calling it a circus in the last edition (The Memo 23/Nov/2023). Journalist Charles Duhigg was inside OpenAI offices during the ‘five-day crisis that some people at Microsoft began calling the Turkey-Shoot Clusterfuck.’
Some members of the OpenAI board had found Altman an unnervingly slippery operator. For example, earlier this fall he’d confronted one member, Helen Toner, a director at the Center for Security and Emerging Technology, at Georgetown University, for co-writing a paper that seemingly criticized OpenAI for “stoking the flames of AI hype.” Toner had defended herself (though she later apologized to the board for not anticipating how the paper might be perceived).
Altman began approaching other board members, individually, about replacing her. When these members compared notes about the conversations, some felt that Altman had misrepresented them as supporting Toner’s removal. “He’d play them off against each other by lying about what other people thought,” the person familiar with the board’s discussions told me.
“Things like that had been happening for years.” (A person familiar with Altman’s perspective said that he acknowledges having been “ham-fisted in the way he tried to get a board member removed,” but that he hadn’t attempted to manipulate the board.)
Read the source New Yorker article.
Read former board member Helen Toner’s statement via Twitter.
Sidenote: Given that Reuters blamed the board coup on Q* and other AGI ‘breakthroughs’ (23/Nov/2023), it just may be that Reuters (and journalists Anna Tong, Jeffrey Dastin, and Krystal Hu) is no longer a reliable source.
As a reminder, the size of the gap between an AI lab having new tech and the public having access to that tech will be due to ‘other human factors’, such as company leaders behaving in very human ways…
Read more: https://lifearchitect.ai/gap/
OpenAI blog: Sam Altman returns as CEO, OpenAI has a new initial board (29/Nov/2023)
OpenAI has finally released a statement about the board and company mission.
TL;DR: nothing has really changed (well, Ilya is out, I suppose).
Sam Altman has returned as CEO of OpenAI, with Mira Murati serving as CTO and Greg Brockman returning as President. The new initial board consists of Bret Taylor (Chair), Larry Summers, and Adam D’Angelo.
Microsoft will have a non-voting seat, and has not revealed the name of this member.
Read more: https://openai.com/blog/sam-altman-returns-as-ceo-openai-has-a-new-initial-board
Read former board member Helen Toner’s statement via Twitter.
Realtime generative AI art is here thanks to LCM-LoRA (16/Nov/2023)
A new machine learning technique, known as LCM-LoRA (Latent Consistency Model Low-Rank Adaptation), has been developed by researchers at Tsinghua University and HuggingFace. This technique enables the generation of AI art in real time, which is a significant leap forward from the previous wait times of a few seconds to minutes.
Try it: https://huggingface.co/spaces/ilumine-AI/LCM-Painter
Introducing SDXL Turbo: a real-time text-to-image generation model (28/Nov/2023)
If you feel that Stability AI only just released a new model recently, you’re right. And here’s another one!
Stability AI has released SDXL Turbo, a real-time text-to-image generation model that uses a novel distillation technique called Adversarial Diffusion Distillation. The model synthesises image outputs in a single step, while maintaining high sampling fidelity.
The live demo video is… unexpected.
Try it via Clipdrop (login): https://clipdrop.co/stable-diffusion-turbo
Watch the video (link):
Amazon Q: Generative AI powered assistant (28/Nov/2023)
Amazon has introduced Amazon Q, a generative AI–powered assistant designed for work, which can streamline tasks, speed decision-making, and spark creativity and innovation at work. The AI can be personalized to your business, understands your company's information, code, and system, personalizes interactions based on your role and permissions, and is built with security and privacy in mind.
Amazon Q offers 40+ built-in connectors to popular enterprise applications and document repositories, including S3, Salesforce, Google Drive, Microsoft 365, ServiceNow, Gmail, Slack, Atlassian, and Zendesk.
The S3 connector is a fairly serious differentiator, and may set Amazon apart in this space. A huge percentage of F500s use AWS and S3, and providing this functionality allows them to bring in their entire knowledge base with a single click (or maybe two!).
While Amazon has their Titan model (my link) available, they also offer access to many other models via Amazon Bedrock:
Amazon Bedrock offers easy access to a choice of high-performing foundation models from leading AI companies, including AI21 Labs [Jurassic-2], Anthropic [Claude], Cohere [Command], Meta [Llama 2], Stability AI [Stable LM], and Amazon [Titan, and the upcoming Olympus 2T]. (— via Amazon)
Read more via Amazon Web Services.
Pika 1.0 (29/Nov/2023)
Pika labs have announced their text-to-video and image-to-video model, and it looks amazing. We are one step closer to generating entire feature length films in time for dinner.
Read the announce: https://twitter.com/pika_labs/status/1729510078959497562
Short film by AI (29/Nov/2023)
The latest AI video outputs are so good that we are moments away from text to movie. Notice the little camera movements in this new AI film. I expect that this kind of output (not by Pika) could be output by a model providing script and scaffolding (like GPT-4), and then the text-to-video model could generate a full movie in a matter of seconds…