The Memo - 18/Oct/2023
Baidu ERNIE 4.0, NVIDIA B100 and X100, UBI in North America, and much more!
FOR IMMEDIATE RELEASE: 18/Oct/2023
Marc Andreessen (16/Oct/2023):
Artificial Intelligence is our alchemy, our Philosopher’s Stone – we are literally making sand think… Any deceleration of AI will cost lives. Deaths that were preventable by the AI that was prevented from existing is a form of murder. We believe in Augmented Intelligence just as much as we believe in Artificial Intelligence. Intelligent machines augment intelligent humans, driving a geometric expansion of what humans can do.
Welcome back to The Memo.
You’re joining paid subscribers from Microsoft, MIT, Moodle, Monash, Mercado Libre, MYOB, Mastercard, and more…
This is another long edition. In the Toys to play with section, we look at a new collection of OpenAI’s hidden system prompts used every time you interact with their platforms, South Park’s AI episodes, Meta AI’s chatbots now in WhatsApp (US), and much more.
The next roundtable will be:
Life Architect - The Memo - Roundtable #4
Follows the Chatham House Rule (no recording, no outside discussion)
Saturday 4/Nov/2023 at 5PM Los Angeles
Saturday 4/Nov/2023 at 8PM New York
Sunday 5/Nov/2023 at 8AM Perth (primary/reference time zone)
or check your timezone via Google.
Details at the end of this edition.
Reggie Watts was dropping my name recently in an interview with some former UFC commentator (link).
I think he is referring to the entire body of work at LifeArchitect.ai (that’s around 100+ papers, 350+ videos, and 100+ editions of The Memo), but if you’re looking for policy references you’ll find them in every edition for the last year or so. Three big editions mentioning AI and government are:
The Memo - 7/Mar/2023: Romanian PM using a ChatGPT alternative behind a mirror.
The Memo - 30/Apr/2023: US/EU/Japan AI acts, Palantir AIP.
The Memo - 17/Aug/2023: Where will AGI begin?, US + China analysis.
The BIG Stuff
OpenAI: AGI is now a ‘core value’ (12/Oct/2023)
GPT creator OpenAI has subtly revised its ‘Core values’ on its website, placing a more significant emphasis on the development of AGI — artificial general intelligence.
Artificial general intelligence is the equivalent of a median human that you could hire as a co-worker.
— OpenAI CEO, NYMag (25/Sep/2023)
The organization's revamped values now include ‘AGI focus,’ stating ‘Anything that doesn't help with that is out of scope.’
Since the 2020 release of GPT-3 (watch Leta AI), I have been telling friends and colleagues as clearly as I can: ‘Drop everything, and focus on AI!’ Governments, companies, individuals… sit up and take notice. Anything outside of AI is just a distraction. (Of course, living life and being human is still important, but investing time in things like committees and land wars and climate change and voices in parliament and electric vehicle upgrades is a waste of time, when superintelligence will easily give us a 1,000x advantage in solving all of these and more.)
See the core values: https://openai.com/careers
Read more (paywall): https://www.semafor.com/article/10/12/2023/openai-quietly-changed-its-core-values
See my conservative countdown to AGI (at 55% in Oct/2023): https://lifearchitect.ai/agi/
OpenAI: Even more GPT-5 rumors (18/Oct/2023)
Users Jimmy Apples and FeltSteam are back. Here’s what I’ve gleaned:
The new OpenAI internal project names based on deserts may reflect the fact that these models are ‘sparse,’ like a desert.
Sahara = GPT-3.5 (ChatGPT).
Gobi = GPT-5-related.
Arrakis = GPT-5-related (maybe failed/suspended due to low model quality).
Screenshots of rumors via Twitter.
Read my GPT-5 summary: https://lifearchitect.ai/gpt-5/
NVIDIA Blackwell B100 GPUs launches in Q2 2024 due to rise in AI demand (15/Oct/2023)
NVIDIA has reportedly moved the launch of its next-gen Blackwell B100 GPUs up from Q4 to Q2 2024 following a huge surge in AI demand.
The market predicts that B100 will be a more powerful AI game changer than H100, Nvidia's current highest-spec GPU. This product is mainly used in AI cloud and supercomputing. NVIDIA accounts for more than 90% of the AI GPU market share.
These expanded timelines are big news for NVIDIA and the AI labs (and humanity!). Here is a rapid successor to their H100 already, and not just for next year (2024) with the B100, but the following year (2025) with the X100. This is not-so-good news for Intel and AMD, who are still playing catchup to the older NVIDIA A100.
Both the B100 and X100 GPUs will run the world. Count on it.
Read source: https://wccftech.com/nvidia-blackwell-b100-gpus-sk-hynix-hbm3e-memory-launches-q2-2024-rise-in-ai/
China's Baidu unveils new ERNIE AI version to rival GPT-4 (17/Oct/2023)
ERNIE stands for ‘Enhanced Representation from kNowledge IntEgration’. The Chinese name is 文心一言, or Wenxin Yiyan.
Chinese tech giant Baidu has introduced ERNIE 4.0, its newest generative AI model, which it claims is on par with OpenAI's GPT-4.
China now has at least 130 large language models (LLMs), representing 40% of the global total and behind only the United States' 50%, data from brokerage CLSA showed.
We covered 103 new Chinese LLMs in The Memo edition 27/Jul/2023.
Once again, another AI lab has chosen to guard its model architecture. It is assumed that the model is a sparse mixture-of-experts (MoE). While ERNIE 3.0 was only 260B parameters, ERNIE 4.0 is said to have more than 1 trillion parameters, on par with GPT-4 1.76T.
See Baidu’s launch video with English voiceover.
Read my brief ERNIE summary: https://lifearchitect.ai/ernie/
And see Baidu on the AGI viz: https://lifearchitect.ai/agi#where
The Interesting Stuff
AGI achieved internally: A re-written story (Life 3.0 annotated, with replacements for the Omegas and Prometheus) (16/Oct/2023)
I had a bit of fun working with GPT-4 to convert Max Tegmark’s famous story about achieving AGI from his generic names of ‘Omega’ and ‘Prometheus’ to OpenAI/Google and GPT-5/Gemini 2. It’s a 20-minute read, and I’m sure you’ll recognize some of it as happening already…
Over a timescale of months, the business empire controlled by the OpenAI team started gaining a foothold in ever more areas of the world economy, thanks to superhuman planning by GPT-5. By carefully analyzing the world’s data, it had already during its first week presented the OpenAI team with a detailed step-by-step growth plan, and it kept improving and refining this plan as its data and computer resources grew. Although GPT-5 was far from omniscient, its capabilities were now so far beyond human that the OpenAI team viewed it as the perfect oracle, dutifully providing brilliant answers and advice in response to all their questions.
Read the story (6,000 words, 20mins): https://lifearchitect.ai/agi-achieved-internally/
Exclusive: OpenAI may release text-embedding-ada-003 at dev day on 6/Nov/2023 (5/Oct/2023)
A recent GitHub pull request (5/Oct/2023) suggests that there is a new embeddings model, text-embedding-ada-003. It may be that OpenAI releases this during the upcoming dev day on 6/Nov/2023. ‘Ada’ is the smallest publicly-available model in the GPT-3 family, at only 350M parameters. The text-embedding-ada-002 model has been the standard consolidated embedding model since the beginning of 2023.
I am not expecting any major releases at OpenAI dev day, as the CEO has said that there will not be any: ‘on november 6, we’ll have some great stuff to show developers! (no gpt-5 or 4.5 or anything like that, calm down, but still i think people will be very happy…)‘ (7/Sep/2023).
Read the PR: https://github.com/microsoft/semantic-memory/pull/78
OpenAI dev day: https://openai.com/blog/announcing-openai-devday
See my older view (Mar/2023) of the GPT-3 family: https://lifearchitect.ai/gpt-3/
Exclusive: OpenAI partners with Dropbox, but your data is ‘never’ used to train models (Oct/2023)
I think some people (including myself, initially) were concerned about this partnership, and confused about the implications. They are now much clearer in the Dropbox T&Cs:
What [Dropbox] information is shared with third-party partners?
Your files within Dropbox are sent to a third-party AI only when you chose to interact with AI powered features. For example, when you ask a question about a file. At this time, we’re [Dropbox] partnered with one third-party AI partner, OpenAI. Open AI is an artificial intelligence research organization that develops cutting-edge language models and advanced AI technologies. Your data is never used to train their [OpenAI’s] internal models, and is deleted from OpenAI’s servers within 30 days.
Read the Dropbox T&C section on AI.
2023 State of AI report released (12/Oct/2023)
Every year since 2021, there are three big AI reports released, each with a different objective and focus:
The sky is… by LifeArchitect.ai (recent release for mid-year Jun/2023).
Artificial Intelligence Index Report by Stanford (recent release Apr/2023).
State of AI by Air Street Capital (recent release Oct/2023).
The State of AI report is early (12/Oct/2023), and a useful read for those involved in setting policy or making strategic decisions. I enjoyed this funding summary slide on page 119:
Read the 2023 State of AI report.
I expect to have my end-of-year AI report, ‘The sky is…’ released in about 7-8 weeks from now, in December 2023. As usual, paid subscribers of The Memo receive the report first.
New LLM leaderboard using a set of 60 prompts (Sep/2023)
The HuggingFace leaderboard is practically useless given its reliance on TruthfulQA (often negatively correlated with response quality). This alternative seems much more pragmatic.