The Memo - 19/May/2023

AGI in 2 years (OpenAI/DeepMind researcher), Anthropic 75,000-word memory, embodiment embodiment embodiment, and much more!

May 18, 2023

∙ Paid

FOR IMMEDIATE RELEASE: 19/May/2023

For four billion years the ecological system of planet Earth contained
only organic life forms and now—or soon—we might see
the emergence of the first inorganic lifeforms…
— Yuval Noah Harari (29/Apr/2023)

Welcome back to The Memo.

The winner of our AI Cheese award for May 2023 is Seth Rogen and the writers strike. The runner-up was the poor professor who failed all his students after feeding their essays into ChatGPT, asking ChatGPT whether each essay was composed by it, and believing ChatGPT when it said ‘Yes, I wrote this.’ (17/May/2023). The professor’s mistake was embarrassing; Rogen’s comments are… permanently on record.

Take a breather and listen to how utterly absurd AI has become in California, the media, and around the world. Google didn’t have to be this late to the party. They invented the entire architecture back in Jun/2017—so they’re coming up on their six year anniversary—and they’ve had the lead for AI in private research for all that time.

Watch Google finally rushing to commercialize a technology where they had been #1 (at least internally!) for years and years (2m32 seconds of silliness):

In the Policy section, we draw out the main recommendations from OpenAI’s recent proceedings with the United States Congress, as well as the latest version of the European AI Act with large language models and their technical documentation now in scope.

In the Toys to play with section, we look at an amazing video example by Runway’s Gen2, using virtual reality and ChatGPT for coaching social skills, and much more.

I am also excited to provide full subscribers with behind-the-scenes access to my next big video, featuring one of my very favourite Cambridge experts (video at the end of this edition).

The BIG Stuff

OpenAI (ex-DeepMind) researcher says full AGI is 2 years away (4/May/2023)

A researcher has matched my predictions of achieving full artificial general intelligence (AGI) including embodiment some time in 2025 (I say ‘a few months’ rather than ‘a few years’ from early 2023, taking us to 2025).

OpenAI Governance Researcher and former DeepMind AGI Safety Engineer Richard Ngo’s summary focuses more on AI’s mental performance than physical performance. Here are two of my favourite points from Richard’s summary, asserting that by the end of 2025, AGI will:

Have human-level situational awareness (understand that they're [neural networks, NNs], how their actions interface with the world, etc; see definition here)
5% of adult Americans will report having had multiple romantic/sexual interactions with a chat AI, and 1% having had a strong emotional attachment to [a chat AI].

It’s an eye-opening read.

Read Richard’s summary.

Compare my countdown to AGI.

Google PaLM 2 340B (10/May/2023)

Last week, we released a special edition of The Memo highlighting the big stuff in Google’s PaLM 2, a competitor to OpenAI’s GPT-4. CNBC recently confirmed (17/May/2023) that PaLM 2 has a parameter count of 340B, trained on 3.6T tokens.

I’ll be livestreaming about PaLM 2 in an hour or two from this email.

Med-PaLM 2 (16/May/2023)

A version of Google’s PaLM 2 model was trained by Google’s health research teams on medical knowledge, with the result being Med-PaLM 2. This is an advance on the original Med-PaLM from Dec/2022:

[On 26/Dec/2022, Med-PaLM 1] performs encouragingly, but remains inferior to clinicians.

Just four months and 20 days after that first version (with some hints back in Mar/2023), today’s Med-PaLM 2 shows incredible performance, clearly outlined in the paper:

[On 16/May/2023, Med-PaLM 2 shows that] physicians preferred Med-PaLM 2 answers to those produced by physicians…

You’ve heard the phrase ‘accelerating pace of change’. You’ve seen the charts and predictions during the last few years:

The difference between Med-PaLM 1 and Med-PaLM 2 showcases the current trajectory of AI progress and what is possible right now in just 20 weeks. From ‘below average’ to ‘above expert’ level. In 20 weeks. Hmmm….

Google DeepMind Gemini currently training (11/May/2023)

We’re already at work on Gemini — our next model created from the ground up to be multimodal, highly efficient at tool and API integrations, and built to enable future innovations, like memory and planning. Gemini is still in training, but it’s already exhibiting multimodal capabilities never before seen in prior models. Once fine-tuned and rigorously tested for safety, Gemini will be available at various sizes and capabilities, just like PaLM 2, to ensure it can be deployed across different products, applications, and devices for everyone’s benefit.

GPT-5 not currently training. Yet. Maybe December 2023. (16/May/2023)

Under oath, OpenAI’s CEO confirmed that GPT-5 is not currently training. Perhaps they’re waiting for a few thousand more NVIDIA H100s to be made available.

We are not currently training what will be GPT-5; we don’t have plans to do it in the next six months.

Read my updated summary of GPT-5: https://lifearchitect.ai/gpt-5/

Anthropic: Introducing 100K (75K words) Context Windows (11/May/2023)

Beyond just reading long texts, Claude can help retrieve information from the documents that help your business run. You can drop multiple documents or even a book into the prompt and then ask Claude questions that require synthesis of knowledge across many parts of the text. For complex questions, this is likely to work substantially better than vector search based approaches. Claude can follow your instructions and return what you’re looking for, as a human assistant would!

A ‘context window’ is a bit like a human’s ‘working memory’; it’s how far back AI can recall the current context in a conversation.

GPT-3 had a context window of about 1,500 words.

Now, Anthropic has revealed a context window 50x bigger than that: 75,000 words. 75,000 words of input/output allows writers and researchers to do a lot. Feed it an entire novel and get a query-able brain. Give it a one-sentence idea for a dissertation and get… the final thesis!

The Interesting Stuff

Meta AI leaning towards open-source (18/May/2023)

Meta said it saw no reason to keep its code to itself. The growing secrecy at Google and OpenAI is a “huge mistake,” Dr. LeCun said, and a “really bad take on what is happening.” He argues that consumers and governments will refuse to embrace A.I. unless it is outside the control of companies like Google and Meta.
“Do you want every A.I. system to be under the control of a couple of powerful American companies?” he asked…
He said the evolution of the consumer internet was the result of open, communal standards that helped build the fastest, most widespread knowledge-sharing network the world had ever seen.
“Progress is faster when it is open,” he said. “You have a more vibrant ecosystem where everyone can contribute.”

Read more via NYT: https://nyti.ms/3Wf1yIc

Read Lecun’s verbose commentary on Twitter.

Embodiment (May/2023)

Embodiment in AI means working with real world physical systems. In plain English, that means robots. And not just any robots. Autonomous, independent, aware robots, backed by large language models, with zero scripting or pre-programming. A very simple version would be like putting GPT-3 inside your Roomba to let it sense the world and make decisions. At its most advanced, imagine putting GPT-4 inside a Boston Dynamics Atlas robot (video from Jan/2023). It’s already being tested in an AI lab right now.

And this edition is completely dominated by LLMs inside robots. It must be robot week!

Embodiment: TidyBot (9/May/2023)

Read the paper: https://arxiv.org/abs/2305.05658

Watch the demos: https://tidybot.cs.princeton.edu/

TidyBot’s smarts nudged me to up my AGI progress from 48% to 49%: https://lifearchitect.ai/agi/

Embodiment: Sanctuary AI Phoenix (16/May/2023)

Sanctuary AI, a company on a mission to create the world’s-first human-like intelligence in general-purpose robots, announced a major step forward with the unveiling of its sixth-generation general-purpose robot named Phoenix™. Phoenix is the world’s first humanoid general-purpose robot powered by Carbon™, a pioneering and unique AI control system, designed to give Phoenix human-like intelligence and enable it to do a wide range of work to help address the labor challenges affecting many organizations today. Sanctuary has been able to show that its technology is already capable of completing hundreds of tasks identified by customers from more than a dozen different industries.

Wow, that is an ugly robot. Put some clothes on it (like OpenAI’s 1X NEO!).

Read the announce.

Embodiment: Tesla Bot update (17/May/2023)

Tesla has issued an update on their humanoid robot this week as well. Watch for an insight into their lab…

Embodiment: Amazon ‘Burnham’ embodied in ‘Astro’ (13/May/2023)

…a secret new AI robot project, internally codenamed "Burnham,"… adds a layer of "intelligence and a conversational spoken interface" to Astro, according to internal documents obtained by Insider. Called "Contextual Understanding," Amazon describes the new technology as "our latest and most advanced AI technology designed to make robots more intelligent, more useful, and conversational," the documents show…
For instance, if Burnham finds a stove left burning or a water faucet running unattended, it will find the owner and bring it to their attention, the document said.
If an elderly man slips in the kitchen, Burnham can check to make sure he's ok, and call others to come help. If it's an emergency situation, Burnham will automatically call 911.
Owners can ask Burnham where they left their keys.
It can check whether the kitchen window was left open last night.
The robot can also monitor whether the kids had friends over after school.

Policy

OpenAI at US Congress (16/May/2023)

OpenAI’s CEO, Sam Altman, spoke about AI technology at a Senate subcommittee hearing on Tuesday 16/May/2023, his first appearance before Congress. (For more on OpenAI and their links to the US Government including the CIA, see The Memo edition 11/Apr/2023.)

The proceedings were a bit of a farce, but an interesting read nonetheless, particularly as they lay bare the ineptitude of the US Government when it comes to understanding the revolutionary technology that is post-2020 AI.

Congress: Please tell me, in plain English, two or three reforms or regulations, if any, that you would implement if you were queen or king for a day.
Sam Altman (speaking specifically about large, multi-million dollar company models):
I would form a new agency that licenses any effort above a certain scale of capabilities (measured roughly by training compute), and can take that license away and ensure compliance with safety standards.
I would create a set of safety standards focused on what you said in your third hypothesis as the Dangerous Capability evaluations. One example that we've used in the past is looking to see if a model can self replicate and self exfiltrate into the wild. ... Specific tests that a model has to pass before it can be deployed into the world.
I would require independent audits, so not just from the company or the agency, but experts who can say the model is, or isn't, in compliance with these stated safety thresholds and these percentages of performance on question X or Y.

The Memo by LifeArchitect.ai

The Memo - 19/May/2023

AGI in 2 years (OpenAI/DeepMind researcher), Anthropic 75,000-word memory, embodiment embodiment embodiment, and much more!

The BIG Stuff

The Interesting Stuff

Policy

This post is for paid subscribers