The Memo - 17/Feb/2023

Luminous Supreme Control 70B (instruct), Replit Ghostwriter, China's chatbot progress, and much more!

Feb 16, 2023

FOR IMMEDIATE RELEASE: 17/Feb/2023

Welcome back to The Memo.

This is a big edition for people who love to read. There is the equivalent of several books worth of new AI content to digest here, and you’ll get enormous value if you make the time!

In the Toys to play with section, we look at a private Tim Ferriss chatbot (free), a new AI programming tool being called ‘ChatGPT for coders’, and my recommended MIT course on AI (free).

The BIG Stuff

Google Bard overview updated (Feb/2023)

I have spent some time expanding my overview of Google Bard. If all goes well, this model will absolutely change search for you and the rest of Google’s 4 billion+ users who perform 8 billion+ searches a day.

Take a look: https://lifearchitect.ai/bard/

The Interesting Stuff

Report: Language Models And Cognitive Automation For Economic Research by Anton Korinek (Feb/2023)

It was great to be cited in a paper this month for the National Bureau of Economic Research (NBER), a US economic advisory think-tank linked to MIT, with members including Myron Scholes (yes, the Black-Scholes guy) and Bernanke.

LLMs have just crossed the threshold to become useful across a wide range of cognitive tasks—as was illustrated by the viral reception of ChatGPT, a version of OpenAI's GPT3.5 model that was released on November 28th, 2022, gained more than 100m users in its first two months, and is now estimated to produce a volume of text every 14 days that is equivalent to all the printed works of humanity (Thompson, 2023).
…a former Chairman for Mensa International reports that ChatGPT has a tested IQ of 147 (99.9th percentile) on a verbal-linguistic IQ test (Thompson, 2023). Moreover, whereas the level of human intelligence is relatively static, LLMs are advancing rapidly, becoming more accurate and powerful with every new iteration.

The full 35-page report provides 26 applications for GPT-3 with examples. It is an excellent read.

Download PDF: direct, archive.

What Is ChatGPT Doing … and Why Does It Work? by Dr Stephen Wolfram (14/Feb/2023)

Stephen is a former child prodigy, and we’ve mentioned his work already in several editions of The Memo.

His latest article is nearly 20,000 words, about the length of a small book. If you really want to understand ChatGPT and neural networks more deeply, and in plain-enough English (with pictures), this is a great resource.

Read it: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/

OpenAI investment report by Contrary.com (Jan/2023)

Some of the numbers need review, but for interest, this 22-page report summarizes most of the ‘need to know’ about OpenAI as a company.

View: https://research.contrary.com/reports/openai

Download PDF: https://app.box.com/s/vgwmimnkg7gfuafak4levm2lm8bsro16

Luminous Supreme Control 70B (instruct) released (14/Feb/2023)

Our steerable model Luminous-supreme-control [70B] has been optimized to work well with zero-shot instructions. This means that they do not necessarily need a set of examples like in few-shot learning.

Try it out: https://app.aleph-alpha.com/playground/completion

You can also use this new model via Riku.ai (discount code available to paid subscribers of The Memo in your ‘welcome’ email): https://riku.ai/

OpenAI working on a new version of ChatGPT (17/Feb/2023)

We believe that AI should be a useful tool for individual people, and thus customizable by each user up to limits defined by society. Therefore, we are developing an upgrade to ChatGPT to allow users to easily customize its behavior.

Read: https://openai.com/blog/how-should-ai-systems-behave/

Download OpenAI’s ChatGPT guidelines ‘snapshot’ (PDF, 3 pages).

OpenAI CEO on ethics and personalized AI (10/Feb/2023)

Where we are right now is not where we want to be. The way this should work is that there are extremely wide bounds of what these systems can do, that are decided by, not Microsoft or OpenAI, but society, governments, something like that, some version of that; people, direct democracy…
And then within those bounds, users should have a huge amount of control over how the AI behaves for them, because different users find very different things offensive or acceptable or whatever. And no company, certainly not either of ours [OpenAI or Microsoft], I think, is in a place where they want to, or should want to say, ‘here are the rules’.
…Again, it’s very new technology. We don’t know how to handle it. And so we’re being conservative. But the right answer, here, is very broad bonds, set by society, that are difficult to break, and then ‘user choice’.
And also, having a vibrant community of people who are trying to press on the limits of the technology, to find where it breaks and where it does unusual, interesting things, is good.

Listen via NYT: https://www.nytimes.com/2023/02/10/podcasts/bings-revenge-and-googles-ai-face-plant.html

AI.com purchased for $11M, redirects to ChatGPT (15/Feb/2023)

This domain now redirects to chat.openai.com: ai.com

Confirmed.

Where is China in all this rush for AI chatbots? (Feb/2023)

The Western media understandably focuses on the small pocket of Silicon Valley. However, there is a lot going on outside of those 5,000km² (1,800miles²)…

Alibaba: chatbot based on M6 (10T sparse) or AliceMind coming soon.

Baidu: ERNIE Bot (260B) coming soon.

Huawei: chatbot based on PanGu-α (200B) patented.

JD.com: ChatJD coming soon.

PICO (Bytedance): chatbot rumoured.

Tencent: chatbot based on HunYuan rumoured.

Xiaoice (ex-Microsoft): chatbot rumoured.

I have some older visualizations (2021) of some of these models at: https://lifearchitect.ai/models/#contents-chinese

(I just realized how funny it is to refer to 2021 as ‘older’ in this fast-paced field!)

ChatYuan: China’s first ChatGPT clone (11/Feb/2023)

Chinese technology companies are taking heed of the country’s strict online moderation policy, as they rush to develop their own versions of ChatGPT, the viral chatbot service of US firm OpenAI.
[CEO of Yuanyu said] “It’s different in China, compared with overseas… We need more layers of filtering and processing in terms of text review.”
Founded last year, Yuanyu Intelligent in January launched ChatYuan, touted as the first generative AI pre-trained by Chinese-language models, ahead of Baidu’s own “Ernie Bot”. Generative AI refers to algorithms that can be used to create new content, including text, audio, images, video and simulations.
ChatYuan ran as a mini-program on Tencent Holdings’ super app WeChat, where users accessed the technology without downloading separate software. Operations of ChatYuan were recently suspended for maintenance updates, but it is expected to resume next week [around 20/Feb/2023].

Website (Chinese): https://www.clueai.cn/

Try it out (model down for maintenance): https://www.clueai.cn/chat

The Great Flood (Dec/1997)

Nearly every time I speak with the media, I want to show them this graphic (and have done so in some video calls). It is a view of the AI landscape in 1997, in a section called ‘The Great Flood’ by the robotics researcher Hans Moravec. It was illustrated and appeared in Max Tegmark’s 2017 book, Life 3.0.

AI: The Great Flood (1997) / The Landscape of Human Competence (2017)

As of Feb/2023, AI has now flooded or encroached on most of this landscape, perhaps with the exception of full cinematography, social interaction, and management.

I’ve pulled together the original sources, with some narrative to bring it all together into 2023: https://lifearchitect.ai/flood/

Mega-engineer develops Copilot for piano (14/Feb/2023)

Dr Linxi ‘Jim’ Fan has a CV longer than mine! Former AI researcher at Google Cloud AI, OpenAI, Baidu Silicon Valley AI Lab, Columbia University, and Mila-Quebec AI Institute, now AI scientist at NVIDIA, wrote the award-winning MineDojo paper, wrote the VIMA paper, and more…

This tech is based on RNN theory from 2018 (pre-Transformer), called Piano Genie (paper), but check out the absurd results.

There are eight colored keys for the human to play, and the neural network is mapping this to what would sound good on a full 88-key grand piano for an AI to play…

Jim Fan @DrJimFan

We’ve seen a gazillion startups using OpenAI APIs to do “co-pilot for X”. What’s next? Enter *physical* co-pilot! Here’s a compelling demo: you improvise by playing a “low resolution” piano, and the co-pilot compiles it real-time to Hi-Fi music! It unleashes our inner pianist.🧵

NASA using AI to design mission hardware (11/Feb/2023)

‘…the AI connects the dots… producing complex structure designs in as little as an hour or two… the parts generated by the algorithm don’t have the stress concentrations that you have with human designs. The stress factors are almost ten times lower than parts produced by an expert human.’

F-16 flies with AI pilot (13/Feb/2023)

An AI agent flew a USAF training aircraft for over 17 hours

Last year, an AI agent beat a real US Air Force instructor in a virtual dogfight conducted in flight simulation. And in December, DARPA managed to use this new technology to help fly a real F-16 jet, closing the simulation to real-world gap.
Air Force Lt. Col. Ryan "Hal" Hefron, the DARPA program manager for ACE, said the AI algorithms were used during takeoff and landing to fly the VISTA [modified F-16D] aircraft.
…DARPA worked with scientists and engineers from academia and industry, including EpiSci, PhysicsAI, Shield AI, and the Johns Hopkins Applied Physics Laboratory to test different types of AI algorithms.

Read more via The Register: https://www.theregister.com/2023/02/14/ai_air_force_f16/

Amazon Zoox (14/Feb/2023)

It’s not as cool as the Waymo or XPeng public stuff, but worth mentioning because it is yet another contender in the AI car market.

Amazon's self-driving company Zoox unveiled its autonomous robotaxi on Monday.

The tests are currently limited to shuttling Zoox employees on a one-mile public route between two office buildings at the company’s headquarters in Foster City, California, at speeds up to 35 miles an hour… Amazon acquired the 9-year-old startup in 2020 and, at the time, shared few details about how it planned to use the company’s technology. Zoox unveiled its custom-built, electric robotaxi in 2020, with an eye on offering on-demand autonomous transportation in urban settings.

Dr Andrej Karpathy rejoins OpenAI (9/Feb/2023)

Karpathy is a founding member of the artificial intelligence research group OpenAI, where he worked from 2015 to 2017 as a research scientist. He recently left Tesla AI, and is returning to OpenAI. (‘Some personal news: I am joining OpenAI (again :)). Like many others both in/out of AI, I am very inspired by the impact of their work and I have personally benefited greatly from it. The future potential is especially exciting; it is a great pleasure to jump back in and build'!’)

Andrej Karpathy @karpathy

Some personal news: I am joining OpenAI (again :)). Like many others both in/out of AI, I am very inspired by the impact of their work and I have personally benefited greatly from it. The future potential is especially exciting; it is a great pleasure to jump back in and build!🪄

More ex-Google AI researchers join OpenAI (Feb/2023)

Besides Dr Andrej above, this month, these scientists joined OpenAI:

Hyung Won Chung
Jason Wei
Shane Gu
David Dohan
and more…

Read the unrolled thread.

Runway’s Gen-1 text-to-video model (7/Feb/2023)

Gen-1: The Next Step Forward for Generative AI. Use words and images to generate new videos out of existing ones.

Announce: https://research.runwayml.com/gen1

Paper: https://arxiv.org/abs/2302.03011

Video:

Translation comparison between models (Feb/2023)

Comparison between GLM-130B, DeepL, Google Translate, ChatGPT, and NLLB-200-1.3b-distilled.

Using ChatGPT via Wolfram to analyze BTC blockchain messages (15/Feb/2023)

That’s a lot of buzzwords, but a fun experiment by a former researcher for Wolfram.

Toys to Play With

Ask AI Tim Ferriss anything (Feb/2023)

Tech stack is text-davinci-003 + text-embedding-ada-002 + Next.js + Supabase (Postgresql with a plugin to support vector similarity search).

Try it: https://tim.nem.ai/

Replit Ghostwriter: ChatGPT for coders (15/Feb/2023)

Meet Ghostwriter, your partner in code. Harness the power of Replit’s AI to boost your productivity and creativity. Built into the world's most popular online IDE.

Try it: https://replit.com/site/ghostwriter

Type.ai (17/Feb/2023)