The Memo - 25/Feb/2023
Meta AI LLaMA-65B, Bain & OpenAI, Fudan MOSS, and much more!
FOR IMMEDIATE RELEASE: 25/Feb/2023
Welcome back to The Memo.
Here’s a 30min interview with Kitco NEWS out of New York (known for their interviews with Robert Kiyosaki and Ron Paul). Video recorded 25/Jan/2023, released 20/Feb/2023. (I look very unimpressed in that thumbnail, and the audio is a bit glitchy.)
It was difficult to choose a winner for the Who Moved My Cheese? AI Awards! this month. But I’ve settled on Clarkesworld (via WaPo), a sci-fi publisher that was so scared by the ‘deluge’ of GPT-generated material submitted this month, that they banned all written submissions entirely!
In the Toys to play with section, we look at a free philosopher chatbot, a new captioning service using OpenAI Whisper, another Conan episode, and VR worlds created via Stable Diffusion!
The BIG Stuff
Meta AI releases LLaMA-65B, Chinchilla-aligned (25/Feb/2023)
This is really big news. The model is the biggest and best around by tokens trained (1.4T). It is ‘open’ to applicants for non-commercial use. ‘LLaMA-65B is competitive with the best models, Chinchilla 70B and PaLM-540B.’
The training time was quicker than I expected. ‘we estimate that we used 2048 A100-80GB for a period of approximately 5 months to develop our models…. When training a 65B-parameter model, our code processes around 380 tokens/sec/GPU on 2048 A100 GPU with 80GB of RAM. This means that training over our dataset containing 1.4T tokens takes approximately 21 days.’ [This is 21 x 2048 = 117 years equivalent on one GPU.]
Read more and download the paper.
Repo and apply to use (non-commercial): https://github.com/facebookresearch/llama
Watch the video:
OpenAI releases a statement on AGI (25/Feb/2023)
We have a nonprofit that governs us and lets us operate for the good of humanity (and can override any for-profit interests), including letting us do things like cancel our equity obligations to shareholders if needed for safety and sponsor the world’s most comprehensive UBI experiment…
We still believe the benefits of society understanding what is happening are huge and that enabling such understanding is the best way to make sure that what gets built is what society collectively wants…
Read more: https://openai.com/blog/planning-for-agi-and-beyond/ (archive)
Bain (+ Coke) + OpenAI (22/Feb/2023)
Besides the Microsoft partnership, Bain is the first major service alliance with OpenAI. This is big because it completely changes the landscape of management consulting (using AI as a service), and the big companies underneath that banner.
‘Equipped with deep expertise in AI technologies, our Advanced Analytics practice doesn’t only advise but also delivers solutions. We pinpoint the generative AI use cases that will create the most value, rapidly deploy a proof of concept, then implement the capabilities across your operating model, businesses processes, and data assets.’
AI at work: Coca-Cola. ‘With ChatGPT and DALL·E, [Bain is] helping Coca-Cola to augment its world-class brands, marketing, and consumer experiences in industry-leading ways. We’re also using OpenAI technology to improve business operations and capabilities.’
Read the release: https://www.bain.com/vector-digital/partnerships-alliance-ecosystem/openai-alliance/
GPT-3 enterprise users as of Dec/2022: https://lifearchitect.ai/enterprise/
The Interesting Stuff
Microsoft using ChatGPT in robotics (20/Feb/2023)
ChatGPT can create new high-level concepts and even low-level code when needed to solve a problem, even fusing existing APIs…
Our work represents only a small fraction of what is possible within the intersection of large language models operating in the robotics space, and we hope to inspire much of the work to come…
Prompt: Take a selfie using a reflective surface.
Response: Sure, you can use the "mirror" object to take a selfie with the drone. Here's the code to fly to the "mirror" and take a selfie:
…
We were impressed by ChatGPT’s ability to make localized code improvements using only language feedback.
Read the release: https://www.microsoft.com/en-us/research/group/autonomous-systems-group-robotics/articles/chatgpt-for-robotics/
Watch the video:
Alan’s AGI countdown upgraded from 39% to 41% (22/Feb/2023)
With Microsoft’s findings above—especially the self improvement plus using a mirror to visualize embodiment (imagine this with DeepMind’s Flamingo vision model)—I have upgraded my conservative countdown to AGI from 39% to 41%.
Read more: https://lifearchitect.ai/agi/
Fudan University’s MOSS chatbot (20/Feb/2023)
They had a rocky start, but the new MOSS chatbot from Fudan University is China’s latest answer to ChatGPT. It is English-first and Chinchilla-aligned (estimate 430B tokens to 20B parameters). It’s currently closed due to overwhelming demand (‘tens of millions of hits’ via Weibo).
Read more: https://lifearchitect.ai/moss/
ChatGPT on cover of TIME (27/Feb/2023)
P.T. Barnum allegedly said ‘There’s no such thing as bad publicity,’ while Oscar Wilde was quoted as saying ‘There’s only one thing in the world worse than being talked about, and that is not being talked about.’
Not all visibility is good visibility, and when it comes to the recent coverage of post-2020 AI by most media outlets, I have been more disappointed than impressed. Still, we don’t get much bigger than TIME. Here’s the real cover for 27/Feb/2023.
Read the article: https://time.com/6255952/ai-impact-chatgpt-microsoft-google/
Amazon + Hugging Face (22/Feb/2023)
Amazon Web Services will make Hugging Face’s products, which include a language generation tool that competes with the technology behind OpenAI’s ChatGPT, available to cloud customers who want to use these tools as the building blocks of their own applications. Hugging Face will build the next version of that language model, called BLOOM, on AWS…
Read more via Bloomberg: https://archive.is/8Rlgy
Google shuts down Everyday Robots (23/Feb/2023)
A separate entity called Everyday Robots was used by Google Research last year for the amazing PaLM-SayCan (and GPT-3) experiments (see my mid-2022 AI report).
‘Everyday Robots will no longer be a separate project within Alphabet. Some of the technology and part of the team will be consolidated into existing robotics efforts within Google Research.’
…Everyday Robots and [Google self-driving car] Waymo lost about $6.1 billion last year.
‘It’s unfortunate to see it shut down. We are starting to see that robots can do meaningful work in a general way. I don't think it’s a sign of a lack of progress. With the right focus, in five years you could have a meaningful product in the market.’
via Wired: https://archive.is/MUBvw
Spotify AI with Sonantic.io (23/Feb/2023)
Spotify has added AI into their DJ/playlist generator, with ‘a dynamic AI voice platform from our Sonantic acquisition that brings to life stunningly realistic voices from text.’
While sonantic.io shut down last year, I’ve still got my sonantic.io licence for another few weeks, and the voices are still some of the best around (pre-ElevenLabs, anyway!). Listen to Una AI powered by sonantic in the Leta series.
Read the release including steps to test the DJ on Spotify (Head to your Music Feed on Home in the Spotify mobile app on your iOS or Android device > Tap Play on the DJ card).
AI for self-driving cars (21/Feb/2023)
Traffic lights could be made obsolete within 20 years, an engineer has claimed, after a landmark trial of driverless vehicles in the UK.
Read the article via Sky: https://news.sky.com/story/traffic-lights-could-be-obsolete-in-20-years-after-uk-driverless-cars-trial-12812875
Toys to Play With
Text-to-skybox: Blockade Labs Skybox (Feb/2023)
Creating VR worlds via AI (Stable Diffusion) using basic text prompting. This has arrived even earlier than I expected!
Blockade says ‘Turn your words into stunning 360° skybox images with Skybox Lab!’
Try your own text prompt here (free, no login): https://skybox.blockadelabs.com/
Watch two video examples by Scottie Fox:
