The Memo - 27/Jan/2023
ChatGPT achievements, DeepL Write, RWKV-4, Lensa AI making >$2M/day, and much more!
FOR IMMEDIATE RELEASE: 27/Jan/2023
Welcome back to The Memo.
Given that a percentage of our readers are from government and large enterprise, I’m trying out a new section in this edition; called ‘Policy’ it brings together all the big guidelines proposed by governments, like NIST’s release yesterday, and new AI-generated Bills across the US.
In the Toys to play with section, we look at a new chatbot using an RNN model, a popular new Transformer-based writing assistant, a new book-writing script to GPT-3.5, web-scraping using GPT-3, and more…
The BIG Stuff
CEOs embracing GPT for productivity (19/Jan/2023)
This section is reserved for BIG stuff. There’s technology big stuff, and then there’s ‘general visibility, availability, and public embracing of technology’ big stuff.
When the 55-year-old CEO of a $2B education company (Coursera) starts using GPT to write emails and design presentations, you know we’re on a roll! Getting visibility for artificial intelligence—and especially integrated AI—has always been the key to my work, and this is a fantastic milestone.
Jeff Maggioncalda, the CEO of online learning provider Coursera, said that when he first tried ChatGPT, he was “dumbstruck.” Now, it’s part of his daily routine.
He uses the powerful new AI chatbot tool to bang out emails. He uses it to craft speeches “in a friendly, upbeat, authoritative tone with mixed cadence.” He even uses it to help break down big strategic questions — such as how Coursera should approach incorporating artificial intelligence tools like ChatGPT into its platform.
“I use it as a writing assistant and as a thought partner… Anybody who doesn’t use this will shortly be at a severe disadvantage. Like, shortly. Like, very soon… I’m just thinking about my cognitive ability with this tool. Versus before, it’s a lot higher, and my efficiency and productivity is way higher.”
Read more via CNN: https://www.cnn.com/2023/01/19/tech/chatgpt-future-davos/index.html
30% of professionals using ChatGPT at work (20/Jan/2023)
Related, ChatGPT is used at work by 30% of professionals from companies including Amazon, Bank of America, Edelman, Google, IBM, JPMorgan, and Twitter. 4,500 employees were surveyed.
The Interesting Stuff
MusicLM by Google AI (26/Jan/2023)
MusicLM, a model generating high-fidelity music from text descriptions such as "a calming violin melody backed by a distorted guitar riff"… generates music at 24 kHz [standard is 44.1kHz for CD/Spotify, up to 192kHz for archival mastering] that remains consistent over several minutes… can transform whistled and hummed melodies according to the style described in a text caption… we publicly release MusicCaps, a dataset composed of 5.5k music-text pairs, with rich text descriptions provided by human experts.
The results are extraordinary.
Have a listen: https://google-research.github.io/seanet/musiclm/examples/
Read the paper: https://arxiv.org/abs/2301.11325
Academics + AI (18/Jan/2023)
Dr Ethan Mollick at Wharton has provided new AI rules for his students…
AI Policy
I expect you to use AI (ChatGPT and image generation tools, at a minimum), in this class. In fact, some assignments will require it. Learning to use AI is an emerging skill, and I provide tutorials in Canvas about how to use them. I am happy to meet and help with these tools during office hours or after class.
Be aware of the limits of ChatGPT:
If you provide minimum effort prompts, you will get low quality results. You will need to refine your prompts in order to get good outcomes. This will take work.
Don't trust anything it says. If it gives you a number or fact, assume it is wrong unless you either know the answer or can check in with another source. You will be responsible for any errors or omissions provided by the tool. It works best for topics you understand.
AI is a tool, but one that you need to acknowledge using. Please include a paragraph at the end of any assignment that uses AI explaining what you used the AI for and what prompts you used to get the results. Failure to do so is in violation of academic honesty policies.
Be thoughtful about when this tool is useful. Don't use it if it isn't appropriate for the case or circumstance.
Read: https://oneusefulthing.substack.com/p/all-my-classes-suddenly-became-ai
ChatGPT would pass a Wharton MBA unit exam (22/Jan/2023)
“Chat GPT3 would have received a B to B- grade on the exam.”
Read the paper: https://mackinstitute.wharton.upenn.edu/2023/would-chat-gpt3-get-a-wharton-mba-new-white-paper-by-christian-terwiesch/
ChatGPT’s achievements (22/Jan/2023)
I’ve consolidated a list of ChatGPT’s achievements to date, including IQ test results and the Wharton MBA exam above.
View the data in Google Sheets.
Watch my video:
Meta AI Chief Scientist Yann LeCun rages against ChatGPT (24/Jan/2023)
A lot of modern AI pioneers feel justifiably miffed by the extraordinary popularity of ChatGPT (one million new users in five days. My ChatGPT Prompt Book sees thousands of views per day).
Dr Yann LeCun created and then pulled the Galactica 120B model in Nov/2022, which had a great layout and solid citations (see The Memo 24/Nov/2022 or my Galactica 120B walkthrough video).
He and I are aligned in our views of the neutered ChatGPT model. He says:
In terms of underlying techniques, ChatGPT is not particularly innovative. It's nothing revolutionary, although that's the way it's perceived in the public. It's just that it's well put together, it's nicely done. OpenAI is not particularly an advance compared to the other labs, at all.
It's not only just Google and Meta, but there are half a dozen startups that basically have very similar technology to it. I don't want to say it's not rocket science, but it's really shared, there's no secret behind it…
You have to realize, ChatGPT uses Transformer architectures that are pre-trained in this self-supervised manner. Self-supervised-learning is something I've been advocating for a long time, even before OpenAI existed…
Why aren't there similar systems from, say, Google and Meta? The answer is, Google and Meta both have a lot to lose by putting out systems that make stuff up…
DuplexGPT based on Google Duplex (25/Jan/2023)
Do you remember back in 2018 when Google was demonstrating AI phonecalls to make reservations? Called Google Duplex (listen to call samples), the technology was interesting. This week, a user tried the same thing with modern tech. ‘Introducing DuplexGPT - automated phone calls using Whisper and GPT. Book restaurant reservations, barber shop appointments, and doctor consultations—without making the call.’
Click to watch the demo video of an automated call to a restaurant (using OpenAI Whisper and OpenAI GPT for the conversation), and there is also a signup link for early access.
The script runs slowly due to API lag and inference, and this must be annoying for the customer service reps, but it is still an interesting application of this AI technology.
DARPA showcasing AI assistants and augmented reality (25/Jan/2023)
It’s not using Transformer, or even large language models, but it is definitely an interesting approach to integrated AI from the $4B/year US R&D lab.
…assistants will learn about tasks relevant to the user by ingesting knowledge from checklists, illustrated manuals, training videos, and other sources of information. They will then combine this task knowledge with a perceptual model of the environment to support mixed-initiative and task-focused user dialogs. The dialogs will assist a user in completing a task…
Read more: https://www.darpa.mil/program/perceptually-enabled-task-guidance
Watch their video:
Boston Dynamics’ latest Atlas progress (18/Jan/2023)
As always, my first reaction is ‘wow’!
But, with however they’re processing their videos, my next reaction is ‘that looks fake!’ Check out the bag throw and the robot’s turns in particular. It may be frame rate, added CGI, mismatched audio, or something else. Turns out this stuff is real though, there’s a behind-the-scenes video, too.
Original video:
‘Making of’ video: https://youtu.be/XPVC4IyRTG8
ChatGPT listed as an author on multiple papers (17/Jan/2023)
As reported in The Memo 18/Jan/2023, ChatGPT is now listed as an author across multiple academic papers, ‘racking up at least four authorship credits on published papers and preprints.’
Read Nature article: https://www.nature.com/articles/d41586-023-00107-z
NYT on Google’s new ‘green lane’ approach to AI (20/Jan/2023)
Mr. Page and Mr. Brin… reviewed Google’s artificial intelligence product strategy, according to two people with knowledge of the meetings who were not allowed to discuss them. They approved plans and pitched ideas to put more chatbot features into Google’s search engine. And they offered advice to company leaders, who have put A.I. front and center in their plans.
Read via NYT: https://archive.ph/pGjg3
GPT-3 in the enterprise (20/Jan/2023)
I’ve made some updates to the ‘GPT-3 in the enterprise’ chart and text, adding KPMG and Al Jazeera to the list of clients leveraging GPT-3 within their organization.
Read more: https://lifearchitect.ai/enterprise/
Updated viz:
Lensa AI brought in $70M in Nov/2022 (20/Jan/2023)
This is my buddy, Karan Rai. This week, Karan is going to run 7 marathons on 7 continents in 7 days (42.1km/26.2 miles each in Antarctica, South Africa, Perth Australia, Dubai, Spain, Brazil, and Miami USA) for a good cause.
He also paid $4 to use Lensa AI, along with a few million other people. In doing so, he helped Lensa AI’s parent company make $2.3M per day.
Lensa AI is based on the Stable Diffusion text-to-image model, and uses Dreambooth-like fine-tuning to recognize the uploaded face (see my video on Google Dreambooth), and some detailed prompt crafting to achieve the output images.
[The Lensa AI app] rocketed to the top of App Store charts after being downloaded more than 20 million times. For $3.99, users could create 50 photorealistic and ethereal avatars of themselves as if drawn by professional digital artists — usually with a flattering twist… It only costs the company about 50 cents to process one pack of Magic Avatars… Prisma Labs made more than $70 million from the app in November [2022, $2.3M/day] alone, a company official estimated.
SparseGPT (2/Jan/2023)
Large Language Models (LLMs) from the Generative Pretrained Transformer (GPT) family have shown remarkable performance on a wide range of tasks, but are difficult to deploy because of their massive size and computational costs. For instance, the top-performing GPT-175B model has 175 billion parameters, which total at least 320GB (counting multiples of 1024) of storage in half-precision (FP16) format, leading it to require at least five A100 GPUs with 80GB of memory each for inference.
…pruning, which removes network elements, from individual weights (unstructured pruning) to higher-granularity components such as entire rows/columns of the weight matrices (structured pruning)…
SparseGPT works by reducing the pruning problem to an extremely large-scale instance of sparse regression.
Read the paper: https://arxiv.org/abs/2301.00774
Policy
NIST releases AI framework (26/Jan/2023)
Yesterday, NIST (the National Institute of Standards and Technology, part of the US Department of Commerce) released the AI Risk Management Framework, along with companion documents.
Download the NIST AI Framework released 26/Jan/2023 (PDF).
Read the background: https://www.nist.gov/itl/ai-risk-management-framework
You can download older policies across 40 or so countries collected by AI Lab Australia here: https://www.ailab.com.au/research/national-artificial-intelligence-strategies/
More: https://lifearchitect.ai/papers/
US Bills drafted by ChatGPT (25/Jan/2023)
Two Massachusetts state legislators, Senator Barry R. Finegold and Representative Josh S. Cutler, used the artificial intelligence chatbot ChatGPT to help draft two proposals to regulate its [own] usage.
Download Bill SD.1827 Regulate ChatGPT (PDF): Original, backup.
Download Bill HD.676 AI & Mental Health (PDF): Original, backup.
Read Josh’ announcement on Linkedin.
Read more: Boston Globe, WBZ.
AI speech via ChatGPT delivered to Congress (27/Jan/2023)
A month after the first AI speech was delivered to the UK Parliament (see The Memo edition 30/Dec/2022), Representative Jake Auchincloss used ChatGPT to do the same thing in the US.
The prompt included:
You are Jake Auchincloss, a Member of Congress. Write 100 words to deliver on the floor of the House of Representatives. Topic: the importance of the United States–Israel Artificial Intelligence Center Act, which the congressman will re-introduce this term.
And the speech began:
Madam Speaker, I stand here today because I am planning to re-introduce the United States-Israel Artificial Intelligence Center Act, a bipartisan piece of legislation that will cement a mutually beneficial partnership between the United States and Israel on artificial intelligence research.
Read more: Original link, The Verge.
Toys to Play With
Run GLM-130B on your local machine (Jan/2023)
Alex J. Champandard got GLM-130B–a Chinese/English model slightly smaller than GPT-3—running on just 4x consumer-grade NVIDIA GeForce RTX 3090 24GB GPUs. Average retail price for those cards right now (Jan/2023) is still around $1,500, so $6,000 total for a consumer-grade inference machine that still takes ‘about 65s to 70s for a single query’!
This really makes you appreciate OpenAI’s mega setup for training and inference, using the massive Microsoft datacentre in Iowa (see my video Where is GPT-4?).
“130B parameters on 4x 3090s…”
“the full GLM weights, 260GB... Convert the model to int4 with convert(.)py, check it's actually 4x smaller [~65GB].”
“You could fit 30B parameters per 24Gb GPU.”
Read the Twitter thread and steps, and maybe try it out yourself.
DeepL releases Write (18/Jan/2023)
Try it for free in English and German: https://www.deepl.com/write
Read a related article: https://archive.ph/yvU5r
RWKV-4 chatbot (19/Jan/2023)
You’d have to get your hands a little dirty with this one! The RWKV-4 language model came out last Nov/2022, and is a true RNN rather than Transformer model. The team have now given it a chat interface via Discord. Note that the RWKV-4 model is only 8% of the size of ChatGPT (14B vs 175B parameters), and the outputs may not be as good.
Original model: https://github.com/BlinkDL/RWKV-LM
Chat model: https://github.com/BlinkDL/ChatRWKV
Chat model on Discord: https://discord.gg/bDSBUMeFpc
Better web scraping with an AI model (19/Jan/2023)
Web scraping up until now has required custom coding or setup with no-code solutions. Kadoa uses generative AI to create web scrapers and extract the data you need automatically. We can collect any data for you. Websites, PDFs, databases, and more.
Try it: https://www.kadoa.co/
Write a 50,000-word book with GPT-3.5 (Jan/2023)
In 2021-2022, I kept an up-to-date list of books co-authored with GPT-3. The process of writing with the language model was always a little messy, forcing the user to generate TOC and outlines before filling in the chapters with content.
Enter a new script to GPT-3.5, with a nice prompt to help out!
The program will generate a Title and Chapter Titles + Content. You will get a detailed structure of the book. The generated books will then be saved in the books folder and will be named after the title of the book.
View the GitHub repo: https://github.com/mikavehns/BookGPT
Read a sample book generated by the script.
via Reddit.
(Bonus: my favourite book writing platform out there now continues to be Sudowrite, responsible for writing hundreds of books via GPT-3. Read a Jul/2022 interview with Leanne Leeds about how she used Sudowrite to generate her best-sellers.)
Next
January 2023 was a big month! Expect an even bigger month next!
All my very best,
Alan
LifeArchitect.ai
Archives | Unsubscribe new account | Unsubscribe old account (before Aug/2022)