The Memo - 20/Feb/2023 (GPT-4 & GPT-5)
GPT-4 already in use, GPT-5 now training (Morgan Stanley), and much more!
FOR IMMEDIATE RELEASE: 20/Feb/2023
Welcome back to The Memo.
This is supposed to be released in monthly editions. Something happened in the last 80 days to change that just a little…
Thank you for your paid support, and I hope you’re having fun with these temporarily-frequent editions!
In the Toys to play with section, we look at a new AI-generated series featuring one of my favourite comedians (using ChatGPT + Stable Diffusion + ElevenLabs), and new podcast episodes generated by AI.
The BIG Stuff
GPT-4 is already in use (Feb/2023)
Last year, in the 30/Dec/2022 edition of The Memo, we talked about four $5-10M investments made by OpenAI:
OpenAI promised to provide these companies with ‘early access to future OpenAI systems’. It is likely that these organizations have been given access to GPT-4 (rather than just a point release of GPT-3 or GPT-3.5). This was recently supported by a number of sources. Let’s explore further…
This month, the second organization listed above, Harvey AI, was noted for:
its wow-factor launch yesterday (16 February [2023]) of ‘co-pilot’ Harvey, which helps lawyers to conduct research and due diligence using natural language instructions, leveraging the most up to date OpenAI large language model.
Founded in 2022 by… former DeepMind, Google Brain, and Meta AI research scientist Gabriel Pereyra, Harvey is a verticalised version of what I understand to be GPT-4, which has been trained on the entire corpus of the internet. By verticalised, I mean that Harvey has further trained the model with legal sector-specific data. Harvey, which in November last year received $5m in investment from OpenAI, has been working with a number of law firms – including A&O – in beta… - via Legal Technology (16/Feb/2023)
A&O’s official press release (PDF) says:
A&O has been trialling Harvey in beta since November 2022 under the leadership of a team of lawyers and developers tasked with disrupting the legal industry, called the Markets Innovation Group (MIG). At the end of the trial, around 3500 of A&O’s lawyers had asked Harvey around 40,000 queries for their day-to-day client work.
‘I have been at the forefront of legal tech for 15 years but I have never seen anything like Harvey. It is a game-changer that can unleash the power of generative AI to transform the legal industry. Harvey can work in multiple languages and across diverse practice areas, delivering unprecedented efficiency and intelligence. In our trial, we saw some amazing results.’
There are further rumors that the new Microsoft Bing Chat (Sydney) is also based on GPT-4. In a private interview, Microsoft’s CEO recently answered some interesting questions:
Nadella: So when I first saw this new model… because the model that you saw today [8/Feb/2023, Bing Chat (Sydney)] is the next-generation model–
Interviewer: Is it GPT-4?
Nadella: Let Sam [Altman, OpenAI CEO], at the right time, talk about his numbers.
Interviewer: Okay.
Nadella: So, it is the next-generation model, and it’s been done. We called it the Prometheus model because, as I said, we’ve done a lot to the model to ground it in search. - via The Verge (8/Feb/2023)
If this interests you, and you really like to read (2,600 words), here is a longer explanation with citations, via a Reddit user. While it is just a hypothesis, he is drawing together a number of technical sources and interviews, and mentioning that (as expected) GPT-4 is just a much bigger GPT-3. For clarity, this means that GPT-4 is not multi-modal, and does not have (m)any extra new features.
…here is a hypothesis no one seems to have brought up: "Bing Sydney is not a RLHF trained GPT-3 model at all! but a GPT-4 model developed in a hurry which has been finetuned on some sample dialogues and possibly some pre-existing dialogue datasets…this would not be the only public GPT-4 use, people say it has lower-latency than ChatGPT which hints at GPT-4, and in general it sounds and acts nothing like ChatGPT - but does sound a lot like a baseline GPT-3 model scaled up.
Read more via a Reddit user (17/Feb/2023).
GPT-5 is now training: Morgan Stanley (Feb/2023)
In a research note to analysts on 10/Feb/2023 (titled ‘Context on the NVIDIA ChatGPT opportunity - and ramifications of large language model enthusiasm’), Morgan Stanley—one of many enterprise users of GPT-3 and possibly an OpenAI partner—went even further, explaining that GPT-5 training is now the current focus.
We think that GPT-5 is currently being trained on 25k GPUs - $225 mm or so of NVIDIA hardware - via Twitter (13/Feb/2023)
Read more updates: https://lifearchitect.ai/gpt-4/
The Interesting Stuff
Singaporean govt implements ChatGPT into workflow (14/Feb/2023)
…up to 90,000 civil servants to be able to use the AI service, adding that it will be rolled out progressively across agencies, starting with the Smart Nation and Digital Government Office (SNDGO)…
…the software aims to ease the load on civil servants when they write and do research. “We want to free officers up for higher-level tasks. This bot can help them get over that tough first draft, or speed up their work by creating sample e-mails or even speeches…
As seen in a demo session by the Pair team on Thursday, the chatbot is able to summarise long chunks of information and draft reports on policy-related topics within seconds.
The project has also struck an agreement with Azure OpenAI to ensure that data handled by the Government is kept confidential and out of sight to Microsoft and OpenAI… Microsoft also offers some customers in highly regulated industries the option to prevent the Azure OpenAI Service from ever seeing the data processed, to protect sensitive data.
Large language models are not sentient yet (17/Feb/2023)
This is a nice and clear summary of the ELIZA effect (wiki), and a response to the more egregious claims from first-time users of AI like ChatGPT.
Having spent a lot of time with these chatbots, I recognize these reactions. But I also think they’re overblown and tilt us dangerously toward a false equivalence of software and sentience. In other words: they fail the AI mirror test. What is important to remember is that chatbots are autocomplete tools.
Read more via The Verge: https://www.theverge.com/23604075/ai-chatbots-bing-chatgpt-intelligent-sentient-mirror-test
Starlink success in early 2023 (17/Feb/2023)
Planet-wide internet is not directly related to AI (although the entire design has been compared to a neural network), but the progress within 3 years is amazing.
Take a look live: https://www.heavens-above.com/StarLink.aspx?lat=0&lng=0&loc=Unspecified&alt=0&tz=UCT
Video:
Toys to Play With
ConanDiffusion Ep. 4 - Paul Rudd and Chris Rock (16/Feb/2023)
This is beautiful and hilarious. 100% generated by AI, from script to video.
Tech stack: Script by ChatGPT, visuals by Stable Diffusion, music by Polymuse AI, voice by ElevenLabs.
Watch the video:
Podcast.ai (Feb/2023)
Alive or dead, it doesn’t matter! AI generates responses to whatever prompt you feed it… Tech stack is GPT + Play.ht for ultra-realistic voices.
Zach Galifianakis talks movies with Quentin Tarantino
Terence McKenna interviews Alan Watts about the Order of the Universe
Lex Fridman interviews Richard Feynman
Joe Rogan interviews Steve Jobs
Listen here: https://podcast.ai/
Next
I’ve updated the cover pages for three of my reports using AI: Midjourney v4c, or Stable Diffusion for the 2022 AI Retrospective. You can see each of these reports in the sidebar (on the right) of any article at LifeArchitect.ai, e.g.:
https://lifearchitect.ai/whats-in-my-ai/
I’ll continue covering the latest updates about the latest models, including their datasets. We’ll forget about ChatGPT in just a few more months!
All my very best,
Alan
LifeArchitect.ai
Archives | Unsubscribe new account | Unsubscribe old account (before Aug/2022)