The Memo - 7/Mar/2023
Romania's AI government advisor, Palmyra 20B, newer DALL-E 2, ChatGPT + Sheets update, and much more!
FOR IMMEDIATE RELEASE: 7/Mar/2023
Welcome back to The Memo.
A quick ‘thank you’ for being a full subscriber here. You are the reason I get to be an independent researcher, unsullied by any overarching company or AI lab. You’ll find a few ‘secrets’ in this edition, too!
It took just a couple of days into the month for our winner of The Who Moved My Cheese? AI Awards! to be decided. And the winner for March 2023 is mega-author Neal Stephenson, who says the output of ChatGPT is ‘simply not interesting’.
…a person would be reading only the output of an algorithm, “and if that’s interesting to you, then fine.”
“Personally, I know a lot of writers who are putting a lot of effort into creating their own original works, and I’d rather support them and hear what they have to say than just look at the output of an algorithm.”
(Alan’s note: I love that these guys are going on permanent written record with these preposterous soundbites! Give it a year or two so we can look back. And check out my list of books co-authored with GPT-3 starting from nearly three years ago.)
The Policy section is back for this edition, with an explosive and frantic 64-page report from the UK government.
In the Toys to play with section, we look at the new Palmyra model, the latest Colin episode, the update to my Google Sheets + ChatGPT functionality, Poe for web (free ChatGPT + Anthropic Claude), and much more…
The BIG Stuff
Romania’s new AI LLM government advisor (1/Mar/2023)
I still get excited about ‘integrated AI,’ especially as more governments step up to the plate to bring it into society. It’s really interesting to see artificial intelligence being embraced in unexpected places…
Ion (pronounced ‘ee-on’) is Romanian for ‘John’.
Romanian Prime Minister Nicolae Ciucă surprised his Cabinet on Wednesday by introducing them to a new member — run completely on artificial intelligence.
Ciucă introduced the new "honorary adviser" called Ion to the rest of his ministers in a demonstration, with a face and words appearing on a digital screen, responding to the prime minister's prompts along with a computerized voice.
Ion was developed by Romanian researchers and will use artificial intelligence to "quickly and automatically capture the opinions and desires" submitted by Romanian citizens, Ciucă said.
"We are talking about the first government adviser to use artificial intelligence," both nationally and internationally, he said. (— via Politico)
Ion takes a physical form as a long, mirror-like structure with a moving graphic at the top suggesting it is listening at all times.
“I have the conviction that the use of AI should not be an option but an obligation to make better-informed decisions,” Ciuca said. (—via Vice)
At first glance, this seems like a gimmick. But this is a real large language model (LLM), with a very real application. The Romanian source below confirms that Ion ‘uses Large Language Models, which manage to take these opinions of Romanians and synthesize them for the Government.’
‘The project is thought out in phases… Ion is in the learning phase, so it is very important for Romanians to convey their thoughts to him, what problems they have, later, based on this data, Ion will process them, synthesize them and be able to produce reports that will inform decisions and policies of the Government’
Read the Romanian source in English via Google Translate.
As a digression on these smaller countries leading innovation in the AI race, I wonder if nominative determinism (or ‘name-driven outcome’) is at play here. Check out bleeding-edge AI from countries like Romania and Malta, as they continue doing big things in the AI world. Romania has the letters ‘A’ and ‘I’ in order (separated by just one letter), while the lowercase ‘L’ in ‘Malta’ makes it look like ‘AI’. Shoutout to my Perth colleague and General Manager Dom Bull, and we’ve all heard of the runner Usain Bolt. You can read more about this effect (‘names such as Dennis or Denise would be overrepresented among dentists, and people with names such as Lawrence or Laura would be overrepresented among lawyers’) in this 2002 SUNY paper. (Welcome to The Memo; I wonder if you’d find that kind of deviation in any other AI report!)
I continue to provide AI consulting via expert calls as well as longer-term advisory to major governments around the world, most under NDA with ‘no identify’ clauses.
For our paid subscribers, here’s a video asset I developed for a very progressive government (not under NDA, though still confidential) with Leta AI able to answer local questions, translate between languages, and even ‘see’.
The Interesting Stuff
DALL-E 2 Experimental (6/Mar/2023)
Nearly a full year after the initial DALL-E 2 release, a few testers are playing with the latest DALL-E Experimental build as of March 2023 (invitations are rumored to be ‘at random’). You can view some of the sample generations here.
EleutherAI becomes a non-profit (3/Mar/2023)
EleutherAI is a team of voluntary researchers, and was the first organization to replicate GPT-2 (as GPT-Neo and GPT-J), the GPT-3 dataset (as The Pile), and offer a big open-source model (as GPT-NeoX-20B). ‘We are forming a non-profit research institute, and we are excited to be able to say that over twenty of our regular contributors are now working full-time doing research.’ Donors include (via Stella):
Stability AI.
Hugging Face.
CoreWeave.
Nat Friedman (GitHub, GNOME, Arc Institute, Midjourney, Xamarin).
Lambda Labs.
Canva.
Read the announcement: https://blog.eleuther.ai/year-two-preface/
ChatGPT and productivity (3/Mar/2023)
My latest video looks at how ChatGPT’s very low costs could recreate the written output of 5,000 professional workers for about $10/day.
Watch the video:
Stability AI Seeks to Raise Funds at $4 Billion Valuation (4/Mar/2023)
Read (not very much) more via Bloomberg.
Bill Gates loves using ChatGPT (4/Mar/2023)
Bill says:
I was using the version that comes after that [probably the newer version of ChatGPT, based on GPT-4] last night and I was telling it to write things in Hindi. Of course, I had no idea what it was writing, but I play around with it whenever I have friends over.
Read (not very much) more via Indian Express.
Google’s Lion may replace Adam optimizer (Feb/2023)
a simple and effective optimization algorithm, Lion (EvoLved Sign Momentum). It is more memory-efficient than Adam as it only keeps track of the momentum… requires a smaller learning rate than Adam due to the larger norm of the update produced by the sign function… The implementation of Lion is publicly available.
Read the paper: https://arxiv.org/abs/2302.06675
Policy
A New National Purpose: Innovation Can Power the Future of Britain (22/Feb/2023)
This entire report is worth reading, with just a few of my favourite highlights below…
AI’s Potential Impact
Progress is happening at breakneck speed, with the practical impact of AI almost entirely contained to the past ten years, especially since 2019… we now have “infinite AI interns for everybody”, while research from MIT suggests that widespread adoption of AI tools in R&D could triple productivity-growth rates…
Special focus should be placed on the transformative potential of next-generation, general-purpose AI systems such as ChatGPT, which could underpin considerable public-service delivery.
Given these AI systems will soon be foundational to all aspects of our society and economy, it would be a risk to our national security and economic competitiveness to become entirely dependent on external providers. Since the technology is sufficiently mature, the government should take on a greater role in its direct development to ensure the UK has sovereign capabilities in this field. Leading actors in the private sector are spending billions of dollars developing such systems so there may only be a few months for policy that will enable domestic firms and our public sector to catch up.
Here are the relevant technical recommendations:
Technical recommendation: Develop and procure sovereign general-purpose AI systems that can disseminate innovation throughout our public services. This should at first be done through procurement in partnership with a consortium of domestic AI firms that may be best placed with technical expertise. Simultaneously, an elite public-sector research effort modelled on visionary labs such as OpenAI and DeepMind should be initiated to ensure public-sector expertise and ownership of this technology.
Technical recommendation: Ensure we have sufficient supercomputing to support specialist AI researchers in the public sector and universities. Some of this should be specifically reserved for developing sovereign large language models to be deployed in public-service delivery. Government departments could submit bids to access the resource, judged by an expert board convened by Number 10 and the Cabinet Office.
Read the report: https://institute.global/policy/new-national-purpose-innovation-can-power-future-britain
Toys to Play With
ChatGPT + Google Sheets (Mar/2023)
I’ve updated my Google Sheets script to use the new ChatGPT completion endpoint.
In theory, because this is 10x cheaper than using the older model (0.2c vs 2c per 1,000 tokens or 750 words), you should be able to generate around 350,000 words for $1. That’s a lot of Excel/Sheets productivity!
Ignore the code; this is really easy to integrate, and I’m even using this for hands-on keynotes with CEOs this year.
Try it out: https://lifearchitect.ai/sheets/
Watch the 2-minute video:
Writer: New Palmyra 20B model for enterprise writing (14/Feb/2023)
Just a quick reminder that I don’t do promotions, paid or otherwise. I found this model myself, and have never spoken to the Writer team or anyone related. This particular model is interesting to me for its fine-tuning on business documentation.
Palmyra is a large language model fine-tuned for writing tasks. It has 20B parameters trained on 300B tokens, making this a Chinchilla-aligned model. Access to the 5B model within the Writer platform is $162/user/year in the teams plan, and the 20B model is available within the bigger enterprise plan.
Leading companies like Spotify, Northwestern Mutual, Intuit, Deloitte, and Adore Me are already reaping the benefits of weaving generative AI into their systems and processes. They use Writer as their go-to AI writing platform, use Writer’s API to integrate the output of their AI apps from Writer, and build their own applications and workflows…
With Palmyra [128M, 5B, and 20B], Writer is giving companies the power to train up their own LLMs. You can further fine tune the model with your own data, brand, and style guidelines. Plus, you don’t have to worry about outdated datasets, “hallucinations” or “plausible BS.” Writer generates outputs in the context of your business and how you’ve trained it. With claim detection, Writer will highlight anything it can’t verify and bring it to your attention.
Take a look: https://writer.com/blog/palmyra/
On the timeline: https://lifearchitect.ai/timeline/
On the models table: https://lifearchitect.ai/models-table/
Colin Diffusion Episode 6 (3/Mar/2023)
Conan was renamed ‘Colin O’Doyle’ but he’s still just as funny. Check him out with the late Steve Jobs and the even later George Washington.
Tech stack: Script by ChatGPT, visuals by Stable Diffusion, music by Polymuse AI, voice by ElevenLabs.
Watch the video:
Poe launched for web (5/Mar/2023)
‘We are excited to launch Poe for desktop today at http://poe.com. With this launch, between our existing iOS app this new web interface, we believe Poe is now the fastest and easiest way to talk to AI across iOS and all desktop platforms.‘ (—via Twitter)
Try it out for web (and iOS): https://poe.com/
Next
You may have seen the recent leak of Meta AI’s LLaMA-65B Chinchilla-aligned model, but it is still under a non-commercial licence, meaning that even though we can download it, we can’t use it for production work.
I’m waiting for Stability AI to release their 70B Chinchilla-aligned model this month or next month. That will be a game changer for openness, and with more power than GPT-3 packed into less than half the size.
All my very best,
Alan
LifeArchitect.ai
Archives | Unsubscribe new account | Unsubscribe old account (before Aug/2022)