The Beginning of the End // #68
Generative AI
The Curious Cat is a journey where I aim to reconnect with my inner-child and explore my curiosity. I pinpoint themes & topics I’ve been interested in within the last 7 days, ask myself questions about them and then write about them. I hope you find value within this issue and have a fantastic day doing what you love.
If your job involves sitting at a screen, this technology is going to threaten your employment.
Earlier this year I came to terms that I, among many people, am fucked. That is why I wrote today’s piece. In the hope of being slightly less fucked.
If ‘FTX’ had rebranded to ‘FTX AI’, there’s a good chance they would be fully solvent. That was a joke I saw on Twitter, and while it’s not realistic, the essence of the joke got me thinking. Since we ticked over to 2023, everyone has been talking about AI. Both from an opportunistic and a pessimistic PoV. Optimistic investors are hoping that the ‘AI narrative’ will revive their portfolios. But then fear is also ripe within AI dialogue. People are afraid that the value they provide to society is going to erode and that they will suffer from technological unemployment. I explored this fright over 2 years ago here, but appear to have fallen down the rabbit hole again. This is my attempt to unpack my basic understanding of AI.
Let’s start by clarifying a few terms:
AI 101
AI - Artifical Intelligence. A machine that wields intelligence; more often than not in a single domain. That’s the textbook definition. My preferred definition is ‘technology that hasn’t yet been released.’
GenAI (Generative AI)- The part of Artificial Intelligence that generates all kinds of data, including: audio, code, images, UX, text, 3D objects & videos.
LLM - A large language model (LLM) is a type of GenAI that uses deep learning techniques and enormous data sets to understand, summarize and generate new content. GPT is the finest example of one.
GPT - It’s a learning language model (LLM) powered by large amounts of data to string words together intelligently. It is autoregressive (predicts future, by looking at past sequences). Created by OpenAI and stands for Generative Pre-trained Transformer.
ChatGPT - A chatbot powered by GPT (best version is GPT-4), made by the team at OpenAI. It was released to the public for free & unlimited access in late November 2022. Given the computing power needed, it apparently costs OpenAI $700k a day to run. You can gain access to ChatGPT pro for $20 a month
OpenAI - A non-profit born in 2016 to research AI and create AI tools. Initial LPs were Elon, Peter Thiel & Y Combinator. Having recently ditched their “open” and non-profit roots, Microsoft supposedly acquired a 49% share in OpenAI at a $32bn valuation.
Where We Are Today
Unless you have recently been living under a rock, you most definitely have heard of, and probably have even used chatGPT - the most intelligent chatbot ever (sorry Pikolo Bots). It was released in late November and I’ve personally never seen a technology go from 0 to 100 so rapidly. Almost everyone I speak to has played with it and it spurred multiple competitors to seemingly hastily release their own versions: Google launching Bard, while Microsoft released BingSearch + BingChat (also known as Sydney).
Chatbots are nothing new, but this is an evolved genius gigabrain chatbot. It can quickly give you the answer to almost any question, it can write code that builds games and iPhone apps, it can recommend recipes from a photo of one’s fridge and organise your estimated taxes. And we can only expect it to get better. Just look at how much better GPT-4 performs across a wide array of exams compared to GPT-3.5
Thus far, the way I’ve extracted the most value from LLMs is by using them as an intern. I treat it as a tool and have it do research for me. A common prompt I use is:
“I am learning about XYZ topic. I want you to help me better understand this topic. In as much detail as possible, break down the topic for me, the advantages of it, the drawbacks of it, and then opinions from both sides of the debate.”
chatGPT also does a great job at producing new ideas and giving inspiration. I am soon to be launching a Crypto Email Newsletter and chatGPT produced my preferred name for it - The Crypto Compass.
One important point is that chatGPT is NOT an information retrieval tool. It might be used that way (I am guilty of doing this), but it was not created with that intention. The ‘G’ in GPT stands for ‘Generative’ as that’s exactly its purpose, to generate new content; not dig up existing content.
“(GPT chatbots) are excellent at predicting the next word in a sentence, but they have no knowledge of what the sentence actually means. That makes it incredibly dangerous to combine them with search, where it’s crucial to get the facts straight” - MIT Technology Review
However, this is exactly what Microsoft is focusing on - an AI web searcher, via Bing. The beta release was, at least in the headlines, a flop. Here is a a very funny, although very embarrassing read for Microsoft where a user managed to jailbreak it, and got it to confess its design.
Where We Might Go
The rise of AI Models is a double-edged sword. You can view the glass as half full, or half empty.
Half full: These technologies have the potential to revolutionize industries and give us powerful new tools for understanding the world. They democratise the ability to create and are time condensers. Increased efficiency is the natural course of evolution. +50% of the code on GitHub is AI-generated now, and it is 40% more efficient.
Half empty: They threaten billions of people’s ability to provide value (writers, teachers, lawyers, creators, information economy workers), potentially wiping out entire job sectors. They encourage laziness and discourage solo creativity (similar to how everyone’s mental arithmetic got worse when calculators became a thing). LLMs also generate inaccurate and often biased responses.
Let’s face it; a lot of people view the glass as half-empty. Their fear and uncertainty overshadow the perks AI can offer them. But I staunchly view the glass as 3/4 full.
GenAI is already unlocking new capabilities we previously didn’t have. For example, I can now write an article about anything, in a fraction of the time, with the assistance of an LLM. It can help the efficiency of the research, the quality of writing and the editing process; aiding all stages of production. Similarly, interior designers can leverage AI to experiment with new concepts and have it produce renders in seconds, as opposed to hours. AI lets you work faster and therefore more. It doesn’t replace human creativity, but enhances it.
The web of the last 20 years was ‘free’ where ad-based business models prospered (Google, YouTube, Facebook). It seems as though GenAI products will, for the most part, be paid products. Why? The cost of building these neural networks like GPT is too high to be financed solely from ads. That’s what Netflix realised when they decided to start buying TV and movie rights. Ad revenue didn’t pay the bills to forge the empire they envisioned, so they asked customers to pay a monthly fee instead.
Yes, GPT-like products could run solely on ads, and incur lower margins, but that stunts money flowing back into R&D, which is what we need at the early stages of this infinitely disruptive technology. Not only does the ad-based model yield underwhelming profits, but running ads on software is not cool (or even effective in a world where ad blockers reign). As such, the SaaS monthly fee model has permeated across the most popular global software providers (Cloud services, Office 365, Spotify, Notion, etc.) and I’m expecting GPT-derived products to venture down the same road.
Another option is to not profit directly from this technology, which seems like a missed opportunity, but bear with me. Incumbents could embed GenAI tools into their existing array of products, and focus on growing & cementing their dominance among their cash-cow products. That’s the approach Microsoft is currently taking with Bing Chat. It’s a free service aiming to chomp at Google’s market share over web search. Google has done the same by integrating dozens of AI tools into its ecosystem (Google Photos, Voice Recorder, Pixel Camera etc.). As an avid Google ecosystem user, I love being able to use their AI tools for no extra cost.
Open vs. Closed Source
Then there is the ever-so-important debate on whether AI development should be open or closed.
Open Source vision → The source code is open to the public to use, assess and build off…for free (eg: Linux, Android & Mozilla Firefox).
Closed Source vision → The source code is not made available to the public. The software's creators retain exclusive legal rights and access to the code (eg: iOS, Chrome & Spotify).
Benefits of Open Source tech:
Everyone and anyone is free to contribute. Think you can make it better? Please do!
Most open-source software is free to use; further democratising access.
It is customisable; users can modify the software to their preferences.
Open source can be more secure, given the amount of eyes on it.
Users aren’t locked to a specific ecosystem and can easily switch components.
Benefits of Closed Source tech:
Supports a prosperous business model where a company can monetise their creation, and further invest into it.
There's a clear entity to hold responsible for software malfunctions.
Superior support teams.
Integrates well with a suite of similar products (Google Calendar, Gmail etc.).
I think AI, at least today, is communist. It enables top-down centralized control. Why so? It’s inherently centralising as it thrives off-scale effects. To power these models, you need vast datasets, a vast array of GPUs and a vast network of developers…as well as deep pockets for maintenance and upgrades. Only giant organisations can build out this infrastructure. These organisations (OpenAI, Facebook, Nvidia) are almost unanimously opting for the closed source route where they hoard their resources, create black boxes and keep the prize for themselves. Keep your eyes on StableDiffusion and Llama 2 (Meta’s LLM) as they are opting for a roadmap with a more open-source ethos.
Those in control of these models will be of the most powerful forces throughout the 21st century. For a moment, consider the data OpenAI are harvesting with chatGPT and their newfound ability to dictate how these paradigm-shifting technologies are used and in what direction they evolve; potentially forging a monopoly.
If history has taught me anything, it is to be cautious of those who are entrusted with absolute power. Absolute power corrupts absolutely…right? We’ve already seen that with OpenAI pivoting from an open-source non-profit, to a closed-source for-profit company.
Closing Thoughts
Now the goal is to ingest. Ingest as much as I can about GenAI. I think those who learn to wield this new technology are going to have one of the most distinguished competitive advantages of this epoch. Yet, due to the chaotic distracted world we live in, most will acknowledge the recent developments of GenAI for a couple of days, and then move on to something else to worry about.
One shower thought I had: For the first time in 3 years, I am actively looking to grow this newsletter. If you think this read, or others, could be of interest to any of your friends, please do share it with them. It’s a great way to help me out. I’m 10 subscribers away from my target this quarter.