Things started moving really fast since OpenAI has pushed the edges of Large Language Models. It demonstrated that by pushing forward transformer architectures, it is possible to create machine learning models with capabilities that were not possible before.
Changes started slowly at 2018 with the release of GPT-1 with gradual evolution to bigger and more capable models until GPT-3.5 achieved viral worldwide popularity in 2022.
Automation of Workflows
Companies and individuals started finding innovative ways to automate workflows. GPT-4 in 2023 further increased the impact and momentum by demonstrating that it can pass US Bar exam within the top 10% of scores.
OpenAI research published with along the GPT-4 outlined possible economic impacts along the lines of:
workforce automation and displacement
new products
more personalized and efficient services
increased inequality
tech acceleration across all sectors
existing players in all sectors get entrenched
Investments in AI
In response to that, entities have started allocating noteable amounts of money to hedge the risks. Here are notable examples:
Microsoft, following its previous investments, invested $10 billion into OpenAI, started bundling offering ChatGPT capabilities within Azure and Bing Search
Accounting giant PricewaterhouseCoopers invests $1B, "aiming to help clients reimagine their businesses using generative AI."
USA budget for 2024 supports innovation and emerging technologies: $25 billion for CHIPS and Science Act-authorized activities.
These examples of investments are followed by a lot of "smaller" investments of all types: starting from Replit (producer of Github Copilot competitor) and to Elon Musk himself, getting 10k of A100 cards for his new AI company.
New Possibilites
Breadth of industry changes can also be shown by looking at the opposite side of the spectrum. What are the smallest known uses of ChatGPT?
ChatGPT helped to save the dog after vets couldn't figure the diagnosis.
People are using ChatGPT as a personal tutor for learning any subject.
Numerous cases of data extraction, manipulation and content generation, starting from personal knowledge bases up to chat bots: Obsidian plugins, Google Spreadsheet Add-in, AutoGPT and Babi AGI, Chat to PDF like Warren Buffet.
It is also important to know that OpenAI and Microsoft aren't holding the monopoly on this technology. It is quite reproducable, and things are evolving fast. It seems that almost every government, company and institute wants to have their own instance of ChatGPT (putting aside governments like Austria and Uganda). Probably things will evolve even faster than assumed because many single bright students and researchers show interest in a ChatGPT solution. That is the most powerful force. Here is a timeline of a single month to put things into the perspective (from a potential Google leak):
Feb 24 2023 - Meta announced LLaMA - OK-ish large language model that wasn't trained like ChatGPT but still had a lot of potential. It can run only on larger GPUs
within a week - Meta was leaked to a public. It wasn't possible to use it legally, but people could experiment
within days - people learned how to compress LLMs to fit on smaller GPUs and even run them on laptops, phones and ultimately on RaspberryPi 4 (retail price of 100 EUR)!
next day - thanks to Stanford Alpaca and LoRa repo, everybody could fine-tune LLMs on a single consumer-grade GPU
within a week - cross-university project called Vicuna reaches parity with Google Bard. While some aspects (data sources and evaluation) are questionable, results are really impressive. It is a fine-tune of LLaMa. Training costs - $300!
within a week - gpt4all is created. It is not just a model, but an ecosystem of open-source language models and chat bots.
Gpt4all was released on March 25, just a month after LLaMA announcement. Naturally, things didn’t stop there. They are still ongoing. Andrej Karpathy calls this “Cambrian explosion”.
Trends
So we can account and plan for a couple of trends:
Models will get smaller, more accessible and more focused
Most of the models will be bundled as services and commoditized
As models get commoditized, the value of data and expertise will rise.
There is a strong chance that the world will start switching to conversational chat-driven interfaces, following the trend set by OpenAI with its powerful plugins.
Conclusion
What could a modern company do in order to benefit the most from the momentum.
focus on competitive advantages of the business
do things that grow business moats (now and in future): gather data, deepen the understanding, grow networks
avoid doing things that are a relative waste of time: jump into fine-tuning before gathering data and exhausting the possibilities of prompt engineering.
continuously reevaluate position to stay aligned with the trends. Trustbit could be your partner in that.
