Embedded Files
On November 30, 2022, OpenAI released ChatGPT to the public as a free research preview. Within the first five days, it had over one million users. Within two months, that number had grown to 100 million, making it the fastest-adopted consumer application in history, surpassing popular platforms like TikTok and Instagram. Soon, the conversation about AI and its implications on civilization bursted.
What is ChatGPT?
ChatGPT is a large language model (LLM), a type of AI system trained on enormous quantities of text data to predict and generate human-like language. It belongs to a family of models developed by OpenAI called GPT (Generative Pre-trained Transformer). The "transformer" in that name refers to a neural network architecture introduced in a landmark 2017 paper by Google researchers called "Attention Is All You Need," which became the foundation for virtually every major language model since.
During training, the model processes hundreds of billions of words of text. Any book, website, or scientific paper that you’ve seen has probably been processed by ChatGPT. With these sources, the model is able to learn the relationships between words, concepts, and ideas, resulting in a system that can generate contextually appropriate text across an enormous range of topics. Even more, it can answer questions, write code, translate languages, and more coherently.
The technology behind it
What made GPT-3 different from other language models? While scale does matter, a key innovation was reinforcement learning from human feedback (RLHF). Notably, RLHF is a training process in which human evaluators can rate the quality of the model's outputs. Using those ratings, the model is able to adjust and fine-tune its behavior. This placed ChatGPT ahead of other models — it became significantly more helpful, coherent, and safe in conversation than earlier versions.
With immense capabilities comes an incredible amount of computational resources. As is true with all models, the text data and countless other pieces of information necessary are staggering. For GPT-3, it was trained on approximately 45 terabytes of text data using thousands of specialized processors over the course of weeks. Because of the incredible amount of energy and hardware required to train AI models, the ethical dimension of AI has become more complicated.
ChatGPT’s legacy
ChatGPT's public release triggered a wave of investment, competition, and public debate about AI that is still ongoing. After seeing ChatGPT’s immense success, companies like Google, Microsoft, and Meta rushed to develop and release competing systems. Now, generative AI tools are seeping into all sorts of professions. From writing to drug discovery, people are using AI in all sorts of fields.
For students and researchers, ChatGPT has raised problematic questions about academic integrity and what it means to learn something in an age when producing text is nearly free. While these are definitely not simple questions, they are incredibly important for thinking critically about the implications of AI systems.
Page updated
Google Sites
Report abuse