What is GPT-3? Is It A True Game Changer For Technology?

Imagine that you’re a lawyer and a few years from now, you have to go through a case log of documents that you want to summarize – from 5,000 cases, to be exact. By the way, each of those cases are 100 pages.

However, what if you could feed these cases into a machine that would summarize all 5,000 cases into the top 5,000 paragraphs? It would be hefty, but a lot more manageable, correct?

Alternatively, what if you’re a programmer who has been writing code in Python and one day, you don’t need the program anymore? That’s right. All you need to do now is deliver sample data into a machine, get the output type you want and analyze it. In essence, you become a trainer for the machine as opposed to a programmer.

It’s a day that could be right around the corner and create dramatic changes in the nature of work across a variety of industries – perhaps even more so than the kind we’re experiencing right now. What type of machine holds such promise and potential? As you’re about to find out, it’s a beast.

The Rise of GPT-3

For a long time, Elon Musk and other notables have been very concerned that certain projects will consume all of the available AI, thus their belief that the latest thinking on machine learning should be distributed to all of the general public. This way, nobody would have an edge against, well, everybody else.

It’s also likely why Musk is paying close attention to the latest moves from Microsoft to commit $1 billion to a company. That company, OpenAI, is led by Sam Altman (former CEO of Y Combinator). OpenAI is an artificial intelligence lab based in San Francisco.

Microsoft has also built a gigantic machine for OpenAI that will be one of the world’s largest supercomputers. At the same time, OpenAI has produced its first commercial product: GPT-3.

GPT-3 (Generative Pre-trained Transformer) represents a new kind of machine learning system, a predictive model that utilizes deep learning to create the most human-like text up to this point. For the first time, machines are creating text that is so high quality, it is virtually indistinguishable from that of a human being.

At its capacity, GPT-3 can manage 175 billion machine learning parameters. To put this in context, the previous language model, from Microsoft, had a capacity of 17 billion parameters – about 10% of what GPT-3 can handle. When the original GPT was launched eight years ago, it had approximately 1.7 billion parameters, only 1% of today’s current model’s capacity.

Photo by Tanner Boriack for UnSplash

What Makes GPT-3 A “Game Changer”?

As I listened to a podcast from the venture firm Andreessen Horowitz, I was struck by a guest’s comments proclaiming GPT-3 to be a bit of a “game-changer.” That term is thrown around a lot, but this time, I think it’s very legitimate once you take a closer look under the hood of what GPT-3 is all about and what it can do. Whereas GPT-2 primarily dealt with recognizing images, GPT-3 is focused upon identifying text.

For example, let’s say you’re building a machine learning system today. You might pick a particular type of text to process and rely upon basic machine learning to “understand” that text. You plan to “teach” this machine to distinguish the right text and what is not. You then go to the system to retrieve this data and get reliable results. Those results may cost you millions of dollars and take a very long to arrive at an accurate outcome.

Contrast this with GPT-3, which has been trained on such an incredible amount of data of every variety that it has “learned” all sorts of things that people didn’t know that it was learning. And there’s more: Rather than giving a GPT-3 system millions of pieces of text to comprehend from, it only requires select samples to understand the text and its purpose. Thus, you can ask questions with minimal input and it returns excellent, accurate answers in a brief period of time.

Let’s say you’re a startup and are looking for answers from a GPT-3 model as a foundation. The solutions provided from this machine may only require you to spend $50,000 instead of trying to obtain $5 million.

If GPT-3 continues to perform well – and there is little reason to suspect that won’t – you can be confident that Google, Amazon, IBM and others will develop their own versions. All you need to be is an expert in figuring out how to use just enough data to give an example of what you’re trying to do. Then it will start providing you the answers you need.

And if it gives you the wrong answer? You correct it and the machine learns from it.

Here’s the implication: Suddenly, you don’t need to be a programmer to deal with this. You need to be more of a teacher adept at explaining what you need and directing the machine on these goals clearly – even in all languages.

So if it appears that the model of GPT-3 works, the emphasis will be instead about feeding the machine the best examples of what you want it to learn. You want it to recognize any kind of text? Feed the text into GPT-3. Then start feeding in newer and newer situations, asking for feedback continuously from the machine. If it passes with flying colors, the machine will not need many examples to accommodate exactly what you’re looking for.

The access to GPT-3 right now is a very tight circle, while OpenAI is fine-tuning it. We can still probably fool it now and then, so those “kinks” need to be eliminated.

However, suppose we can successfully build a system with 175 billion parameters. In that case, it’s only a matter of time before a machine is made with trillions of parameters – and by the way, the human brain has about 7 trillion synapses. Then, we will have reached an Earth-shattering milestone: A machine that “thinks,” “behaves” and “sounds” the closest to a human that makes decisions.


Talman Advantage #5: A Real Partner With A Plan

When a recruiter talks to you on the phone for 20 minutes just once, there’s only so much they know about you beyond the resume. On the other hand, Roy Talman & Associates will work with you to gain a robust understanding of your skill set, goals, work style preferences and more. Then, rather than “blasting” your resume out to the hiring universe with random results, we’ll make a plan with you on what order we will present you to various firms that we feel are a best fit.

Your career deserves more than a quick chat. Partner with a recruiter who can help you feel more in control of the process – as you should be. Talk to Talman first.


Photo by Mahdis Mousavi for UnSplash

Is there any sign that more people will get access to GPT-3 technologies soon?

GPT-3 has been announced as a commercial product, so as some rigorous testing is completed, we should see access open up gradually to the general public as a platform. And that platform will be able to handle a variety of inputs flowing into it continually with answers flowing out continuously, presumably charging the user per activity.

That’s the functionality of the machine that will change in time. But what about the functionality of the technical profession that may change as we know it?

Remember – we have seen machine learning via the GPT jump from its original level to its third iteration in just eight years. Where will we be in another eight years? We can expect the most advanced machine learning level to become a reality today and for another advancement level to be right on its heels. The role of tech professionals who can direct machines from one destination of computation to another will be essential.

Yes, you will still need to recognize when a machine errs and redirect it to the proper path, so it learns not to err in that manner again. If it calls for fewer samples to be fed into it rather than millions of them, you will still need to feed the machine with clear examples of what it needs to learn so that the “learning curve” is short and accurate. And when the answers flow out of a machine, you will still need to decipher what those answers mean in relation to the landscape of overarching goals. After all, a machine may perform individual tasks flawlessly, but we don’t always know that such a machine can see the “big picture” of what we’re trying to accomplish through a series of tasks.

This is how our work can shift and change. Even though we don’t know what every promising machine learning model can and cannot do, we can still plan for the inevitable progress and evolution to come. Sitting still just isn’t an option.

That’s why forward-thinking candidates and companies Talk To Talman First. We know that today’s skills may very well need to change tomorrow, so it’s good to meet with the recruiter who has been entirely in tune with these developments for over 30 years. What new tests should a candidate be challenged with before the interview? What companies are running toward machine learning systems instead of away from them? It’s good to rely on one resource that’s known all over in a landscape of unknowns. From Chicago to New York and many points in between.