October 21, 2021
GPT-J, an open-source alternative to GPT-3
By Sofía Sánchez González
The GPT-3 model, which came out last year and impressed the entire world with its capabilities, now has an open-source version: GPT-J. It is a language model created by Eleuther AI, a group of researchers who seek to democratize artificial intelligence.
In this post we explain why a model accessible to all is important and what it’s capable of doing.
Why launch an open-source model?
For those of you who aren’t aware of GPT-3, know that it’s a language model developed by OpenAI that learns from existing text and can provide different ways of finishing a sentence (similar to predictive text). Once trained, it can save you a lot of time, providing linguistic richness and variability along with perfect grammar. In addition, it’s capable of answering a wide variety of questions.
GPT-3 is a language model developed by OpenAI
But nobody’s perfect. It isn’t very accessible. OpenAI granted limited access to a privileged few (including Narrativa for a few months). So, who gets full access to the model and all the data needed to train it? Unfortunately, only a handful of the world’s biggest companies.
Eleuther AI wanted to change this and allow everyone to have access to this technology, both companies and individuals. It’s a necessary step to ensure that artificial intelligence reaches its full potential. Otherwise, it’ll be yet another of society’s inequalities.
GPT-J has 60 billion parameters
On their official page you can see other models that they have democratized.
What can we do with the GPT-J model?
While the GPT-3 model has 175 billion parameters, GPT-J has 60 billion parameters. Does this mean that it is worse? Absolutely not! In fact, GPT-J is better than GPT-3 in code generation tasks. In addition, it can be used to create:
- Chatbots
- Story writing
- Translation of text
- Information searches
If you want to try it, you just have to click here.
The future of these models
It’s very difficult to surpass the achievements of GPT-3 simply because there isn’t any more written text in English to provide further training. This means it can’t continue to grow. So how can the model evolve? GPT-3 would have to be able to understand the world, not just one language. When a child is told what a car is, the next time they see one they understand what it is (even if it is a different type of car). For the GPT-3 model to understand what a car is, it would have to see thousands of images of different cars. For now, that’s a skill only humans possess. The question is: for how long?
Advances in artificial intelligence and in the field of natural processing are increasing. Much is published on the subject and if we read any digital newspaper, it’s very easy to come across news related to AI, but to what extent is all this progress available to society? It’s necessary that both users and recipients are included in the cycle of research, design, development, application and resolution. If more groups like Eleuther AI continue to make the latest models available to society, then we will easily achieve this goal!
Can it be used for regulatory submissions like clinical study reports (CSRs)?
No, it cannot be used for regulatory submissions. The current structure and functions of GPT-J do not support processing and understanding a complex set of inputs from millions of data points and patients. Therefore, this model cannot be utilized to automate processes such as creating Tables, Lists, and Figures (TLFs) and/or patient safety narratives, which are essential for CSR creation and finalization. However, if you need to automate regulatory submissions, Narrativa has the answer.
We have developed several solutions to help pharma and biotech companies expedite the process of introducing their life-saving treatments to the market by automating tiresome, repetitive regulatory tasks. Explore our automation solutions for patient safety narratives and TLFs. We help you reduce financial burdens and save time by supplementing the creation of regulatory-compliant documents.
About us
Narrativa is an internationally recognized content services company that uses its proprietary artificial intelligence and machine learning platforms to build and deploy digital content solutions for enterprises. Its technology suite, consisting of data extraction, data analysis, natural language processing (NLP) and natural language generation (NLG) tools, all seamlessly work together to power a lineup of smart content creation, automated business intelligence reporting and process optimization products for a variety of industries.
Contact us to learn more about our solutions!
Share
