Seven trends in NLG/NLP for 2022

What will the new year bring to the worlds of Natural Language Generation and Processing? In 2021 we have already witnessed great advances such as the launch of GPT-3. But by 2022 we will not only witness record-breaking feats of technology, but more deeply experience the link between humans and technology—that is to say, technology that can really help people. We’ve interviewed our own expert at Narrativa, Manuel Romero, and these are the NLG/NLP trends he says we should look out for in 2022.

Less power, more utility

NLG/NLP researchers have one goal this year, and that is to make models smarter and more useful for everyday implementation. We are well aware that power without control in a car, for instance, is useless. Well, in artificial intelligence, power without utility is useless. Models need to be created with one question in mind: Who will use this?

For this reason, efforts are already being made so that when pre-training an artificial intelligence model, a fine-tuning process is not necessary. We will not have to teach the model tasks, but rather it learns them by itself and adapts.

There are two reasons to end fine-tuning:

• The models are getting bigger and bigger and fine-tuning is progressively more difficult, (not to mention prohibitively expensive). Only the largest global companies can afford it.
• To perform various tasks, numerous models are needed. With the fine-tuning approach, we must define the individual tasks we want it to do (QA, translation, summarization, classification, etc.), which is all very time consuming and cumbersome.

Few-shot learning (FSL)

Few-shot learning (FSL) is a machine learning method in which the training dataset contains limited information. It is focused on creating models capable of understating and doing tasks with just a few examples. 

Imagine that you are training a model to translate. You simply provide the AI with three example sentences and voilà!… you find it can do the fourth one on its own! Unbelievable, right?

Zero-shot learning

Even better: Let’s suppose you don’t provide the AI with examples. Just say: “translate this text,” “paraphrase this in a different way,” etc. The possibilities are becoming endless! 

1.In search of a super intelligence: AGI

What is AGI? Although it’s not a term that we are used to hearing frequently, artificial general intelligence (AGI) is the hypothetical ability of an intelligent agent to understand or learn any intellectual task that a human being can.

If we compare it with “normal” artificial intelligence, the big difference is that while AI is designed to solve a specific problem without possessing cognitive abilities, AGI is capable of understanding multiple fields at a time in context to:

• Reason
• Represent knowledge
• Plan
• Learn
• Communicate in natural language

2022 will see a big push toward AGI.

2. Polyglot artificial intelligence models

We are used to English being the universal language. But this won’t last forever as artificial intelligence is facilitating instant translation between them. Currently, work is being done to make the T0 model multilingual and adapt to several languages.

According to the newspaper El País, Spanish still represents less than 30% of the world market for natural language processing technologies. As such, Narrativa stands out as a pioneering startup that has been generating news in Spanish since it began. While our automatic content can be written in any language, our AI has a special predilection for Spanish.

3. Transformers are in fashion

A few months ago we told you about what a transformer was with regard to NLP. They became famous for translation tasks, but now they are used for Computer Vision such as images and text. Without a doubt, better results are constantly being obtained and it’s even applied in speech recognition too.

4. Can I program without knowing anything about programming?

The answer is yes! In 2021 we were surprised by GitHub Copilot, an artificial intelligence tool developed by GitHub and OpenAI. Copilot helps you write code faster and with less work, all thanks to AI.

In addition to Copilot, this year further progress will be made on the development of MLOps tools that will make the technology more user friendly. We’ll also be keeping an eye out for developments on Codex, another OpenAI tool, that translates natural language into code. 

5. Big companies will (more or less) democratize their technology

Big giants like Google will improve their tools so that their average users, without any knowledge in fields such as artificial intelligence, can use them. They will aim to serve up their array of technologies as easily as possible to establish or maintain market share. We’ll keep you updated on future developments!

6. DALL·E will gain a brother

Do you remember DALL·E, the artificial intelligence model capable of generating images from text? It combines both an understanding of natural language with the generation of realistic images.

Well, it’s likely that in 2022 it will be given a brother. Models that work with images and text as well as diffusion models will improve with image synthesis at a higher quality.

GAUGAN, the Nvidia model, can already create its own models with very believable effects: if we provide it the instruction to “create a landscape with a beach,” we will be given a very realistic image of just what you’d expect. We can even draw elements for our landscape such as stones, mountains and palm trees. This will assist graphic designers and the world at large!

Will we see GPT-4 this year?

Maybe. But of course, improvements will be focused on utility, not on increasing capacity. Models that are useful on a day-to-day basis are needed, and research departments want to make them lighter instead of more powerful so that they can fit average user/company hardware. 

About Narrativa

Narrativa is an internationally recognized content services company that uses its proprietary artificial intelligence and machine learning platforms to build and deploy digital content solutions for enterprises. Its technology suite, consisting of data extraction, data analysis, natural language processing (NLP) and natural language generation (NLG) tools, all seamlessly work together to power a lineup of smart content creation, automated business intelligence reporting and process optimization products for a variety of industries.

Contact us to learn more about our solutions!ds in NLG/NLP for 2022