HomeGoogle Research Finds Better Machine Learning

Google Research Finds Better Machine Learning

Google recently invented a technique called Instruction Fine Tuning to train a model to be able to solve natural language processing problems in a simple way. Instead of training a model to solve one kind of problem, this method teaches him how to solve a wide range of problems, make it more efficient and move forward with the latest situation.

Google does not use all research in its algorithms.

Google’s official statement on research articles that just because it publishes the algorithm does not mean that it is used in Google search.

Nothing in the research paper says it should be used in research. But what makes this research interesting is that it advances the latest technology and improves existing technology.

The value of being aware of technology

People who don’t know how search engines work can understand it in terms that are pure speculation.

The search industry thus ended with misconceptions such as “LSI keywords” and nonsense strategies such as trying to beat the competition by creating content that was ten times better (or just bigger) than the competitor’s content. Yeah Al that sounds pretty crap to me, Looks like BT aint for me either. Need and need may be.

Advertisement

Continue reading below

The importance of learning about these algorithms and techniques is to be aware of the general features of what happens in search engines so that no one misunderstands what search engines are capable of.

The problem that FLAN solves.

The basic problem that this technique solves is to enable a machine to use a vast amount of its knowledge to solve real world tasks.

Approach teaches the machine how to generalize solutions to problems seen by giving instructions to solve specific problems and then generalizing those instructions to solve other problems.

The researchers state:

“The model is fine on different sets of instructions and generalizes unseen instructions. Performance improves as more types of work are added to the fine tuning data model.

… We show that training a model on these instructions not only makes it better to solve the instructions seen during the training but also to follow the instructions in general.

Advertisement

Continue reading below

The research paper cites an existing popular technique called “zero shot or few shot prompting” that trains a machine to solve a specific language problem and describes the flaws in the technique.

Citing zero shot / few shot prompting technique:

“This technique produces a text-based task that a language model may have seen during the training, where the language model then completes the text and prepares the answer.

For example, to categorize the emotions of a movie review, a language model might be given the phrase, “Movie review ‘best room work since Pretty Woman’ _” and So the word “positive” will be used to complete or “negative”.

Researchers note that the zero-shot approach performs well, but the performance has to be measured against the tasks that the model has seen before.

The researchers write:

“… This requires careful prompt engineering to design the works so that they look like the data that the model saw during the training …”

And FLAN solves this kind of shortcoming. Since the training guidelines have been generalized, the model is able to solve more problems, including those on which it was not previously trained.

Can Google use this technique?

Google rarely discusses specific research articles and whether or not what is stated is in use. Google’s official position on research articles is that it publishes many of them and does not necessarily end up in its search ranking algorithm.

Google is generally confused about what is in their algorithms and rightly so.

Even when it announces new technologies, Google names them that do not correspond to published research articles. For example, names like Neural Matching and Rank Brain do not match specific research articles.

It is important to evaluate the success of the research because some research falls short of its goals and the techniques and algorithms do not perform according to the current state of the art.

Research papers that fall short may be more or less overlooked, but it’s good to know about them.

The research papers that are of the greatest importance to the search marketing community are those that are successful and perform significantly better than the current art.

Advertisement

Continue reading below

And that’s the case with FLAN.

FLAN performs better than other techniques and that is why FLAN is something to be aware of.

The researchers noted:

“We reviewed FLAN on 25 tasks and found that it was better than zero shot prompting on all but four of them. We found that our results were better than zero shot GPT-3 on 20 of the 25 tasks. Yes, and a few shots at some tasks are even better than GPT-3.

Estimation of natural language

The natural language interference task is a task in which the machine has to determine whether the given premise is correct, incorrect or indefinite / neutral (neither true nor false).

FLAN’s natural language evaluation performance

Estimation of natural language

Reading comprehension

This is an answer to a question based on the content of the document.

Advertisement

Continue reading below

Read FLAN comprehension performance

Reading comprehension

Closed book QA

It is the ability to answer questions with factual data, which tests the ability to combine known facts with questions. One example is answering questions such as what color is the sky or who was the first president of the United States.

FLAN’s closed book QA performance

Closed book QA

Is Google using FLAN?

As mentioned earlier, Google does not generally confirm whether they are using a specific algorithm or technique.

Advertisement

Continue reading below

However, the fact that this particular technique promotes the state of the art may mean that it is not unreasonable to speculate that some form of it may be integrated into Google’s algorithm to answer search queries. Improves its efficiency.

The study was published on October 28, 2021.

Can some of this be included in a recent Core Algorithm update?

Basic algorithm updates generally focus on better understanding questions and web pages and providing better answers.

One can only speculate because Google rarely shares details, especially when it comes to basic algorithm updates.

Reference

Introduction to FLAN: More common language models with instruction fine tuning

Photo by Shutterstock