AI teams shared prize fund at Up Great READ//ABLE contest
Within the framework of the Up Great technology contest READ//ABLE, the first test cycle has been completed and the winners of the "Grammar" and "Grammar.Eng" nominations have been announced.
The Up Great contest READ//ABLE is aimed at creating an AI algorithm capable of helping teachers identify logical, factual, stylistic and other semantic errors in schoolchildren's essays in Russian, literature, history, social studies and English. The goal of the contest is to advance the development of strong AI and support the developers community in the field of machine learning.
More than 180 applications were submitted for the first cycle of the contest. According to the qualification results, 11 teams were admitted to the tests, where they showed effective solutions. As part of the tests, AI algorithms checked 500 essays in Russian and the same number in English language. The automated platform compared the texts tested by AI algorithms and real teachers, and identified the best solutions.
The winners of special nominations "Grammar" and "Grammar. Eng”, whose AI algorithms correctly identified the largest number of grammatical and speech errors, shared the prize fund of 20 million rubles.
The results of the nomination "Grammar" for texts in Russian:
- "Raketa" (developers from MIPT and Lomonosov Moscow State University) – 1st place, 7 million rubles.
- "Antiplagiat" (developers of "Antiplagiat") – 2nd place *
- Chemist (developers from MIPT) – 3rd place, 2 million rubles
- NugumanovTeam (developers from Ufa) – 4th place, 1 million rubles.
* In accordance with the rules of the competition, one team can apply for the prize fund only in one of the nominations. The "Antiplagiat" team took 2nd place both in the "Grammar" and "Grammar.Eng" categories. By the decision of the team, the cash prize is awarded in the category "Grammar.Eng".
The results of the nomination "Grammar.Eng" for texts in English:
- "DeepPavlov" (developers of DeepPavlov) – 1st place, 7 million rubles.
- "Antiplagiat" (developers of "Antiplagiat") – 2nd place, 2 million rubles.
- "Nanosemantics" (developers of "Ashmanov's Neural Network" and ORFO service) – 3rd place, 1 million rubles.
In terms of the main task of the contest, none of the teams has yet achieved a result corresponding to the level of teachers. Participants will be able to compete again for the prize fund of the contest in the amount of 200 million rubles in tests of the second cycle, scheduled for the spring of 2021. In the course of the analysis of texts in Russian, the best intermediate result was shown by the "Antiplagiat" team (accuracy level – 57.8%). For the English language, the “NeuroReading” team demonstrated the highest quality solution (accuracy level – 84.5%). The teams "Raketa" and "Nanosemantics" as well as the team from St. Petersburg Seedh AI were also among the leaders. At this stage, the AI algorithms were most successful in identifying lexical, grammatical and speech errors. The greatest difficulties arise when determining logical errors.
The solutions of the participants are planned for pilot implementation and operation in 2021-22 in a number of schools in different regions of Russia.
Yuri Molodykh, the head of the organizing committee of the Up Great technology contest READ//ABLE:
“The test results showed that the task of creating an AI algorithm capable of detecting all types of semantic errors can be solved on the basis of existing technologies. The teams have shown effective solutions that can already be used as an effective teacher assistant at school. Such an assistant will be able to facilitate the daily routine work, allowing teachers to concentrate their efforts on the most important and complex pedagogical tasks, the formation of logical and critical thinking."
Mikhail Burtsev, Head of the Laboratory of Neural Systems and Deep Learning at MIPT, “NeuroReading” team:
“We are very pleased that we have managed to create the best system for checking essays in English and took first place in the nomination for English grammar. Our library DeepPavlov helped us with this, with an orchestrator of error annotators and skills for correcting them being built on it. The high speed of processing the essay – only 5-6 seconds – confirmed that the library is ready to work with large amounts of data in a real industrial load. In addition, the teamwork of our laboratory and the support of Phystech were the most powerful factors on the way to victory. I hope that we will be able to continue to take part in the competition, although it may take more than one year to overcome the technological barrier.”
Yuri Chekhovich, Executive Director of “Antiplagiat”:
“This is the first time we took part in the competition, which did not concern the area of our core competence – content similarity detection. Moreover, we are delighted that the results of our solution for the Russian language have confirmed the expertise of the “Antiplagiat” team. I am sure that we will be able to overcome the technological barrier in the coming stages that is to create a solution to evaluate essays not worse than a teacher."
Operator of the Up Great technology contests – RVC. Co-organizers – Skolkovo Foundation and ASI. The technical partner of the contest READ//ABLE is the AI competence center at MIPT.