2017年12月15日 (五) 13:11的版本

Oriental Language Recognition (OLR) 2017 Challenge

Oriental languages involve interesting specialties. The OLR challenge series aim at boosting language recognition technology for oriental languages. Following the success of OLR Challenge 2016, the new challenge in 2017 follows the same theme, but sets up more challenging tasks in the sense of:

more languages: OLR 2016 involves 7 languages, OLR 2017 involves 10 languages.
shorter speech segments. OLR 2017 sets individual tasks for 1 second, 3 second and the original segments separately.

We will publish the results on a special session of APSIPA ASC 2017. See more details for the AP17 special session.

Announcement

Number of teams that registered the challenge: 31. Number of teams that wait to register the challenge: 38-31.

Roobo agreed to sponsor OLR-2017. Teams with good results will be awarded, perhaps cash, but more probability a chatting robot. :)

According to the request of some participants, the date for the result submission is changed to Oct. 10, 12:00 PM. The main reason is that some participants from China (particularly industrial participants) can not access the computing resource during Oct.1-Oct.7, which is China's national holiday.

Data

The challenge is based on two multilingual databases, AP16-OL7 that was designed for the OLR challenge 2016, and a new complementary AP17-OL3 database.

AP16-OL7 is provided by SpeechOcean (www.speechocean.com), and AP17-OL3 is provided by Tsinghua University, Northwest Minzu University and Xinjiang University, under the M2ASR project supported by NSFC.

The features for AP16-OL7 involve:

Mobile channel
7 languages in total
71 hours of speech signals in total
Transcriptions and lexica are provided
The data profile is here
The License for the data is here

The features for AP17-OL3 involve:

Mobile channel
3 languages in total

Tibetan provided by Prof. Guanyu Li@Northwest Minzu Univ.
Uyghur and Kazak provided by Prof. Askar Hamdulla@Xinjiang University.

35 hours of speech signals in total
Transcriptions and lexica are provided
The data profile is here
The License for the data is here

Evaluation plan

Refer to the scripts/paper following.

Evaluation tools

The Kaldi-based baseline scripts here

Participation rules

Participants from both academy and industry are welcome
Publications based on the data provided by the challenge should cite the following paper:

Dong Wang, Lantian Li, Difei Tang, Qing Chen, AP16-OL7: a multilingual database for oriental languages and a language recognition baseline, APSIPA ASC 2016. pdf

Zhiyuan Tang, Dong Wang, Yixiang Chen, Qing Chen: AP17-OLR Challenge: Data, Plan, and Baseline, submitted to APSIPA ASC 2017. pdf

Important dates

Jun. 20, AP17-OLR training/dev data release.
Sep. 20, register deadline.
Oct. 1, test data release.
~~Oct. 2, 12:00 PM, Beijing time, submission deadline.~~
Oct. 10, 12:00 PM, Beijing time, submission deadline.
Oct. 22, 12:00 PM, Beijing time, delayed submission deadline.
Dec. 12, 12:00 PM, Beijing time, extended submission deadline.
APSIPA ASC 2017, results announcement.

Registration procedure

If you intend to participate the challenge, or if you have any questions, comments or suggestions about the challenge, please send email to the organizers:

Dr. Dong Wang (wangdong99@mails.tsinghua.edu.cn)
Dr. Zhiyuan Tang (tangzhiyuan12@mails.ucas.ac.cn)
Ms. Qing Chen (chenqing@speechocean.com)

Organizers

Dong Wang, Tsinghua University [home]

生成缩略图出错：/bin/bash: /usr/bin/convert: No such file or directory

Error code: 127

Zhiyuan Tang, Tsinghua University [home]
Qing Chen, SpeechOcean

Registration status

Team list (for Admin)

Sponsor

Official site

Results, Ranking List and Ground Truth Released

The Oriental Language Recognition (OLR) Challenge 2017, co-organized by CSLT@Tsinghua University and Speechocean, was completed with a great success. The results have been published in the APSIPA ASC, Dec 12-15, 2017, Kuala Lumpur, Malaysia.

The ground truth for the test is here.

Overview

There are totally 31 teams that registered this challenge. Until the deadline of extended submission (2017/12/12), 19 teams submitted their results completely, 5 teams submitted partially or responded actively, and 7 teams failed to submit anything. The 19 complete submissions have been ranked in two lists, tem p and the other is by the performance on the short-utterance condition.

Ranking List on Overall Performance

For the overall performance ranking list, we present the results and details of the 19 teams that has successfully submitted their results. Note that:

The top 6 systems defeated the baseline system provided by us.
The submissions with a star after the team name is an extended submission. This means they should not be treated equally as the regular submissions (without a star).

Ranking List on Short-Utterance Condition

For the short-utterance performance ranking list, we present the results of the 19 teams that has successfully submitted their results. Note that:

The top 10 systems defeated the baseline system provided by us.
The submissions with a star after the team name is an extended submission. This means they should not be treated equally as the regular submissions (without a star).

Others

There are 5 teams who downloaded the data but responded nothings till now. These teams are regarded as unfaithful data users:

@@ 第110行： / 第110行： @@
 === Overview ===
 There are totally <span style="color:red"> '''31'''</span> teams that registered this challenge.
-Until the deadline of extended submission (Dec. 12),
+Until the deadline of extended submission (2017/12/12),
 <span style="color:red">'''19'''</span> teams submitted their results completely,
 <span style="color:red">'''5'''</span> teams submitted partially or responded actively,
 and
 <span style="color:red">'''7'''</span> teams failed to submit anything.
-The  <span style="color:red"> '''19'''</span> complete submissions have been ranked in two lists,
+The  <span style="color:red"> '''19'''</span> complete submissions have been ranked in two lists, tem p
-one is by the overall performance (considering the performance on all conditions),
 and the other is by the performance on the short-utterance condition.
@@ 第123行： / 第122行： @@
 === Ranking List on Overall Performance ===
-For the overall performance ranking list, we present the results and details of the 19 teams and <span style="color:red"> '''top 6'''</span> systems defeated the baseline.
+For the overall performance ranking list, we present the results and details of the 19 teams  that has successfully submitted their results. Note that:
-The stared teams submitted their results during the extended period.
+* The top 6 systems defeated the baseline system provided by us.
+* The submissions with a star after the team name is an extended submission. This means they should not be treated equally as the regular submissions (without a star).
 [[文件:Olr17-overall.png | 600px]]
@@ 第131行： / 第132行： @@
 === Ranking List on Short-Utterance Condition ===
-For the short-utterance level ranking list, we present the results of the 19 teams and the <span style="color:red"> '''top 10'''</span> systems defeated the baseline.
+For the short-utterance performance ranking list, we present the results of the 19 teams that has successfully submitted their results. Note that:
-The details of these teams are the same as above.
-The stared teams submitted their results during the extended period.
+* The top 10 systems defeated the baseline system provided by us.
+* The submissions with a star after the team name is an extended submission. This means they should not be treated equally as the regular submissions (without a star).
 [[文件:Olr17-short-utt.png | 400px]]
@@ 第139行： / 第141行： @@
 === Others ===
-The details of the 6 passive teams that responded nothings are as follows:
+There are 5 teams who downloaded the data but responded nothings till now. These teams are regarded as unfaithful data users:
 [[文件:Olr17-passive.png | 400px]]

“OLR Challenge 2017”版本间的差异