目录
2. Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation [PDF] 摘要
9. Hybrid Attention-Based Transformer Block Model for Distant Supervision Relation Extraction [PDF] 摘要
13. BaitWatcher: A lightweight web interface for the detection of incongruent news headlines [PDF] 摘要
16. Can Embeddings Adequately Represent Medical Terminology? New Large-Scale Medical Term Similarity Datasets Have the Answer! [PDF] 摘要
17. XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization [PDF] 摘要
19. COVID-19 and Computer Audition: An Overview on What Speech & Sound Analysis Could Contribute in the SARS-CoV-2 Corona Crisis [PDF] 摘要
20. EQL -- an extremely easy to learn knowledge graph query language, achieving highspeed and precise search [PDF] 摘要
摘要
1. The Medical Scribe: Corpus Development and Model Performance Analyses [PDF] 返回目录
Izhak Shafran, Nan Du, Linh Tran, Amanda Perry, Lauren Keyes, Mark Knichel, Ashley Domin, Lei Huang, Yuhui Chen, Gang Li, Mingqiu Wang, Laurent El Shafey, Hagen Soltau, Justin S. Paul
Abstract: There is a growing interest in creating tools to assist in clinical note generation using the audio of provider-patient encounters. Motivated by this goal and with the help of providers and medical scribes, we developed an annotation scheme to extract relevant clinical concepts. We used this annotation scheme to label a corpus of about 6k clinical encounters. This was used to train a state-of-the-art tagging model. We report ontologies, labeling results, model performances, and detailed analyses of the results. Our results show that the entities related to medications can be extracted with a relatively high accuracy of 0.90 F-score, followed by symptoms at 0.72 F-score, and conditions at 0.57 F-score. In our task, we not only identify where the symptoms are mentioned but also map them to canonical forms as they appear in the clinical notes. Of the different types of errors, in about 19-38% of the cases, we find that the model output was correct, and about 17-32% of the errors do not impact the clinical note. Taken together, the models developed in this work are more useful than the F-scores reflect, making it a promising approach for practical applications.
摘要：在创建工具使用提供门诊遇到的音频在临床笔记代以帮助越来越大的兴趣。通过这一目标，并与供应商和医药学家的帮助启发，我们开发了一个注释计划提取相关临床概念。我们使用这个注释方案标记6K左右临床会诊的语料库。这被用于训练状态的最先进的标签模型。我们报告本体，标记结果，模型表演，结果的详细分析。我们的研究结果表明，有关药物的实体可以用0.90的F值的精度比较高，然后在0.72 F-得分症状，并在0.57 F-分数条件下提取。在我们的任务，我们不仅查明被提到的症状，而且他们，因为他们出现在临床记录映射到规范形式。不同类型的错误，在的情况下，约19-38％，我们发现模型的输出是正确的，并且约17-32误差％不影响临床笔记。总之，在这项工作中开发的模型比F-分数反映出更加有用，使得它在实际应用中有前途的方法。
Izhak Shafran, Nan Du, Linh Tran, Amanda Perry, Lauren Keyes, Mark Knichel, Ashley Domin, Lei Huang, Yuhui Chen, Gang Li, Mingqiu Wang, Laurent El Shafey, Hagen Soltau, Justin S. Paul
Abstract: There is a growing interest in creating tools to assist in clinical note generation using the audio of provider-patient encounters. Motivated by this goal and with the help of providers and medical scribes, we developed an annotation scheme to extract relevant clinical concepts. We used this annotation scheme to label a corpus of about 6k clinical encounters. This was used to train a state-of-the-art tagging model. We report ontologies, labeling results, model performances, and detailed analyses of the results. Our results show that the entities related to medications can be extracted with a relatively high accuracy of 0.90 F-score, followed by symptoms at 0.72 F-score, and conditions at 0.57 F-score. In our task, we not only identify where the symptoms are mentioned but also map them to canonical forms as they appear in the clinical notes. Of the different types of errors, in about 19-38% of the cases, we find that the model output was correct, and about 17-32% of the errors do not impact the clinical note. Taken together, the models developed in this work are more useful than the F-scores reflect, making it a promising approach for practical applications.
摘要：在创建工具使用提供门诊遇到的音频在临床笔记代以帮助越来越大的兴趣。通过这一目标，并与供应商和医药学家的帮助启发，我们开发了一个注释计划提取相关临床概念。我们使用这个注释方案标记6K左右临床会诊的语料库。这被用于训练状态的最先进的标签模型。我们报告本体，标记结果，模型表演，结果的详细分析。我们的研究结果表明，有关药物的实体可以用0.90的F值的精度比较高，然后在0.72 F-得分症状，并在0.57 F-分数条件下提取。在我们的任务，我们不仅查明被提到的症状，而且他们，因为他们出现在临床记录映射到规范形式。不同类型的错误，在的情况下，约19-38％，我们发现模型的输出是正确的，并且约17-32误差％不影响临床笔记。总之，在这项工作中开发的模型比F-分数反映出更加有用，使得它在实际应用中有前途的方法。
2. Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation [PDF] 返回目录
Haiyan Yin, Dingcheng Li, Xu Li, Ping Li
Abstract: Training generative models that can generate high-quality text with sufficient diversity is an important open problem for Natural Language Generation (NLG) community. Recently, generative adversarial models have been applied extensively on text generation tasks, where the adversarially trained generators alleviate the exposure bias experienced by conventional maximum likelihood approaches and result in promising generation quality. However, due to the notorious defect of mode collapse for adversarial training, the adversarially trained generators face a quality-diversity trade-off, i.e., the generator models tend to sacrifice generation diversity severely for increasing generation quality. In this paper, we propose a novel approach which aims to improve the performance of adversarial text generation via efficiently decelerating mode collapse of the adversarial training. To this end, we introduce a cooperative training paradigm, where a language model is cooperatively trained with the generator and we utilize the language model to efficiently shape the data distribution of the generator against mode collapse. Moreover, instead of engaging the cooperative update for the generator in a principled way, we formulate a meta learning mechanism, where the cooperative update to the generator serves as a high level meta task, with an intuition of ensuring the parameters of the generator after the adversarial update would stay resistant against mode collapse. In the experiment, we demonstrate our proposed approach can efficiently slow down the pace of mode collapse for the adversarial text generators. Overall, our proposed method is able to outperform the baseline approaches with significant margins in terms of both generation quality and diversity in the testified domains.
摘要：培训生成模型，可以产生足够的多样性，高品质的文本是自然语言生成（NLG）社区一个重要的开放问题。近日，生成对抗性模型已被广泛的文本生成任务，其中adversarially训练的发电机减轻传统的最大似然经历了曝光补偿方法和结果看好代品质应用。然而，由于模式崩溃了对抗性训练的臭名昭著的缺陷，在adversarially训练的发电机面临质量多样性的权衡，即发电机模型往往严重牺牲一代多样性提高发电质量。在本文中，我们提出了一种新的方法，旨在通过有效的减速对抗性训练模式崩溃，提高对抗性的文本生成的性能。为此，我们引入了合作培训模式，其中语言模型协同与发电机培训，我们利用语言模型能够有效地对塑造模式崩溃发电机的数据分布。此外，而不是一个有原则的方式参与合作更新发电机，我们制订了元学习机制，在合作更新到发电机作为高层次元的任务，以确保发电机后的参数的直觉对抗升级将保持对模式抗倒塌。在实验中，我们证明了我们提出的方法能有效地减缓模式崩溃的步伐，为对抗文本发电机。总体而言，我们提出的方法能够跑赢基准有显著利润率接近在作证领域都产生质量和多样性方面。
Haiyan Yin, Dingcheng Li, Xu Li, Ping Li
Abstract: Training generative models that can generate high-quality text with sufficient diversity is an important open problem for Natural Language Generation (NLG) community. Recently, generative adversarial models have been applied extensively on text generation tasks, where the adversarially trained generators alleviate the exposure bias experienced by conventional maximum likelihood approaches and result in promising generation quality. However, due to the notorious defect of mode collapse for adversarial training, the adversarially trained generators face a quality-diversity trade-off, i.e., the generator models tend to sacrifice generation diversity severely for increasing generation quality. In this paper, we propose a novel approach which aims to improve the performance of adversarial text generation via efficiently decelerating mode collapse of the adversarial training. To this end, we introduce a cooperative training paradigm, where a language model is cooperatively trained with the generator and we utilize the language model to efficiently shape the data distribution of the generator against mode collapse. Moreover, instead of engaging the cooperative update for the generator in a principled way, we formulate a meta learning mechanism, where the cooperative update to the generator serves as a high level meta task, with an intuition of ensuring the parameters of the generator after the adversarial update would stay resistant against mode collapse. In the experiment, we demonstrate our proposed approach can efficiently slow down the pace of mode collapse for the adversarial text generators. Overall, our proposed method is able to outperform the baseline approaches with significant margins in terms of both generation quality and diversity in the testified domains.
摘要：培训生成模型，可以产生足够的多样性，高品质的文本是自然语言生成（NLG）社区一个重要的开放问题。近日，生成对抗性模型已被广泛的文本生成任务，其中adversarially训练的发电机减轻传统的最大似然经历了曝光补偿方法和结果看好代品质应用。然而，由于模式崩溃了对抗性训练的臭名昭著的缺陷，在adversarially训练的发电机面临质量多样性的权衡，即发电机模型往往严重牺牲一代多样性提高发电质量。在本文中，我们提出了一种新的方法，旨在通过有效的减速对抗性训练模式崩溃，提高对抗性的文本生成的性能。为此，我们引入了合作培训模式，其中语言模型协同与发电机培训，我们利用语言模型能够有效地对塑造模式崩溃发电机的数据分布。此外，而不是一个有原则的方式参与合作更新发电机，我们制订了元学习机制，在合作更新到发电机作为高层次元的任务，以确保发电机后的参数的直觉对抗升级将保持对模式抗倒塌。在实验中，我们证明了我们提出的方法能有效地减缓模式崩溃的步伐，为对抗文本发电机。总体而言，我们提出的方法能够跑赢基准有显著利润率接近在作证领域都产生质量和多样性方面。
3. Masakhane -- Machine Translation For Africa [PDF] 返回目录
Iroro Orife, Julia Kreutzer, Blessing Sibanda, Daniel Whitenack, Kathleen Siminyu, Laura Martinus, Jamiil Toure Ali, Jade Abbott, Vukosi Marivate, Salomon Kabongo, Musie Meressa, Espoir Murhabazi, Orevaoghene Ahia, Elan van Biljon, Arshath Ramkilowan, Adewale Akinfaderin, Alp Öktem, Wole Akin, Ghollah Kioko, Kevin Degila, Herman Kamper, Bonaventure Dossou, Chris Emezue, Kelechi Ogueji, Abdallah Bashir
Abstract: Africa has over 2000 languages. Despite this, African languages account for a small portion of available resources and publications in Natural Language Processing (NLP). This is due to multiple factors, including: a lack of focus from government and funding, discoverability, a lack of community, sheer language complexity, difficulty in reproducing papers and no benchmarks to compare techniques. To begin to address the identified problems, MASAKHANE, an open-source, continent-wide, distributed, online research effort for machine translation for African languages, was founded. In this paper, we discuss our methodology for building the community and spurring research from the African continent, as well as outline the success of the community in terms of addressing the identified problems affecting African NLP.
摘要：非洲拥有超过2000种语言。尽管如此，非洲语言占可用资源和出版物的自然语言处理（NLP）的一小部分。这是由于多种因素，其中包括：政府和资金，可发现一个缺乏重点，缺乏社区的，纯粹的语言复杂，再现试卷难度和无基准进行比较的技术。要开始解决所识别的问题，MASAKHANE，一个开源的，全大陆，机器翻译的非洲语言分布，在线研究工作，成立了。在本文中，我们讨论了我们的方法对来自非洲大陆的社区建设和刺激的研究，以及轮廓社区的成功解决影响非洲NLP中发现的问题的条款。
Iroro Orife, Julia Kreutzer, Blessing Sibanda, Daniel Whitenack, Kathleen Siminyu, Laura Martinus, Jamiil Toure Ali, Jade Abbott, Vukosi Marivate, Salomon Kabongo, Musie Meressa, Espoir Murhabazi, Orevaoghene Ahia, Elan van Biljon, Arshath Ramkilowan, Adewale Akinfaderin, Alp Öktem, Wole Akin, Ghollah Kioko, Kevin Degila, Herman Kamper, Bonaventure Dossou, Chris Emezue, Kelechi Ogueji, Abdallah Bashir
Abstract: Africa has over 2000 languages. Despite this, African languages account for a small portion of available resources and publications in Natural Language Processing (NLP). This is due to multiple factors, including: a lack of focus from government and funding, discoverability, a lack of community, sheer language complexity, difficulty in reproducing papers and no benchmarks to compare techniques. To begin to address the identified problems, MASAKHANE, an open-source, continent-wide, distributed, online research effort for machine translation for African languages, was founded. In this paper, we discuss our methodology for building the community and spurring research from the African continent, as well as outline the success of the community in terms of addressing the identified problems affecting African NLP.
摘要：非洲拥有超过2000种语言。尽管如此，非洲语言占可用资源和出版物的自然语言处理（NLP）的一小部分。这是由于多种因素，其中包括：政府和资金，可发现一个缺乏重点，缺乏社区的，纯粹的语言复杂，再现试卷难度和无基准进行比较的技术。要开始解决所识别的问题，MASAKHANE，一个开源的，全大陆，机器翻译的非洲语言分布，在线研究工作，成立了。在本文中，我们讨论了我们的方法对来自非洲大陆的社区建设和刺激的研究，以及轮廓社区的成功解决影响非洲NLP中发现的问题的条款。
4. Generating Major Types of Chinese Classical Poetry in a Uniformed Framework [PDF] 返回目录
Jinyi Hu, Maosong Sun
Abstract: Poetry generation is an interesting research topic in the field of text generation. As one of the most valuable literary and cultural heritages of China, Chinese classical poetry is very familiar and loved by Chinese people from generation to generation. It has many particular characteristics in its language structure, ranging from form, sound to meaning, thus is regarded as an ideal testing task for text generation. In this paper, we propose a GPT-2 based uniformed framework for generating major types of Chinese classical poems. We define a unified format for formulating all types of training samples by integrating detailed form information, then present a simple form-stressed weighting method in GPT-2 to strengthen the control to the form of the generated poems, with special emphasis on those forms with longer body length. Preliminary experimental results show this enhanced model can generate Chinese classical poems of major types with high quality in both form and content, validating the effectiveness of the proposed strategy. The model has been incorporated into Jiuge, the most influential Chinese classical poetry generation system developed by Tsinghua University (Guo et al., 2019).
摘要：诗歌产生是在文本生成领域的一个有趣的研究课题。作为中国最有价值的文学和文化遗产之一，中国古典诗歌是非常熟悉和喜爱的中国人代代相传。它在它的语言结构的许多具体特点，从形式，声音的意思，因此被视为理想的文本生成测试任务。在本文中，我们提出了产生主要类型的中国古典诗词的GPT-2的穿制服的框架。我们定义了一个统一的格式对通过集成的详细形式的信息制定所有类型的训练样本，那么在给出一个简单的形式加应力加权方法GPT-2，以加强控制到生成的诗的形式，特别着重于与那些形式更长的车身长度。初步实验结果表明，该模型增强可产生主要类型，在形式和内容的高品质的中国古典诗词，验证该策略的有效性。该模型已被纳入九哥，由清华大学研制的最有影响力的中国古典诗歌生成系统（Guo等，2019）。
Jinyi Hu, Maosong Sun
Abstract: Poetry generation is an interesting research topic in the field of text generation. As one of the most valuable literary and cultural heritages of China, Chinese classical poetry is very familiar and loved by Chinese people from generation to generation. It has many particular characteristics in its language structure, ranging from form, sound to meaning, thus is regarded as an ideal testing task for text generation. In this paper, we propose a GPT-2 based uniformed framework for generating major types of Chinese classical poems. We define a unified format for formulating all types of training samples by integrating detailed form information, then present a simple form-stressed weighting method in GPT-2 to strengthen the control to the form of the generated poems, with special emphasis on those forms with longer body length. Preliminary experimental results show this enhanced model can generate Chinese classical poems of major types with high quality in both form and content, validating the effectiveness of the proposed strategy. The model has been incorporated into Jiuge, the most influential Chinese classical poetry generation system developed by Tsinghua University (Guo et al., 2019).
摘要：诗歌产生是在文本生成领域的一个有趣的研究课题。作为中国最有价值的文学和文化遗产之一，中国古典诗歌是非常熟悉和喜爱的中国人代代相传。它在它的语言结构的许多具体特点，从形式，声音的意思，因此被视为理想的文本生成测试任务。在本文中，我们提出了产生主要类型的中国古典诗词的GPT-2的穿制服的框架。我们定义了一个统一的格式对通过集成的详细形式的信息制定所有类型的训练样本，那么在给出一个简单的形式加应力加权方法GPT-2，以加强控制到生成的诗的形式，特别着重于与那些形式更长的车身长度。初步实验结果表明，该模型增强可产生主要类型，在形式和内容的高品质的中国古典诗词，验证该策略的有效性。该模型已被纳入九哥，由清华大学研制的最有影响力的中国古典诗歌生成系统（Guo等，2019）。
5. Tigrinya Neural Machine Translation with Transfer Learning for Humanitarian Response [PDF] 返回目录
Alp Öktem, Mirko Plitt, Grace Tang
Abstract: We report our experiments in building a domain-specific Tigrinya-to-English neural machine translation system. We use transfer learning from other Ge'ez script languages and report an improvement of 1.3 BLEU points over a classic neural baseline. We publish our development pipeline as an open-source library and also provide a demonstration application.
摘要：我们在建立一个特定领域的提格雷语到英语神经机器翻译系统报告我们的实验。我们使用来自其他吉兹字母语言迁移学习并通过经典的神经基线报告的1.3 BLEU点的提高。我们发布我们的发展管道作为一个开放源码库，并且还提供了示范应用。
Alp Öktem, Mirko Plitt, Grace Tang
Abstract: We report our experiments in building a domain-specific Tigrinya-to-English neural machine translation system. We use transfer learning from other Ge'ez script languages and report an improvement of 1.3 BLEU points over a classic neural baseline. We publish our development pipeline as an open-source library and also provide a demonstration application.
摘要：我们在建立一个特定领域的提格雷语到英语神经机器翻译系统报告我们的实验。我们使用来自其他吉兹字母语言迁移学习并通过经典的神经基线报告的1.3 BLEU点的提高。我们发布我们的发展管道作为一个开放源码库，并且还提供了示范应用。
6. Matching Text with Deep Mutual Information Estimation [PDF] 返回目录
Xixi Zhou, Chengxi Li, Jiajun Bu, Chengwei Yao, Keyue Shi, Zhi Yu, Zhou Yu
Abstract: Text matching is a core natural language processing research problem. How to retain sufficient information on both content and structure information is one important challenge. In this paper, we present a neural approach for general-purpose text matching with deep mutual information estimation incorporated. Our approach, Text matching with Deep Info Max (TIM), is integrated with a procedure of unsupervised learning of representations by maximizing the mutual information between text matching neural network's input and output. We use both global and local mutual information to learn text representations. We evaluate our text matching approach on several tasks including natural language inference, paraphrase identification, and answer selection. Compared to the state-of-the-art approaches, the experiments show that our method integrated with mutual information estimation learns better text representation and achieves better experimental results of text matching tasks without exploiting pretraining on external data.
摘要：文本匹配是一个核心的自然语言处理研究的问题。如何留住在内容和结构信息充分的信息是一个重要的挑战。在本文中，我们提出了以合并深互信息估计通用文本匹配神经的方法。我们的做法，文本深层信息Max（TIM）的匹配，通过最大化文本匹配的神经网络的输入和输出之间的互信息与交涉无监督学习的过程集成。我们使用全局和局部互信息学文本表示。我们评估我们在几个任务，包括自然语言推理，释义识别和答案选择文本匹配方法。相比于接近先进国家的，实验表明，我们的方法与互信息估计获悉更好的文本表示和实现的文本匹配任务较好的实验结果综合开发利用无训练前对外部数据。
Xixi Zhou, Chengxi Li, Jiajun Bu, Chengwei Yao, Keyue Shi, Zhi Yu, Zhou Yu
Abstract: Text matching is a core natural language processing research problem. How to retain sufficient information on both content and structure information is one important challenge. In this paper, we present a neural approach for general-purpose text matching with deep mutual information estimation incorporated. Our approach, Text matching with Deep Info Max (TIM), is integrated with a procedure of unsupervised learning of representations by maximizing the mutual information between text matching neural network's input and output. We use both global and local mutual information to learn text representations. We evaluate our text matching approach on several tasks including natural language inference, paraphrase identification, and answer selection. Compared to the state-of-the-art approaches, the experiments show that our method integrated with mutual information estimation learns better text representation and achieves better experimental results of text matching tasks without exploiting pretraining on external data.
摘要：文本匹配是一个核心的自然语言处理研究的问题。如何留住在内容和结构信息充分的信息是一个重要的挑战。在本文中，我们提出了以合并深互信息估计通用文本匹配神经的方法。我们的做法，文本深层信息Max（TIM）的匹配，通过最大化文本匹配的神经网络的输入和输出之间的互信息与交涉无监督学习的过程集成。我们使用全局和局部互信息学文本表示。我们评估我们在几个任务，包括自然语言推理，释义识别和答案选择文本匹配方法。相比于接近先进国家的，实验表明，我们的方法与互信息估计获悉更好的文本表示和实现的文本匹配任务较好的实验结果综合开发利用无训练前对外部数据。
7. Joint Multiclass Debiasing of Word Embeddings [PDF] 返回目录
Radomir Popović, Florian Lemmerich, Markus Strohmaier
Abstract: Bias in Word Embeddings has been a subject of recent interest, along with efforts for its reduction. Current approaches show promising progress towards debiasing single bias dimensions such as gender or race. In this paper, we present a joint multiclass debiasing approach that is capable of debiasing multiple bias dimensions simultaneously. In that direction, we present two approaches, HardWEAT and SoftWEAT, that aim to reduce biases by minimizing the scores of the Word Embeddings Association Test (WEAT). We demonstrate the viability of our methods by debiasing Word Embeddings on three classes of biases (religion, gender and race) in three different publicly available word embeddings and show that our concepts can both reduce or even completely eliminate bias, while maintaining meaningful relationships between vectors in word embeddings. Our work strengthens the foundation for more unbiased neural representations of textual data.
摘要：偏差在Word曲面嵌入一直是近期备受关注的话题，与它的减排力度一起。目前的做法显示出有前途的走向消除直流偏压单偏置尺寸，如性别或种族的进步。在本文中，我们提出了一个联合多类消除直流偏压的方法，其能够同时消除直流偏压的多个偏压尺寸。在该方向中，我们提出两种方法，HardWEAT和SoftWEAT，旨在通过最小化字曲面嵌入协会测试（WEAT）的得分，以减少偏差。我们通过在三个不同的公开可用的字的嵌入三类偏见（宗教，性别和种族），并表明我们的概念既可以减少甚至完全消除偏差，同时保持向量之间有意义的关系消除直流偏压的Word曲面嵌入证明我们的方法的可行性在Word中的嵌入。我们的工作加强了对文本数据的更公正的神经表征的基础。
Radomir Popović, Florian Lemmerich, Markus Strohmaier
Abstract: Bias in Word Embeddings has been a subject of recent interest, along with efforts for its reduction. Current approaches show promising progress towards debiasing single bias dimensions such as gender or race. In this paper, we present a joint multiclass debiasing approach that is capable of debiasing multiple bias dimensions simultaneously. In that direction, we present two approaches, HardWEAT and SoftWEAT, that aim to reduce biases by minimizing the scores of the Word Embeddings Association Test (WEAT). We demonstrate the viability of our methods by debiasing Word Embeddings on three classes of biases (religion, gender and race) in three different publicly available word embeddings and show that our concepts can both reduce or even completely eliminate bias, while maintaining meaningful relationships between vectors in word embeddings. Our work strengthens the foundation for more unbiased neural representations of textual data.
摘要：偏差在Word曲面嵌入一直是近期备受关注的话题，与它的减排力度一起。目前的做法显示出有前途的走向消除直流偏压单偏置尺寸，如性别或种族的进步。在本文中，我们提出了一个联合多类消除直流偏压的方法，其能够同时消除直流偏压的多个偏压尺寸。在该方向中，我们提出两种方法，HardWEAT和SoftWEAT，旨在通过最小化字曲面嵌入协会测试（WEAT）的得分，以减少偏差。我们通过在三个不同的公开可用的字的嵌入三类偏见（宗教，性别和种族），并表明我们的概念既可以减少甚至完全消除偏差，同时保持向量之间有意义的关系消除直流偏压的Word曲面嵌入证明我们的方法的可行性在Word中的嵌入。我们的工作加强了对文本数据的更公正的神经表征的基础。
8. Vector logic and counterfactuals [PDF] 返回目录
Eduardo Mizraji
Abstract: In this work we investigate the representation of counterfactual conditionals using the vector logic, a matrix-vectors formalism for logical functions and truth values. With this formalism, we can describe the counterfactuals as complex matrix operators that appear preprocessing the implication matrix with one of the square roots of the negation, a complex matrix. This mathematical approach puts in evidence the virtual character of the counterfactuals. The reason of this fact, is that this representation of a counterfactual proposition produces a valuation that is the superposition the two opposite truth values weighted, respectively, by two complex conjugated coefficients. This result shows that this procedure produces a uncertain evaluation projected on the complex domain. After this basic representation, the judgment of the plausibility of a given counterfactual allows us to shift the decision towards an acceptance or a refusal represented by the real vectors 'true' or 'false', and we can represent symbolically this shift applying for a second time the two square roots of the negation.
摘要：在这项工作中，我们调查使用向量的逻辑，矩阵矢量形式主义为逻辑功能和真值反事实条件的表示。有了这种形式主义，我们可以描述反事实复杂的矩阵运算符出现与平方根否定的一个，一个复杂的矩阵预处理蕴涵矩阵。这种数学方法放入证据反事实的虚拟人物。这一事实的原因，是一个反命题的这种表示产生一个估值是叠加两个相对真值加权，分别由两个复共轭的系数。这一结果表明，此过程会产生不确定的评价投射在复杂领域。这个基本的表示之后，给定的反事实的合理性的判断，使我们的决定对接受或通过真正的载体“真”或“假”表示拒绝转移，我们可以代表象征这一转变在申请第二时间否定的两个平方根。
Eduardo Mizraji
Abstract: In this work we investigate the representation of counterfactual conditionals using the vector logic, a matrix-vectors formalism for logical functions and truth values. With this formalism, we can describe the counterfactuals as complex matrix operators that appear preprocessing the implication matrix with one of the square roots of the negation, a complex matrix. This mathematical approach puts in evidence the virtual character of the counterfactuals. The reason of this fact, is that this representation of a counterfactual proposition produces a valuation that is the superposition the two opposite truth values weighted, respectively, by two complex conjugated coefficients. This result shows that this procedure produces a uncertain evaluation projected on the complex domain. After this basic representation, the judgment of the plausibility of a given counterfactual allows us to shift the decision towards an acceptance or a refusal represented by the real vectors 'true' or 'false', and we can represent symbolically this shift applying for a second time the two square roots of the negation.
摘要：在这项工作中，我们调查使用向量的逻辑，矩阵矢量形式主义为逻辑功能和真值反事实条件的表示。有了这种形式主义，我们可以描述反事实复杂的矩阵运算符出现与平方根否定的一个，一个复杂的矩阵预处理蕴涵矩阵。这种数学方法放入证据反事实的虚拟人物。这一事实的原因，是一个反命题的这种表示产生一个估值是叠加两个相对真值加权，分别由两个复共轭的系数。这一结果表明，此过程会产生不确定的评价投射在复杂领域。这个基本的表示之后，给定的反事实的合理性的判断，使我们的决定对接受或通过真正的载体“真”或“假”表示拒绝转移，我们可以代表象征这一转变在申请第二时间否定的两个平方根。
9. Hybrid Attention-Based Transformer Block Model for Distant Supervision Relation Extraction [PDF] 返回目录
Yan Xiao, Yaochu Jina, Ran Cheng, Kuangrong Hao
Abstract: With an exponential explosive growth of various digital text information, it is challenging to efficiently obtain specific knowledge from massive unstructured text information. As one basic task for natural language processing (NLP), relation extraction aims to extract the semantic relation between entity pairs based on the given text. To avoid manual labeling of datasets, distant supervision relation extraction (DSRE) has been widely used, aiming to utilize knowledge base to automatically annotate datasets. Unfortunately, this method heavily suffers from wrong labelling due to the underlying strong assumptions. To address this issue, we propose a new framework using hybrid attention-based Transformer block with multi-instance learning to perform the DSRE task. More specifically, the Transformer block is firstly used as the sentence encoder to capture syntactic information of sentences, which mainly utilizes multi-head self-attention to extract features from word level. Then, a more concise sentence-level attention mechanism is adopted to constitute the bag representation, aiming to incorporate valid information of each sentence to effectively represent the bag. Experimental results on the public dataset New York Times (NYT) demonstrate that the proposed approach can outperform the state-of-the-art algorithms on the evaluation dataset, which verifies the effectiveness of our model for the DSRE task.
摘要：随着各种数字文本信息的指数爆炸性增长，这是具有挑战性的有效获得海量非结构化文本信息的特定知识。作为自然语言处理（NLP）一个基本任务，关系抽取目标提取基于给定文本实体对之间的语义关系。为了避免数据集的手动贴标签，遥远监督关系抽取（DSRE）已被广泛使用，目的是利用知识库自动注释的数据集。不幸的是，这种方法在很大程度上借鉴贴错标签遭受由于潜在的强有力的假设。为了解决这个问题，我们使用混合的关注，基于变压器块多实例学习执行任务DSRE提出了一个新的框架。更具体地讲，变压器块首先作为句子编码器捕捉到句子的句法信息，这主要是利用多头自注意力从词一级特征提取。然后，更简洁的语句级注意机制采用构成袋表示，旨在整合各句的有效信息，以有效地代表袋。对公共数据集纽约时报（NYT）的实验结果表明，该方法可以超越对评价数据集，从而验证了我们对DSRE任务模型的有效性国家的最先进的算法。
Yan Xiao, Yaochu Jina, Ran Cheng, Kuangrong Hao
Abstract: With an exponential explosive growth of various digital text information, it is challenging to efficiently obtain specific knowledge from massive unstructured text information. As one basic task for natural language processing (NLP), relation extraction aims to extract the semantic relation between entity pairs based on the given text. To avoid manual labeling of datasets, distant supervision relation extraction (DSRE) has been widely used, aiming to utilize knowledge base to automatically annotate datasets. Unfortunately, this method heavily suffers from wrong labelling due to the underlying strong assumptions. To address this issue, we propose a new framework using hybrid attention-based Transformer block with multi-instance learning to perform the DSRE task. More specifically, the Transformer block is firstly used as the sentence encoder to capture syntactic information of sentences, which mainly utilizes multi-head self-attention to extract features from word level. Then, a more concise sentence-level attention mechanism is adopted to constitute the bag representation, aiming to incorporate valid information of each sentence to effectively represent the bag. Experimental results on the public dataset New York Times (NYT) demonstrate that the proposed approach can outperform the state-of-the-art algorithms on the evaluation dataset, which verifies the effectiveness of our model for the DSRE task.
摘要：随着各种数字文本信息的指数爆炸性增长，这是具有挑战性的有效获得海量非结构化文本信息的特定知识。作为自然语言处理（NLP）一个基本任务，关系抽取目标提取基于给定文本实体对之间的语义关系。为了避免数据集的手动贴标签，遥远监督关系抽取（DSRE）已被广泛使用，目的是利用知识库自动注释的数据集。不幸的是，这种方法在很大程度上借鉴贴错标签遭受由于潜在的强有力的假设。为了解决这个问题，我们使用混合的关注，基于变压器块多实例学习执行任务DSRE提出了一个新的框架。更具体地讲，变压器块首先作为句子编码器捕捉到句子的句法信息，这主要是利用多头自注意力从词一级特征提取。然后，更简洁的语句级注意机制采用构成袋表示，旨在整合各句的有效信息，以有效地代表袋。对公共数据集纽约时报（NYT）的实验结果表明，该方法可以超越对评价数据集，从而验证了我们对DSRE任务模型的有效性国家的最先进的算法。
10. From Algebraic Word Problem to Program: A Formalized Approach [PDF] 返回目录
Adam Wiemerslage, Shafiuddin Rehan Ahmed
Abstract: In this paper, we propose a pipeline to convert grade school level algebraic word problem into program of a formal languageA-IMP. Using natural language processing tools, we break the problem into sentence fragments which can then be reduced to functions. The functions are categorized by the head verb of the sentence and its structure, as defined by (Hosseini et al., 2014). We define the function signature and extract its arguments from the text using dependency parsing. We have a working implementation of the entire pipeline which can be found on our github repository.
摘要：在本文中，我们提出了一个管道转换小学水平的代数应用题变成正式languageA-IMP的程序。使用自然语言处理工具，我们把问题分解成，然后可以降低到功能完整的句子。该功能由句子及其结构的头部动词分类，由（胡赛尼等人，2014）所定义的。我们定义函数签名，并使用依赖解析从文本中提取它的参数。我们有整个管道，可以在我们的github存储库中找到的工作的落实。
Adam Wiemerslage, Shafiuddin Rehan Ahmed
Abstract: In this paper, we propose a pipeline to convert grade school level algebraic word problem into program of a formal languageA-IMP. Using natural language processing tools, we break the problem into sentence fragments which can then be reduced to functions. The functions are categorized by the head verb of the sentence and its structure, as defined by (Hosseini et al., 2014). We define the function signature and extract its arguments from the text using dependency parsing. We have a working implementation of the entire pipeline which can be found on our github repository.
摘要：在本文中，我们提出了一个管道转换小学水平的代数应用题变成正式languageA-IMP的程序。使用自然语言处理工具，我们把问题分解成，然后可以降低到功能完整的句子。该功能由句子及其结构的头部动词分类，由（胡赛尼等人，2014）所定义的。我们定义函数签名，并使用依赖解析从文本中提取它的参数。我们有整个管道，可以在我们的github存储库中找到的工作的落实。
11. Keyword-Attentive Deep Semantic Matching [PDF] 返回目录
Changyu Miao, Zhen Cao, Yik-Cheung Tam
Abstract: Deep Semantic Matching is a crucial component in various natural language processing applications such as question and answering (QA), where an input query is compared to each candidate question in a QA corpus in terms of relevance. Measuring similarities between a query-question pair in an open domain scenario can be challenging due to diverse word tokens in the queryquestion pair. We propose a keyword-attentive approach to improve deep semantic matching. We first leverage domain tags from a large corpus to generate a domain-enhanced keyword dictionary. Built upon BERT, we stack a keyword-attentive transformer layer to highlight the importance of keywords in the query-question pair. During model training, we propose a new negative sampling approach based on keyword coverage between the input pair. We evaluate our approach on a Chinese QA corpus using various metrics, including precision of retrieval candidates and accuracy of semantic matching. Experiments show that our approach outperforms existing strong baselines. Our approach is general and can be applied to other text matching tasks with little adaptation.
摘要：深层语义匹配是各种自然语言处理等应用问题和回答（QA），其中输入查询相比，在相关性方面的QA语料库每个候选问题的重要组成部分。在开放域情况测量queryquestion对之间的相似性可以是具有挑战性由于queryquestion对多样字令牌。我们提出了一个关键词，周到的方法来改善深层语义匹配。我们从大量语料第一杠杆域标签生成域增强关键词词典。建立在BERT，我们堆栈中的关键字周到变压器层突出的关键字查询，对问题的重要性。在模型训练中，我们提出了基于输入对之间的关键字覆盖了新的负面抽样方法。我们评估我们使用各种指标，包括检索考生的精度和语义匹配的精度中国QA语料库方法。实验结果表明，我们的方法比现有的强大的基线。我们的方法是一般性的，可以适用于其他的文本匹配任务，有点不适应。
Changyu Miao, Zhen Cao, Yik-Cheung Tam
Abstract: Deep Semantic Matching is a crucial component in various natural language processing applications such as question and answering (QA), where an input query is compared to each candidate question in a QA corpus in terms of relevance. Measuring similarities between a query-question pair in an open domain scenario can be challenging due to diverse word tokens in the queryquestion pair. We propose a keyword-attentive approach to improve deep semantic matching. We first leverage domain tags from a large corpus to generate a domain-enhanced keyword dictionary. Built upon BERT, we stack a keyword-attentive transformer layer to highlight the importance of keywords in the query-question pair. During model training, we propose a new negative sampling approach based on keyword coverage between the input pair. We evaluate our approach on a Chinese QA corpus using various metrics, including precision of retrieval candidates and accuracy of semantic matching. Experiments show that our approach outperforms existing strong baselines. Our approach is general and can be applied to other text matching tasks with little adaptation.
摘要：深层语义匹配是各种自然语言处理等应用问题和回答（QA），其中输入查询相比，在相关性方面的QA语料库每个候选问题的重要组成部分。在开放域情况测量queryquestion对之间的相似性可以是具有挑战性由于queryquestion对多样字令牌。我们提出了一个关键词，周到的方法来改善深层语义匹配。我们从大量语料第一杠杆域标签生成域增强关键词词典。建立在BERT，我们堆栈中的关键字周到变压器层突出的关键字查询，对问题的重要性。在模型训练中，我们提出了基于输入对之间的关键字覆盖了新的负面抽样方法。我们评估我们使用各种指标，包括检索考生的精度和语义匹配的精度中国QA语料库方法。实验结果表明，我们的方法比现有的强大的基线。我们的方法是一般性的，可以适用于其他的文本匹配任务，有点不适应。
12. Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings [PDF] 返回目录
Haoran Zhang, Amy X. Lu, Mohamed Abdalla, Matthew McDermott, Marzyeh Ghassemi
Abstract: In this work, we examine the extent to which embeddings may encode marginalized populations differently, and how this may lead to a perpetuation of biases and worsened performance on clinical tasks. We pretrain deep embedding models (BERT) on medical notes from the MIMIC-III hospital dataset, and quantify potential disparities using two approaches. First, we identify dangerous latent relationships that are captured by the contextual word embeddings using a fill-in-the-blank method with text from real clinical notes and a log probability bias score quantification. Second, we evaluate performance gaps across different definitions of fairness on over 50 downstream clinical prediction tasks that include detection of acute and chronic conditions. We find that classifiers trained from BERT representations exhibit statistically significant differences in performance, often favoring the majority group with regards to gender, language, ethnicity, and insurance status. Finally, we explore shortcomings of using adversarial debiasing to obfuscate subgroup information in contextual word embeddings, and recommend best practices for such deep embedding models in clinical settings.
摘要：在这项工作中，我们考察到的嵌入可以编码不同的边缘化人群的程度，以及如何这可能导致偏见的永久化和恶化的表现临床任务。我们pretrain从模仿-III医院集医疗票据深嵌入模型（BERT），并利用定量两种方法潜在的差距。首先，我们确定通过使用填充式的空白方法与真正的临床笔记文字和数概率偏差分数量化的背景字的嵌入捕捉危险的潜在关系。其次，我们评估不同的公平的定义不同的性能差距对包括急性和慢性疾病的检测超过50个下游临床预测任务。我们发现，从BERT表示训练后的分类表现出的性能统计显著的差异，往往有利于广大组关于性别，语言，种族，以及保险状态。最后，我们探索利用对抗去除偏差在上下文字的嵌入模糊处理子组信息的不足之处，并建议在临床设置，深嵌入模式的最佳实践。
Haoran Zhang, Amy X. Lu, Mohamed Abdalla, Matthew McDermott, Marzyeh Ghassemi
Abstract: In this work, we examine the extent to which embeddings may encode marginalized populations differently, and how this may lead to a perpetuation of biases and worsened performance on clinical tasks. We pretrain deep embedding models (BERT) on medical notes from the MIMIC-III hospital dataset, and quantify potential disparities using two approaches. First, we identify dangerous latent relationships that are captured by the contextual word embeddings using a fill-in-the-blank method with text from real clinical notes and a log probability bias score quantification. Second, we evaluate performance gaps across different definitions of fairness on over 50 downstream clinical prediction tasks that include detection of acute and chronic conditions. We find that classifiers trained from BERT representations exhibit statistically significant differences in performance, often favoring the majority group with regards to gender, language, ethnicity, and insurance status. Finally, we explore shortcomings of using adversarial debiasing to obfuscate subgroup information in contextual word embeddings, and recommend best practices for such deep embedding models in clinical settings.
摘要：在这项工作中，我们考察到的嵌入可以编码不同的边缘化人群的程度，以及如何这可能导致偏见的永久化和恶化的表现临床任务。我们pretrain从模仿-III医院集医疗票据深嵌入模型（BERT），并利用定量两种方法潜在的差距。首先，我们确定通过使用填充式的空白方法与真正的临床笔记文字和数概率偏差分数量化的背景字的嵌入捕捉危险的潜在关系。其次，我们评估不同的公平的定义不同的性能差距对包括急性和慢性疾病的检测超过50个下游临床预测任务。我们发现，从BERT表示训练后的分类表现出的性能统计显著的差异，往往有利于广大组关于性别，语言，种族，以及保险状态。最后，我们探索利用对抗去除偏差在上下文字的嵌入模糊处理子组信息的不足之处，并建议在临床设置，深嵌入模式的最佳实践。
13. BaitWatcher: A lightweight web interface for the detection of incongruent news headlines [PDF] 返回目录
Kunwoo Park, Taegyun Kim, Seunghyun Yoon, Meeyoung Cha, Kyomin Jung
Abstract: In digital environments where substantial amounts of information are shared online, news headlines play essential roles in the selection and diffusion of news articles. Some news articles attract audience attention by showing exaggerated or misleading headlines. This study addresses the \textit{headline incongruity} problem, in which a news headline makes claims that are either unrelated or opposite to the contents of the corresponding article. We present \textit{BaitWatcher}, which is a lightweight web interface that guides readers in estimating the likelihood of incongruence in news articles before clicking on the headlines. BaitWatcher utilizes a hierarchical recurrent encoder that efficiently learns complex textual representations of a news headline and its associated body text. For training the model, we construct a million scale dataset of news articles, which we also release for broader research use. Based on the results of a focus group interview, we discuss the importance of developing an interpretable AI agent for the design of a better interface for mitigating the effects of online misinformation.
摘要：在信息大量在网上共享的数字环境中，新闻标题发挥新闻报道的选择和扩散至关重要的作用。一些新闻文章通过展示夸大或误导性的标题吸引观众的眼球。本研究地址\ textit {标题不协调}问题，在这种新闻标题使得其或者是无关或相对的相应文章的内容的权利要求。我们目前\ textit {} BaitWatcher，这是一个轻量级的Web界面，在点击标题之前估计不一致的新闻文章的可能性引导读者。 BaitWatcher利用分层反复编码器，有效地学习新闻标题及其相关正文的复杂文本表示。对于训练模型，我们构建的新闻文章，我们还推出了更广阔的研究利用一百万规模的数据集。根据焦点小组访谈的结果，我们将讨论开发一个解释AI剂更好的接口的设计减轻网上误传的影响的重要性。
Kunwoo Park, Taegyun Kim, Seunghyun Yoon, Meeyoung Cha, Kyomin Jung
Abstract: In digital environments where substantial amounts of information are shared online, news headlines play essential roles in the selection and diffusion of news articles. Some news articles attract audience attention by showing exaggerated or misleading headlines. This study addresses the \textit{headline incongruity} problem, in which a news headline makes claims that are either unrelated or opposite to the contents of the corresponding article. We present \textit{BaitWatcher}, which is a lightweight web interface that guides readers in estimating the likelihood of incongruence in news articles before clicking on the headlines. BaitWatcher utilizes a hierarchical recurrent encoder that efficiently learns complex textual representations of a news headline and its associated body text. For training the model, we construct a million scale dataset of news articles, which we also release for broader research use. Based on the results of a focus group interview, we discuss the importance of developing an interpretable AI agent for the design of a better interface for mitigating the effects of online misinformation.
摘要：在信息大量在网上共享的数字环境中，新闻标题发挥新闻报道的选择和扩散至关重要的作用。一些新闻文章通过展示夸大或误导性的标题吸引观众的眼球。本研究地址\ textit {标题不协调}问题，在这种新闻标题使得其或者是无关或相对的相应文章的内容的权利要求。我们目前\ textit {} BaitWatcher，这是一个轻量级的Web界面，在点击标题之前估计不一致的新闻文章的可能性引导读者。 BaitWatcher利用分层反复编码器，有效地学习新闻标题及其相关正文的复杂文本表示。对于训练模型，我们构建的新闻文章，我们还推出了更广阔的研究利用一百万规模的数据集。根据焦点小组访谈的结果，我们将讨论开发一个解释AI剂更好的接口的设计减轻网上误传的影响的重要性。
14. Adversarial Multi-Binary Neural Network for Multi-class Classification [PDF] 返回目录
Haiyang Xu, Junwen Chen, Kun Han, Xiangang Li
Abstract: Multi-class text classification is one of the key problems in machine learning and natural language processing. Emerging neural networks deal with the problem using a multi-output softmax layer and achieve substantial progress, but they do not explicitly learn the correlation among classes. In this paper, we use a multi-task framework to address multi-class classification, where a multi-class classifier and multiple binary classifiers are trained together. Moreover, we employ adversarial training to distinguish the class-specific features and the class-agnostic features. The model benefits from better feature representation. We conduct experiments on two large-scale multi-class text classification tasks and demonstrate that the proposed architecture outperforms baseline approaches.
摘要：多类文本分类是机器学习和自然语言处理的关键问题之一。新兴的神经网络处理问题使用多输出SOFTMAX层，实现实质性的进展，但他们并没有明确学习班之间的相关性。在本文中，我们使用了多任务框架来解决多类分类，其中多级分类器和多个二进制分类器一起训练。此外，我们采用对抗性训练区分类特定的功能和类无关的功能。从更好的特征表示模型的好处。我们两个大型的多类文本分类任务进行实验，证明所提出的结构性能优于基准方法。
Haiyang Xu, Junwen Chen, Kun Han, Xiangang Li
Abstract: Multi-class text classification is one of the key problems in machine learning and natural language processing. Emerging neural networks deal with the problem using a multi-output softmax layer and achieve substantial progress, but they do not explicitly learn the correlation among classes. In this paper, we use a multi-task framework to address multi-class classification, where a multi-class classifier and multiple binary classifiers are trained together. Moreover, we employ adversarial training to distinguish the class-specific features and the class-agnostic features. The model benefits from better feature representation. We conduct experiments on two large-scale multi-class text classification tasks and demonstrate that the proposed architecture outperforms baseline approaches.
摘要：多类文本分类是机器学习和自然语言处理的关键问题之一。新兴的神经网络处理问题使用多输出SOFTMAX层，实现实质性的进展，但他们并没有明确学习班之间的相关性。在本文中，我们使用了多任务框架来解决多类分类，其中多级分类器和多个二进制分类器一起训练。此外，我们采用对抗性训练区分类特定的功能和类无关的功能。从更好的特征表示模型的好处。我们两个大型的多类文本分类任务进行实验，证明所提出的结构性能优于基准方法。
15. Learning Syntactic and Dynamic Selective Encoding for Document Summarization [PDF] 返回目录
Haiyang Xu, Yahao He, Kun Han, Junwen Chen, Xiangang Li
Abstract: Text summarization aims to generate a headline or a short summary consisting of the major information of the source text. Recent studies employ the sequence-to-sequence framework to encode the input with a neural network and generate abstractive summary. However, most studies feed the encoder with the semantic word embedding but ignore the syntactic information of the text. Further, although previous studies proposed the selective gate to control the information flow from the encoder to the decoder, it is static during the decoding and cannot differentiate the information based on the decoder states. In this paper, we propose a novel neural architecture for document summarization. Our approach has the following contributions: first, we incorporate syntactic information such as constituency parsing trees into the encoding sequence to learn both the semantic and syntactic information from the document, resulting in more accurate summary; second, we propose a dynamic gate network to select the salient information based on the context of the decoder state, which is essential to document summarization. The proposed model has been evaluated on CNN/Daily Mail summarization datasets and the experimental results show that the proposed approach outperforms baseline approaches.
摘要：文本摘要的目的，产生一个标题或由原文的主要信息的简短摘要。最近的研究采用序列到序列框架编码与神经网络的输入值，生成抽象概括。然而，大多数研究饲料与语义字嵌入编码器，但忽略文本的句法信息。此外，虽然以前的研究中提出的选择性栅极，以控制从编码器到解码器的信息流，它是在解码期间静态的，不能区分基于所述解码器状态的信息。在本文中，我们提出了文档文摘一种新的神经结构。我们的方法有以下贡献：第一，我们结合语法信息，如选区解析树到编码序列学习从文档都语义和句法信息，从而更准确的概括;第二，我们提出一种动态栅极网络来选择基于解码器的状态，这是文档文摘必需的上下文中的显着信息。该模型已被评估在CNN /每日邮报汇总数据集，实验结果表明，该方法比基线方法。
Haiyang Xu, Yahao He, Kun Han, Junwen Chen, Xiangang Li
Abstract: Text summarization aims to generate a headline or a short summary consisting of the major information of the source text. Recent studies employ the sequence-to-sequence framework to encode the input with a neural network and generate abstractive summary. However, most studies feed the encoder with the semantic word embedding but ignore the syntactic information of the text. Further, although previous studies proposed the selective gate to control the information flow from the encoder to the decoder, it is static during the decoding and cannot differentiate the information based on the decoder states. In this paper, we propose a novel neural architecture for document summarization. Our approach has the following contributions: first, we incorporate syntactic information such as constituency parsing trees into the encoding sequence to learn both the semantic and syntactic information from the document, resulting in more accurate summary; second, we propose a dynamic gate network to select the salient information based on the context of the decoder state, which is essential to document summarization. The proposed model has been evaluated on CNN/Daily Mail summarization datasets and the experimental results show that the proposed approach outperforms baseline approaches.
摘要：文本摘要的目的，产生一个标题或由原文的主要信息的简短摘要。最近的研究采用序列到序列框架编码与神经网络的输入值，生成抽象概括。然而，大多数研究饲料与语义字嵌入编码器，但忽略文本的句法信息。此外，虽然以前的研究中提出的选择性栅极，以控制从编码器到解码器的信息流，它是在解码期间静态的，不能区分基于所述解码器状态的信息。在本文中，我们提出了文档文摘一种新的神经结构。我们的方法有以下贡献：第一，我们结合语法信息，如选区解析树到编码序列学习从文档都语义和句法信息，从而更准确的概括;第二，我们提出一种动态栅极网络来选择基于解码器的状态，这是文档文摘必需的上下文中的显着信息。该模型已被评估在CNN /每日邮报汇总数据集，实验结果表明，该方法比基线方法。
16. Can Embeddings Adequately Represent Medical Terminology? New Large-Scale Medical Term Similarity Datasets Have the Answer! [PDF] 返回目录
Claudia Schulz, Damir Juric
Abstract: A large number of embeddings trained on medical data have emerged, but it remains unclear how well they represent medical terminology, in particular whether the close relationship of semantically similar medical terms is encoded in these embeddings. To date, only small datasets for testing medical term similarity are available, not allowing to draw conclusions about the generalisability of embeddings to the enormous amount of medical terms used by doctors. We present multiple automatically created large-scale medical term similarity datasets and confirm their high quality in an annotation study with doctors. We evaluate state-of-the-art word and contextual embeddings on our new datasets, comparing multiple vector similarity metrics and word vector aggregation techniques. Our results show that current embeddings are limited in their ability to adequately encode medical terms. The novel datasets thus form a challenging new benchmark for the development of medical embeddings able to accurately represent the whole medical terminology.
摘要：大量训练有素的医疗数据的嵌入的纷纷涌现，但他们如何代表医学术语，特别是语义相似的医学术语的密切关系是否在这些的嵌入编码仍不清楚。迄今为止，测试医学术语相似只有很小的数据集可用，不允许得出关于的嵌入的普适性由医生使用医学术语数额巨大的结论。我们设置了多重自动创建大型医学术语相似的数据集，并确认与医生注释研究他们的高品质。我们评估的国家的最先进的字和我们的新的数据集的上下文的嵌入，比较多个向量的相似度量和词汇向量聚合技术。我们的研究结果表明，目前的嵌入在自己的能力有限，以充分编码的医学术语。因此，该新颖的数据集形成用于能够精确地表示整个医疗术语的嵌入医疗的发展一个具有挑战性的新的基准。
Claudia Schulz, Damir Juric
Abstract: A large number of embeddings trained on medical data have emerged, but it remains unclear how well they represent medical terminology, in particular whether the close relationship of semantically similar medical terms is encoded in these embeddings. To date, only small datasets for testing medical term similarity are available, not allowing to draw conclusions about the generalisability of embeddings to the enormous amount of medical terms used by doctors. We present multiple automatically created large-scale medical term similarity datasets and confirm their high quality in an annotation study with doctors. We evaluate state-of-the-art word and contextual embeddings on our new datasets, comparing multiple vector similarity metrics and word vector aggregation techniques. Our results show that current embeddings are limited in their ability to adequately encode medical terms. The novel datasets thus form a challenging new benchmark for the development of medical embeddings able to accurately represent the whole medical terminology.
摘要：大量训练有素的医疗数据的嵌入的纷纷涌现，但他们如何代表医学术语，特别是语义相似的医学术语的密切关系是否在这些的嵌入编码仍不清楚。迄今为止，测试医学术语相似只有很小的数据集可用，不允许得出关于的嵌入的普适性由医生使用医学术语数额巨大的结论。我们设置了多重自动创建大型医学术语相似的数据集，并确认与医生注释研究他们的高品质。我们评估的国家的最先进的字和我们的新的数据集的上下文的嵌入，比较多个向量的相似度量和词汇向量聚合技术。我们的研究结果表明，目前的嵌入在自己的能力有限，以充分编码的医学术语。因此，该新颖的数据集形成用于能够精确地表示整个医疗术语的嵌入医疗的发展一个具有挑战性的新的基准。
17. XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization [PDF] 返回目录
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, Melvin Johnson
Abstract: Much recent progress in applications of machine learning models to NLP has been driven by benchmarks that evaluate models across a wide variety of tasks. However, these broad-coverage benchmarks have been mostly limited to English, and despite an increasing interest in multilingual models, a benchmark that enables the comprehensive evaluation of such methods on a diverse range of languages and tasks is still missing. To this end, we introduce the Cross-lingual TRansfer Evaluation of Multilingual Encoders XTREME benchmark, a multi-task benchmark for evaluating the cross-lingual generalization capabilities of multilingual representations across 40 languages and 9 tasks. We demonstrate that while models tested on English reach human performance on many tasks, there is still a sizable gap in the performance of cross-lingually transferred models, particularly on syntactic and sentence retrieval tasks. There is also a wide spread of results across languages. We release the benchmark to encourage research on cross-lingual learning methods that transfer linguistic knowledge across a diverse and representative set of languages and tasks.
摘要：最近的许多机器学习模型NLP应用的进程已经通过在各种不同任务的评估模型的基准驱动。然而，这些广泛的覆盖面基准已大多仅限于英语，尽管多语言模型的兴趣越来越大，一个标杆，使这种方法的综合评价上的各种不同的语言和任务，至今下落不明。为此，我们推出多语言编码器XTREME标杆，跨40种语言和9个任务评估多语言表述的跨语言概括能力的多任务基准的跨语言转移评价。我们表明，虽然型号上英语达到人类的性能测试的许多任务，还有跨舌转移模型的性能相当大的差距，特别是在语法和句子检索任务。还有一个宽跨语言成绩蔓延。我们发布的基准，以鼓励跨多元化和代表性的语言和任务转移语言知识跨语言学习方法的研究。
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, Melvin Johnson
Abstract: Much recent progress in applications of machine learning models to NLP has been driven by benchmarks that evaluate models across a wide variety of tasks. However, these broad-coverage benchmarks have been mostly limited to English, and despite an increasing interest in multilingual models, a benchmark that enables the comprehensive evaluation of such methods on a diverse range of languages and tasks is still missing. To this end, we introduce the Cross-lingual TRansfer Evaluation of Multilingual Encoders XTREME benchmark, a multi-task benchmark for evaluating the cross-lingual generalization capabilities of multilingual representations across 40 languages and 9 tasks. We demonstrate that while models tested on English reach human performance on many tasks, there is still a sizable gap in the performance of cross-lingually transferred models, particularly on syntactic and sentence retrieval tasks. There is also a wide spread of results across languages. We release the benchmark to encourage research on cross-lingual learning methods that transfer linguistic knowledge across a diverse and representative set of languages and tasks.
摘要：最近的许多机器学习模型NLP应用的进程已经通过在各种不同任务的评估模型的基准驱动。然而，这些广泛的覆盖面基准已大多仅限于英语，尽管多语言模型的兴趣越来越大，一个标杆，使这种方法的综合评价上的各种不同的语言和任务，至今下落不明。为此，我们推出多语言编码器XTREME标杆，跨40种语言和9个任务评估多语言表述的跨语言概括能力的多任务基准的跨语言转移评价。我们表明，虽然型号上英语达到人类的性能测试的许多任务，还有跨舌转移模型的性能相当大的差距，特别是在语法和句子检索任务。还有一个宽跨语言成绩蔓延。我们发布的基准，以鼓励跨多元化和代表性的语言和任务转移语言知识跨语言学习方法的研究。
18. Utilizing Deep Learning to Identify Drug Use on Twitter Data [PDF] 返回目录
Joseph Tassone, Peizhi Yan, Mackenzie Simpson, Chetan Mendhe, Vijay Mago, Salimur Choudhury
Abstract: The collection and examination of social media has become a useful mechanism for studying the mental activity and behavior tendencies of users. Through the analysis of collected Twitter data, models were developed for classifying drug-related tweets. Using topic pertaining keywords, such as slang and methods of drug consumption, a set of tweets was generated. Potential candidates were then preprocessed resulting in a dataset of 3,696,150 rows. The classification power of multiple methods was compared including support vector machines (SVM), XGBoost, and convolutional neural network (CNN) based classifiers. Rather than simple feature or attribute analysis, a deep learning approach was implemented to screen and analyze the tweets' semantic meaning. The two CNN-based classifiers presented the best result when compared against other methodologies. The first was trained with 2,661 manually labeled samples, while the other included synthetically generated tweets culminating in 12,142 samples. The accuracy scores were 76.35% and 82.31%, with an AUC of 0.90 and 0.91. Additionally, association rule mining showed that commonly mentioned drugs had a level of correspondence with frequently used illicit substances, proving the practical usefulness of the system. Lastly, the synthetically generated set provided increased scores, improving the classification capability and proving the worth of this methodology.
摘要：收集和社交媒体的审查已经成为研究用户的心理活动和行为倾向的有效机制。通过收集Twitter的数据进行分析，模型的分类与毒品有关的tweet开发。使用有关话题的关键词，如俚语和药品消费的方法，产生一组鸣叫。然后，潜在的候选人进行预处理导致3696150行的数据集。的多种方法的分类功率进行比较包括支持向量机（SVM），XGBoost和卷积神经网络（CNN）基于分类器。而不是简单的功能或属性的分析，深刻的学习方法的实施是为了屏幕和分析的tweet语义。当与其他方法相比，两种基于CNN-分类呈现最好的结果。第一用2661个手动标记的样品的训练，而另一个包括合成产生的鸣叫在12142个最终样品。精度得分分别为76.35％和82.31％，为0.90和0.91的AUC。此外，关联规则挖掘发现，常提到的药物有经常使用的非法物质的对应程度，证明了系统的实用性。最后，综合生成的一组提供了更多的分数，提高分类能力，并证明了这种方法的价值。
Joseph Tassone, Peizhi Yan, Mackenzie Simpson, Chetan Mendhe, Vijay Mago, Salimur Choudhury
Abstract: The collection and examination of social media has become a useful mechanism for studying the mental activity and behavior tendencies of users. Through the analysis of collected Twitter data, models were developed for classifying drug-related tweets. Using topic pertaining keywords, such as slang and methods of drug consumption, a set of tweets was generated. Potential candidates were then preprocessed resulting in a dataset of 3,696,150 rows. The classification power of multiple methods was compared including support vector machines (SVM), XGBoost, and convolutional neural network (CNN) based classifiers. Rather than simple feature or attribute analysis, a deep learning approach was implemented to screen and analyze the tweets' semantic meaning. The two CNN-based classifiers presented the best result when compared against other methodologies. The first was trained with 2,661 manually labeled samples, while the other included synthetically generated tweets culminating in 12,142 samples. The accuracy scores were 76.35% and 82.31%, with an AUC of 0.90 and 0.91. Additionally, association rule mining showed that commonly mentioned drugs had a level of correspondence with frequently used illicit substances, proving the practical usefulness of the system. Lastly, the synthetically generated set provided increased scores, improving the classification capability and proving the worth of this methodology.
摘要：收集和社交媒体的审查已经成为研究用户的心理活动和行为倾向的有效机制。通过收集Twitter的数据进行分析，模型的分类与毒品有关的tweet开发。使用有关话题的关键词，如俚语和药品消费的方法，产生一组鸣叫。然后，潜在的候选人进行预处理导致3696150行的数据集。的多种方法的分类功率进行比较包括支持向量机（SVM），XGBoost和卷积神经网络（CNN）基于分类器。而不是简单的功能或属性的分析，深刻的学习方法的实施是为了屏幕和分析的tweet语义。当与其他方法相比，两种基于CNN-分类呈现最好的结果。第一用2661个手动标记的样品的训练，而另一个包括合成产生的鸣叫在12142个最终样品。精度得分分别为76.35％和82.31％，为0.90和0.91的AUC。此外，关联规则挖掘发现，常提到的药物有经常使用的非法物质的对应程度，证明了系统的实用性。最后，综合生成的一组提供了更多的分数，提高分类能力，并证明了这种方法的价值。
19. COVID-19 and Computer Audition: An Overview on What Speech & Sound Analysis Could Contribute in the SARS-CoV-2 Corona Crisis [PDF] 返回目录
Björn W. Schuller, Dagmar M. Schuller, Kun Qian, Juan Liu, Huaiyuan Zheng, Xiao Li
Abstract: At the time of writing, the world population is suffering from more than 10,000 registered COVID-19 disease epidemic induced deaths since the outbreak of the Corona virus more than three months ago now officially known as SARS-CoV-2. Since, tremendous efforts have been made worldwide to counter-steer and control the epidemic by now labelled as pandemic. In this contribution, we provide an overview on the potential for computer audition (CA), i.e., the usage of speech and sound analysis by artificial intelligence to help in this scenario. We first survey which types of related or contextually significant phenomena can be automatically assessed from speech or sound. These include the automatic recognition and monitoring of breathing, dry and wet coughing or sneezing sounds, speech under cold, eating behaviour, sleepiness, or pain to name but a few. Then, we consider potential use-cases for exploitation. These include risk assessment and diagnosis based on symptom histograms and their development over time, as well as monitoring of spread, social distancing and its effects, treatment and recovery, and patient wellbeing. We quickly guide further through challenges that need to be faced for real-life usage. We come to the conclusion that CA appears ready for implementation of (pre-)diagnosis and monitoring tools, and more generally provides rich and significant, yet so far untapped potential in the fight against COVID-19 spread.
摘要：在写这篇文章的时候，世界人口从超过10,000注册COVID-19病疫情致人死亡以来，冠状病毒超过三个月前爆发现在正式被称为SARS-COV-2痛苦。因为，巨大的努力，在全球已作出反转向和控制疫情现在标为流行。在这方面的贡献，我们提供有关计算机试听（CA），即语音和声音分析的人工智能在这种情况下使用，以帮助潜在的概述。我们第一次调查哪些类型的相关或内容显著的现象可以从语音或声音来自动评估。这些措施包括自动识别，并在寒冷的监测呼吸，干，湿咳嗽或打喷嚏的声音，讲话，进食行为，嗜睡，或疼痛仅举几例。然后，我们考虑潜在的使用情况进行开采。这些措施包括风险评估，并根据症状直方图及其随时间的发展诊断，以及监测蔓延，社会距离和它的作用，治疗和恢复，以及病人的健康。我们快速通过需要面对现实生活中使用的挑战进一步引导。我们得出的结论是，CA似乎准备实施（预）的诊断和监控工具，一般多提供了丰富和显著，但至今尚未开发的潜力在打击COVID-19蔓延的斗争。
Björn W. Schuller, Dagmar M. Schuller, Kun Qian, Juan Liu, Huaiyuan Zheng, Xiao Li
Abstract: At the time of writing, the world population is suffering from more than 10,000 registered COVID-19 disease epidemic induced deaths since the outbreak of the Corona virus more than three months ago now officially known as SARS-CoV-2. Since, tremendous efforts have been made worldwide to counter-steer and control the epidemic by now labelled as pandemic. In this contribution, we provide an overview on the potential for computer audition (CA), i.e., the usage of speech and sound analysis by artificial intelligence to help in this scenario. We first survey which types of related or contextually significant phenomena can be automatically assessed from speech or sound. These include the automatic recognition and monitoring of breathing, dry and wet coughing or sneezing sounds, speech under cold, eating behaviour, sleepiness, or pain to name but a few. Then, we consider potential use-cases for exploitation. These include risk assessment and diagnosis based on symptom histograms and their development over time, as well as monitoring of spread, social distancing and its effects, treatment and recovery, and patient wellbeing. We quickly guide further through challenges that need to be faced for real-life usage. We come to the conclusion that CA appears ready for implementation of (pre-)diagnosis and monitoring tools, and more generally provides rich and significant, yet so far untapped potential in the fight against COVID-19 spread.
摘要：在写这篇文章的时候，世界人口从超过10,000注册COVID-19病疫情致人死亡以来，冠状病毒超过三个月前爆发现在正式被称为SARS-COV-2痛苦。因为，巨大的努力，在全球已作出反转向和控制疫情现在标为流行。在这方面的贡献，我们提供有关计算机试听（CA），即语音和声音分析的人工智能在这种情况下使用，以帮助潜在的概述。我们第一次调查哪些类型的相关或内容显著的现象可以从语音或声音来自动评估。这些措施包括自动识别，并在寒冷的监测呼吸，干，湿咳嗽或打喷嚏的声音，讲话，进食行为，嗜睡，或疼痛仅举几例。然后，我们考虑潜在的使用情况进行开采。这些措施包括风险评估，并根据症状直方图及其随时间的发展诊断，以及监测蔓延，社会距离和它的作用，治疗和恢复，以及病人的健康。我们快速通过需要面对现实生活中使用的挑战进一步引导。我们得出的结论是，CA似乎准备实施（预）的诊断和监控工具，一般多提供了丰富和显著，但至今尚未开发的潜力在打击COVID-19蔓延的斗争。
20. EQL -- an extremely easy to learn knowledge graph query language, achieving highspeed and precise search [PDF] 返回目录
Han Liu, Shantao Liu
Abstract: EQL, also named as Extremely Simple Query Language, can be widely used in the field of knowledge graph, precise search, strong artificial intelligence, database, smart speaker ,patent search and other fields. EQL adopt the principle of minimalism in design and pursues simplicity and easy to learn so that everyone can master it quickly. EQL language and lambda calculus are interconvertible, that reveals the mathematical nature of EQL language, and lays a solid foundation for rigor and logical integrity of EQL language. The EQL language and a comprehensive knowledge graph system with the world's commonsense can together form the foundation of strong AI in the future, and make up for the current lack of understanding of world's commonsense by current AI system. EQL language can be used not only by humans, but also as a basic language for data query and data exchange between robots.
摘要：EQL，又称极其简单查询语言，可广泛应用于知识图，精准搜索，强大的人工智能，数据库，智能音箱，专利检索等领域的领域。 EQL采用极简主义的原则，在设计和追求简单，易学，使每个人都可以快速掌握它。 EQL语言和演算是可以相互转换，揭示EQL语言的数学本质，并奠定了严谨和EQL语言的逻辑完整性一个坚实的基础。该EQL语言和世界常识的综合知识图系统可以共同形成强大的AI，在未来的基础，弥补目前缺乏由目前的AI系统了解世界的常识的。 EQL语言可以用来不仅人类，也可作为数据查询和机器人之间数据交换的基本语言。
Han Liu, Shantao Liu
Abstract: EQL, also named as Extremely Simple Query Language, can be widely used in the field of knowledge graph, precise search, strong artificial intelligence, database, smart speaker ,patent search and other fields. EQL adopt the principle of minimalism in design and pursues simplicity and easy to learn so that everyone can master it quickly. EQL language and lambda calculus are interconvertible, that reveals the mathematical nature of EQL language, and lays a solid foundation for rigor and logical integrity of EQL language. The EQL language and a comprehensive knowledge graph system with the world's commonsense can together form the foundation of strong AI in the future, and make up for the current lack of understanding of world's commonsense by current AI system. EQL language can be used not only by humans, but also as a basic language for data query and data exchange between robots.
摘要：EQL，又称极其简单查询语言，可广泛应用于知识图，精准搜索，强大的人工智能，数据库，智能音箱，专利检索等领域的领域。 EQL采用极简主义的原则，在设计和追求简单，易学，使每个人都可以快速掌握它。 EQL语言和演算是可以相互转换，揭示EQL语言的数学本质，并奠定了严谨和EQL语言的逻辑完整性一个坚实的基础。该EQL语言和世界常识的综合知识图系统可以共同形成强大的AI，在未来的基础，弥补目前缺乏由目前的AI系统了解世界的常识的。 EQL语言可以用来不仅人类，也可作为数据查询和机器人之间数据交换的基本语言。
注：中文为机器翻译结果！