目录
3. Can you tell? SSNet -- a Sagittal Stratum-inspired Neural Network Framework for Sentiment Analysis [PDF] 摘要
4. NLPContributions: An Annotation Scheme for Machine Reading of Scholarly Contributions in Natural Language Processing Literature [PDF] 摘要
6. Keyframe Segmentation and Positional Encoding for Video-guided Machine Translation Challenge 2020 [PDF] 摘要
摘要
1. Combining Neural Language Models for WordSense Induction [PDF] 返回目录
Nikolay Arefyev, Boris Sheludko, Tatiana Aleksashina
Abstract: Word sense induction (WSI) is the problem of grouping occurrences of an ambiguous word according to the expressed sense of this word. Recently a new approach to this task was proposed, which generates possible substitutes for the ambiguous word in a particular context using neural language models, and then clusters sparse bag-of-words vectors built from these substitutes. In this work, we apply this approach to the Russian language and improve it in two ways. First, we propose methods of combining left and right contexts, resulting in better substitutes generated. Second, instead of fixed number of clusters for all ambiguous words we propose a technique for selecting individual number of clusters for each word. Our approach established new state-of-the-art level, improving current best results of WSI for the Russian language on two RUSSE 2018 datasets by a large margin.
摘要：词义归纳（WSI）是根据这个词所表达的意义分组歧义词出现的问题。最近提出了这一任务的新方法，利用神经语言模型产生了歧义词可能的替代品在特定环境下，然后从聚类这些替代内置稀疏袋的词向量。在这项工作中，我们将这种方法用于俄罗斯语言和提高它在两个方面。首先，我们建议结合左右环境中，导致产生更好的替代方法。其次，代替固定数量的集群为所有暧昧的话，我们建议选择集群的个体数为每个单词的技术。我们的方法建立的国家的最先进的新水平，大幅度提高了两个RUSSE 2018个数据集WSI俄罗斯语言目前最好的结果。
Nikolay Arefyev, Boris Sheludko, Tatiana Aleksashina
Abstract: Word sense induction (WSI) is the problem of grouping occurrences of an ambiguous word according to the expressed sense of this word. Recently a new approach to this task was proposed, which generates possible substitutes for the ambiguous word in a particular context using neural language models, and then clusters sparse bag-of-words vectors built from these substitutes. In this work, we apply this approach to the Russian language and improve it in two ways. First, we propose methods of combining left and right contexts, resulting in better substitutes generated. Second, instead of fixed number of clusters for all ambiguous words we propose a technique for selecting individual number of clusters for each word. Our approach established new state-of-the-art level, improving current best results of WSI for the Russian language on two RUSSE 2018 datasets by a large margin.
摘要：词义归纳（WSI）是根据这个词所表达的意义分组歧义词出现的问题。最近提出了这一任务的新方法，利用神经语言模型产生了歧义词可能的替代品在特定环境下，然后从聚类这些替代内置稀疏袋的词向量。在这项工作中，我们将这种方法用于俄罗斯语言和提高它在两个方面。首先，我们建议结合左右环境中，导致产生更好的替代方法。其次，代替固定数量的集群为所有暧昧的话，我们建议选择集群的个体数为每个单词的技术。我们的方法建立的国家的最先进的新水平，大幅度提高了两个RUSSE 2018个数据集WSI俄罗斯语言目前最好的结果。
2. Domain Adaptation for Semantic Parsing [PDF] 返回目录
Zechang Li, Yuxuan Lai, Yansong Feng, Dongyan Zhao
Abstract: Recently, semantic parsing has attracted much attention in the community. Although many neural modeling efforts have greatly improved the performance, it still suffers from the data scarcity issue. In this paper, we propose a novel semantic parser for domain adaptation, where we have much fewer annotated data in the target domain compared to the source domain. Our semantic parser benefits from a two-stage coarse-to-fine framework, thus can provide different and accurate treatments for the two stages, i.e., focusing on domain invariant and domain specific information, respectively. In the coarse stage, our novel domain discrimination component and domain relevance attention encourage the model to learn transferable domain general structures. In the fine stage, the model is guided to concentrate on domain related details. Experiments on a benchmark dataset show that our method consistently outperforms several popular domain adaptation strategies. Additionally, we show that our model can well exploit limited target data to capture the difference between the source and target domain, even when the target domain has far fewer training instances.
摘要：近日，语义分析备受关注的社区。虽然许多神经建模工作极大地提高了性能，它仍然从数据匮乏的问题受到影响。在本文中，我们提出了一个新的语义解析域调整，其中我们比较源域在目标域少得多注释的数据。从两阶段粗到细的框架提供了语义解析器好处，从而可以分别用于两个阶段提供不同和准确的治疗，即，着眼于域不变和域特定信息。在粗阶段，我们的新领域歧视成分和域相关性的重视鼓励模型学习转让域名的一般结构。在精细阶段，该模型被引导专心域相关的详细信息。在基准数据集显示，我们的方法始终优于几种流行的域名适应战略的实验。此外，我们表明，我们的模型可以很好地利用有限的目标数据采集源和目标域之间的差异，即使在目标域有少得多的训练实例。
Zechang Li, Yuxuan Lai, Yansong Feng, Dongyan Zhao
Abstract: Recently, semantic parsing has attracted much attention in the community. Although many neural modeling efforts have greatly improved the performance, it still suffers from the data scarcity issue. In this paper, we propose a novel semantic parser for domain adaptation, where we have much fewer annotated data in the target domain compared to the source domain. Our semantic parser benefits from a two-stage coarse-to-fine framework, thus can provide different and accurate treatments for the two stages, i.e., focusing on domain invariant and domain specific information, respectively. In the coarse stage, our novel domain discrimination component and domain relevance attention encourage the model to learn transferable domain general structures. In the fine stage, the model is guided to concentrate on domain related details. Experiments on a benchmark dataset show that our method consistently outperforms several popular domain adaptation strategies. Additionally, we show that our model can well exploit limited target data to capture the difference between the source and target domain, even when the target domain has far fewer training instances.
摘要：近日，语义分析备受关注的社区。虽然许多神经建模工作极大地提高了性能，它仍然从数据匮乏的问题受到影响。在本文中，我们提出了一个新的语义解析域调整，其中我们比较源域在目标域少得多注释的数据。从两阶段粗到细的框架提供了语义解析器好处，从而可以分别用于两个阶段提供不同和准确的治疗，即，着眼于域不变和域特定信息。在粗阶段，我们的新领域歧视成分和域相关性的重视鼓励模型学习转让域名的一般结构。在精细阶段，该模型被引导专心域相关的详细信息。在基准数据集显示，我们的方法始终优于几种流行的域名适应战略的实验。此外，我们表明，我们的模型可以很好地利用有限的目标数据采集源和目标域之间的差异，即使在目标域有少得多的训练实例。
3. Can you tell? SSNet -- a Sagittal Stratum-inspired Neural Network Framework for Sentiment Analysis [PDF] 返回目录
Apostol Vassilev, Munawar Hasan
Abstract: When people try to understand nuanced language they typically process multiple input sensor modalities to complete this cognitive task. It turns out the human brain has even a specialized neuron formation, called sagittal stratum, to help us understand sarcasm. We use this biological formation as the inspiration for designing a neural network architecture that combines predictions of different models on the same text to construct a robust, accurate and computationally efficient classifier for sentiment analysis. Experimental results on representative benchmark datasets and comparisons to other methods1show the advantages of the new network architecture.
摘要：当人们试着去理解细致入微的语言，但通常处理多个输入传感器方式来完成这个认知任务。原来，人的大脑有甚至专门的神经元的形成，叫矢状阶层，来帮助我们理解讽刺。我们用这个生物的形成为灵感设计的神经网络结构上相同的文字不同机型的联合预测，以构建情感分析一个可靠，精确和高效计算的分类。有代表性的基准数据集和比较实验结果到其他methods1show新的网络架构的优势。
Apostol Vassilev, Munawar Hasan
Abstract: When people try to understand nuanced language they typically process multiple input sensor modalities to complete this cognitive task. It turns out the human brain has even a specialized neuron formation, called sagittal stratum, to help us understand sarcasm. We use this biological formation as the inspiration for designing a neural network architecture that combines predictions of different models on the same text to construct a robust, accurate and computationally efficient classifier for sentiment analysis. Experimental results on representative benchmark datasets and comparisons to other methods1show the advantages of the new network architecture.
摘要：当人们试着去理解细致入微的语言，但通常处理多个输入传感器方式来完成这个认知任务。原来，人的大脑有甚至专门的神经元的形成，叫矢状阶层，来帮助我们理解讽刺。我们用这个生物的形成为灵感设计的神经网络结构上相同的文字不同机型的联合预测，以构建情感分析一个可靠，精确和高效计算的分类。有代表性的基准数据集和比较实验结果到其他methods1show新的网络架构的优势。
4. NLPContributions: An Annotation Scheme for Machine Reading of Scholarly Contributions in Natural Language Processing Literature [PDF] 返回目录
Jennifer D'Souza, Sören Auer
Abstract: We describe an annotation initiative to capture the scholarly contributions in natural language processing (NLP) articles, particularly, for the articles that discuss machine learning (ML) approaches for various information extraction tasks. We develop the annotation task based on a pilot annotation exercise on 50 NLP-ML scholarly articles presenting contributions to the five information extraction tasks 1. machine translation, 2. named entity recognition, 3. question answering, 4. relation classification, and 5. text classification. In this article, we describe the outcomes of this pilot annotation phase. Through the exercise we have obtained an annotation methodology; and found eight core information units that reflect the contribution of the NLP-ML scholarly investigations. The resulting annotation scheme we developed based on these information units is called NLPContributions. The overarching goal of our endeavor is: 1) to find a systematic set of patterns of subject-predicate-object statements for the semantic structuring of scholarly contributions that are more or less generically applicable for NLP-ML research articles; 2) to apply the discovered patterns in the creation of a larger annotated dataset for training machine readers of research contributions; 3) to ingest the dataset into the Open Research Knowledge Graph (ORKG) infrastructure as a showcase for creating user-friendly state-of-the-art overviews; 4) to integrate the machine readers into the ORKG to assist users in the manual curation of their respective article contributions. We envision that the NLPContributions methodology engenders a wider discussion on the topic toward its further refinement and development. Our pilot annotated dataset of 50 NLP-ML scholarly articles per the NLPContributions scheme is available at this https URL.
摘要：我们描述一个注释主动捕捉自然语言处理（NLP）的文章，对学术贡献特别，对于讨论机器学习（ML）方法对各种信息提取的任务物品。我们开发了基于50 NLP-ML学术文章提出五个信息提取的任务1.机器翻译2.命名实体识别，3答疑，4关系分类，5贡献试点注释锻炼注解任务。文本分类。在这篇文章中，我们描述了这个试点注释阶段的成果。通过锻炼，我们已经获得了注释的方法;并且发现，反映了NLP-ML学术研究的贡献八大核心信息单元。我们开发了基于这些信息单元产生的注解方案被称为NLPContributions。我们努力的总体目标是：1）找到一个系统的一套那或多或少一般适用于NLP-ML研究论文学术贡献的语义结构主谓对象语句的模式; 2）应用在创造训练的研究贡献机器的读者一个更大的数据集注释的发现的模式; 3）摄取数据集到开放研究知识图（ORKG）基础设施作为用于创建用户友好的状态的最先进的概述的展示; 4）机器读者融入ORKG帮助用户在各自的文章贡献的人工管理。我们设想，在NLPContributions方法论上滋生朝其进一步改进和发展议题的更广泛的讨论。我们的飞行员注释50 NLP-ML学术每NLPContributions方案的文章数据集可在此HTTPS URL。
Jennifer D'Souza, Sören Auer
Abstract: We describe an annotation initiative to capture the scholarly contributions in natural language processing (NLP) articles, particularly, for the articles that discuss machine learning (ML) approaches for various information extraction tasks. We develop the annotation task based on a pilot annotation exercise on 50 NLP-ML scholarly articles presenting contributions to the five information extraction tasks 1. machine translation, 2. named entity recognition, 3. question answering, 4. relation classification, and 5. text classification. In this article, we describe the outcomes of this pilot annotation phase. Through the exercise we have obtained an annotation methodology; and found eight core information units that reflect the contribution of the NLP-ML scholarly investigations. The resulting annotation scheme we developed based on these information units is called NLPContributions. The overarching goal of our endeavor is: 1) to find a systematic set of patterns of subject-predicate-object statements for the semantic structuring of scholarly contributions that are more or less generically applicable for NLP-ML research articles; 2) to apply the discovered patterns in the creation of a larger annotated dataset for training machine readers of research contributions; 3) to ingest the dataset into the Open Research Knowledge Graph (ORKG) infrastructure as a showcase for creating user-friendly state-of-the-art overviews; 4) to integrate the machine readers into the ORKG to assist users in the manual curation of their respective article contributions. We envision that the NLPContributions methodology engenders a wider discussion on the topic toward its further refinement and development. Our pilot annotated dataset of 50 NLP-ML scholarly articles per the NLPContributions scheme is available at this https URL.
摘要：我们描述一个注释主动捕捉自然语言处理（NLP）的文章，对学术贡献特别，对于讨论机器学习（ML）方法对各种信息提取的任务物品。我们开发了基于50 NLP-ML学术文章提出五个信息提取的任务1.机器翻译2.命名实体识别，3答疑，4关系分类，5贡献试点注释锻炼注解任务。文本分类。在这篇文章中，我们描述了这个试点注释阶段的成果。通过锻炼，我们已经获得了注释的方法;并且发现，反映了NLP-ML学术研究的贡献八大核心信息单元。我们开发了基于这些信息单元产生的注解方案被称为NLPContributions。我们努力的总体目标是：1）找到一个系统的一套那或多或少一般适用于NLP-ML研究论文学术贡献的语义结构主谓对象语句的模式; 2）应用在创造训练的研究贡献机器的读者一个更大的数据集注释的发现的模式; 3）摄取数据集到开放研究知识图（ORKG）基础设施作为用于创建用户友好的状态的最先进的概述的展示; 4）机器读者融入ORKG帮助用户在各自的文章贡献的人工管理。我们设想，在NLPContributions方法论上滋生朝其进一步改进和发展议题的更广泛的讨论。我们的飞行员注释50 NLP-ML学术每NLPContributions方案的文章数据集可在此HTTPS URL。
5. Inductive Unsupervised Domain Adaptation for Few-Shot Classification via Clustering [PDF] 返回目录
Xin Cong, Bowen Yu, Tingwen Liu, Shiyao Cui, Hengzhu Tang, Bin Wang
Abstract: Few-shot classification tends to struggle when it needs to adapt to diverse domains. Due to the non-overlapping label space between domains, the performance of conventional domain adaptation is limited. Previous work tackles the problem in a transductive manner, by assuming access to the full set of test data, which is too restrictive for many real-world applications. In this paper, we set out to tackle this issue by introducing a inductive framework, DaFeC, to improve Domain adaptation performance for Few-shot classification via Clustering. We first build a representation extractor to derive features for unlabeled data from the target domain (no test data is necessary) and then group them with a cluster miner. The generated pseudo-labeled data and the labeled source-domain data are used as supervision to update the parameters of the few-shot classifier. In order to derive high-quality pseudo labels, we propose a Clustering Promotion Mechanism, to learn better features for the target domain via Similarity Entropy Minimization and Adversarial Distribution Alignment, which are combined with a Cosine Annealing Strategy. Experiments are performed on the FewRel 2.0 dataset. Our approach outperforms previous work with absolute gains (in classification accuracy) of 4.95%, 9.55%, 3.99% and 11.62%, respectively, under four few-shot settings.
摘要：很少拍分类趋向时，它需要适应不同的领域奋斗。由于域之间的非重叠标签空间，常规域适应的性能的限制。以前的工作铲球的转导方式的问题，假设获得了全套的测试数据，这是过于严格了许多现实世界的应用。在本文中，我们设置了通过引入感应框架，DaFeC，以提高通过聚类为数不多的镜头分类域自适应性能来解决这个问题。我们首先建立一个表示提取器，用于从目标域中未标记的数据派生特征（无测试数据是必要的），然后将它们与群集矿工基。所生成的伪标记的数据和标记源域数据被用作监督更新少数次分类器的参数。为了导出高品质的伪标签，我们提出了一个集群推进机制，更好地学习功能对于通过相似熵极小和对抗式分布对齐，这是结合了余弦退火战略目标域。实验在FewRel 2.0数据集进行的。我们的方法优于与绝对收益（在分类准确度）分别为4.95％，9.55％，3.99％和11.62％，以前的工作，在四为数不多的拍摄设置。
Xin Cong, Bowen Yu, Tingwen Liu, Shiyao Cui, Hengzhu Tang, Bin Wang
Abstract: Few-shot classification tends to struggle when it needs to adapt to diverse domains. Due to the non-overlapping label space between domains, the performance of conventional domain adaptation is limited. Previous work tackles the problem in a transductive manner, by assuming access to the full set of test data, which is too restrictive for many real-world applications. In this paper, we set out to tackle this issue by introducing a inductive framework, DaFeC, to improve Domain adaptation performance for Few-shot classification via Clustering. We first build a representation extractor to derive features for unlabeled data from the target domain (no test data is necessary) and then group them with a cluster miner. The generated pseudo-labeled data and the labeled source-domain data are used as supervision to update the parameters of the few-shot classifier. In order to derive high-quality pseudo labels, we propose a Clustering Promotion Mechanism, to learn better features for the target domain via Similarity Entropy Minimization and Adversarial Distribution Alignment, which are combined with a Cosine Annealing Strategy. Experiments are performed on the FewRel 2.0 dataset. Our approach outperforms previous work with absolute gains (in classification accuracy) of 4.95%, 9.55%, 3.99% and 11.62%, respectively, under four few-shot settings.
摘要：很少拍分类趋向时，它需要适应不同的领域奋斗。由于域之间的非重叠标签空间，常规域适应的性能的限制。以前的工作铲球的转导方式的问题，假设获得了全套的测试数据，这是过于严格了许多现实世界的应用。在本文中，我们设置了通过引入感应框架，DaFeC，以提高通过聚类为数不多的镜头分类域自适应性能来解决这个问题。我们首先建立一个表示提取器，用于从目标域中未标记的数据派生特征（无测试数据是必要的），然后将它们与群集矿工基。所生成的伪标记的数据和标记源域数据被用作监督更新少数次分类器的参数。为了导出高品质的伪标签，我们提出了一个集群推进机制，更好地学习功能对于通过相似熵极小和对抗式分布对齐，这是结合了余弦退火战略目标域。实验在FewRel 2.0数据集进行的。我们的方法优于与绝对收益（在分类准确度）分别为4.95％，9.55％，3.99％和11.62％，以前的工作，在四为数不多的拍摄设置。
6. Keyframe Segmentation and Positional Encoding for Video-guided Machine Translation Challenge 2020 [PDF] 返回目录
Tosho Hirasawa, Zhishen Yang, Mamoru Komachi, Naoaki Okazaki
Abstract: Video-guided machine translation as one of multimodal neural machine translation tasks targeting on generating high-quality text translation by tangibly engaging both video and text. In this work, we presented our video-guided machine translation system in approaching the Video-guided Machine Translation Challenge 2020. This system employs keyframe-based video feature extractions along with the video feature positional encoding. In the evaluation phase, our system scored 36.60 corpus-level BLEU-4 and achieved the 1st place on the Video-guided Machine Translation Challenge 2020.
摘要：视频引导机器翻译的对，确实地从事视频和文本生成高质量的文本翻译瞄准多式联运神经机器翻译的任务之一。在这项工作中，我们提出我们的视频指导的机器翻译系统在接近视频引导机器翻译挑战2020年该系统采用基于关键帧的视频特征提取与视频功能位置编码一起。在评估阶段，我们的系统拿下36.60语料库级BLEU-4和视频引导机器翻译挑战2020获得第一名。
Tosho Hirasawa, Zhishen Yang, Mamoru Komachi, Naoaki Okazaki
Abstract: Video-guided machine translation as one of multimodal neural machine translation tasks targeting on generating high-quality text translation by tangibly engaging both video and text. In this work, we presented our video-guided machine translation system in approaching the Video-guided Machine Translation Challenge 2020. This system employs keyframe-based video feature extractions along with the video feature positional encoding. In the evaluation phase, our system scored 36.60 corpus-level BLEU-4 and achieved the 1st place on the Video-guided Machine Translation Challenge 2020.
摘要：视频引导机器翻译的对，确实地从事视频和文本生成高质量的文本翻译瞄准多式联运神经机器翻译的任务之一。在这项工作中，我们提出我们的视频指导的机器翻译系统在接近视频引导机器翻译挑战2020年该系统采用基于关键帧的视频特征提取与视频功能位置编码一起。在评估阶段，我们的系统拿下36.60语料库级BLEU-4和视频引导机器翻译挑战2020获得第一名。
7. Unsupervised Evaluation of Interactive Dialog with DialoGPT [PDF] 返回目录
Shikib Mehri, Maxine Eskenazi
Abstract: It is important to define meaningful and interpretable automatic evaluation metrics for open-domain dialog research. Standard language generation metrics have been shown to be ineffective for dialog. This paper introduces the FED metric (fine-grained evaluation of dialog), an automatic evaluation metric which uses DialoGPT, without any fine-tuning or supervision. It also introduces the FED dataset which is constructed by annotating a set of human-system and human-human conversations with eighteen fine-grained dialog qualities. The FED metric (1) does not rely on a ground-truth response, (2) does not require training data and (3) measures fine-grained dialog qualities at both the turn and whole dialog levels. FED attains moderate to strong correlation with human judgement at both levels.
摘要：定义为开放域对话框的研究意义和解释的自动评估指标是非常重要的。标准语言生成的指标已被证明是无效的对话框。本文介绍了FED度量（对话的细粒度评价），自动评估度量其使用DialoGPT，没有任何微调或监督。它还介绍了一种通过注释的一组人的系统和人 - 人交谈的用十八细粒对话框素质构成的FED数据集。美联储指标（1）不依赖于地面实况响应，（2）不需要之交和整个对话两级训练数据和（3）措施细粒度对话框品质。 FED温和无所获在这两个级别与人的判断密切相关。
Shikib Mehri, Maxine Eskenazi
Abstract: It is important to define meaningful and interpretable automatic evaluation metrics for open-domain dialog research. Standard language generation metrics have been shown to be ineffective for dialog. This paper introduces the FED metric (fine-grained evaluation of dialog), an automatic evaluation metric which uses DialoGPT, without any fine-tuning or supervision. It also introduces the FED dataset which is constructed by annotating a set of human-system and human-human conversations with eighteen fine-grained dialog qualities. The FED metric (1) does not rely on a ground-truth response, (2) does not require training data and (3) measures fine-grained dialog qualities at both the turn and whole dialog levels. FED attains moderate to strong correlation with human judgement at both levels.
摘要：定义为开放域对话框的研究意义和解释的自动评估指标是非常重要的。标准语言生成的指标已被证明是无效的对话框。本文介绍了FED度量（对话的细粒度评价），自动评估度量其使用DialoGPT，没有任何微调或监督。它还介绍了一种通过注释的一组人的系统和人 - 人交谈的用十八细粒对话框素质构成的FED数据集。美联储指标（1）不依赖于地面实况响应，（2）不需要之交和整个对话两级训练数据和（3）措施细粒度对话框品质。 FED温和无所获在这两个级别与人的判断密切相关。
8. Exploring Software Naturalness throughNeural Language Models [PDF] 返回目录
Luca Buratti, Saurabh Pujar, Mihaela Bornea, Scott McCarley, Yunhui Zheng, Gaetano Rossiello, Alessandro Morari, Jim Laredo, Veronika Thost, Yufan Zhuang, Giacomo Domeniconi
Abstract: The Software Naturalness hypothesis argues that programming languages can be understood through the same techniques used in natural language processing. We explore this hypothesis through the use of a pre-trained transformer-based language model to perform code analysis tasks.
摘要：软件自然性假说认为，编程语言可以通过自然语言处理中使用的相同的技术来理解。我们通过使用预先训练的基于变压器的语言模型的探索这一假设进行代码分析任务。
Luca Buratti, Saurabh Pujar, Mihaela Bornea, Scott McCarley, Yunhui Zheng, Gaetano Rossiello, Alessandro Morari, Jim Laredo, Veronika Thost, Yufan Zhuang, Giacomo Domeniconi
Abstract: The Software Naturalness hypothesis argues that programming languages can be understood through the same techniques used in natural language processing. We explore this hypothesis through the use of a pre-trained transformer-based language model to perform code analysis tasks.
摘要：软件自然性假说认为，编程语言可以通过自然语言处理中使用的相同的技术来理解。我们通过使用预先训练的基于变压器的语言模型的探索这一假设进行代码分析任务。
注：中文为机器翻译结果！