目录
2. Capturing document context inside sentence-level neural machine translation models with self-training [PDF] 摘要
5. TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages [PDF] 摘要
8. Masking Orchestration: Multi-task Pretraining for Multi-role Dialogue Representation Learning [PDF] 摘要
11. Unsupervised and Interpretable Domain Adaptation to Rapidly Filter Social Web Data for Emergency Services [PDF] 摘要
14. Investigating an approach for low resource language dataset creation, curation and classification: Setswana and Sepedi [PDF] 摘要
15. Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT [PDF] 摘要
18. Improving Reliability of Latent Dirichlet Allocation by Assessing Its Stability Using Clustering Techniques on Replicated Runs [PDF] 摘要
23. Localized Flood DetectionWith Minimal Labeled Social Media Data Using Transfer Learning [PDF] 摘要
24. A Comparative Study of Sequence Classification Models for Privacy Policy Coverage Analysis [PDF] 摘要
27. KryptoOracle: A Real-Time Cryptocurrency Price Prediction Platform Using Twitter Sentiments [PDF] 摘要
摘要
1. Brazilian Lyrics-Based Music Genre Classification Using a BLSTM Network [PDF] 返回目录
Raul de Araújo Lima, Rômulo César Costa de Sousa, Simone Diniz Junqueira Barbosa, Hélio Cortês Vieira Lopes
Abstract: Organize songs, albums, and artists in groups with shared similarity could be done with the help of genre labels. In this paper, we present a novel approach for automatic classifying musical genre in Brazilian music using only the song lyrics. This kind of classification remains a challenge in the field of Natural Language Processing. We construct a dataset of 138,368 Brazilian song lyrics distributed in 14 genres. We apply SVM, Random Forest and a Bidirectional Long Short-Term Memory (BLSTM) network combined with different word embeddings techniques to address this classification task. Our experiments show that the BLSTM method outperforms the other models with an F1-score average of $0.48$. Some genres like "gospel", "funk-carioca" and "sertanejo", which obtained 0.89, 0.70 and 0.69 of F1-score, respectively, can be defined as the most distinct and easy to classify in the Brazilian musical genres context.
摘要：组织歌曲，专辑和艺术家群体共享相似性可以与流派标签的帮助下完成的。在本文中，我们只用歌词本作自动分类音乐风格的新方法在巴西音乐。这种分类仍然是自然语言处理领域的挑战。我们构造的分布在14个流派138368和巴西歌词的数据集。我们采用SVM，随机森林和双向长短期记忆（BLSTM）网络具有不同的字的嵌入技术来解决这个分类任务相结合。我们的实验表明，BLSTM方法优于其他车型与F1-得分平均为$ 0.48 $。一些类型如“福音”，“放克-卡里奥卡”和“sertanejo”，这得到0.89，0.70和F1-得分为0.69，可以分别定义为最截然不同的且易于在巴西音乐流派上下文进行分类。
Raul de Araújo Lima, Rômulo César Costa de Sousa, Simone Diniz Junqueira Barbosa, Hélio Cortês Vieira Lopes
Abstract: Organize songs, albums, and artists in groups with shared similarity could be done with the help of genre labels. In this paper, we present a novel approach for automatic classifying musical genre in Brazilian music using only the song lyrics. This kind of classification remains a challenge in the field of Natural Language Processing. We construct a dataset of 138,368 Brazilian song lyrics distributed in 14 genres. We apply SVM, Random Forest and a Bidirectional Long Short-Term Memory (BLSTM) network combined with different word embeddings techniques to address this classification task. Our experiments show that the BLSTM method outperforms the other models with an F1-score average of $0.48$. Some genres like "gospel", "funk-carioca" and "sertanejo", which obtained 0.89, 0.70 and 0.69 of F1-score, respectively, can be defined as the most distinct and easy to classify in the Brazilian musical genres context.
摘要：组织歌曲，专辑和艺术家群体共享相似性可以与流派标签的帮助下完成的。在本文中，我们只用歌词本作自动分类音乐风格的新方法在巴西音乐。这种分类仍然是自然语言处理领域的挑战。我们构造的分布在14个流派138368和巴西歌词的数据集。我们采用SVM，随机森林和双向长短期记忆（BLSTM）网络具有不同的字的嵌入技术来解决这个分类任务相结合。我们的实验表明，BLSTM方法优于其他车型与F1-得分平均为$ 0.48 $。一些类型如“福音”，“放克-卡里奥卡”和“sertanejo”，这得到0.89，0.70和F1-得分为0.69，可以分别定义为最截然不同的且易于在巴西音乐流派上下文进行分类。
2. Capturing document context inside sentence-level neural machine translation models with self-training [PDF] 返回目录
Elman Mansimov, Gábor Melis, Lei Yu
Abstract: Neural machine translation (NMT) has arguably achieved human level parity when trained and evaluated at the sentence-level. Document-level neural machine translation has received less attention and lags behind its sentence-level counterpart. The majority of the proposed document-level approaches investigate ways of conditioning the model on several source or target sentences to capture document context. These approaches require training a specialized NMT model from scratch on parallel document-level corpora. We propose an approach that doesn't require training a specialized model on parallel document-level corpora and is applied to a trained sentence-level NMT model at decoding time. We process the document from left to right multiple times and self-train the sentence-level model on pairs of source sentences and generated translations. Our approach reinforces the choices made by the model, thus making it more likely that the same choices will be made in other sentences in the document. We evaluate our approach on three document-level datasets: NIST Chinese-English, WMT'19 Chinese-English and OpenSubtitles English-Russian. We demonstrate that our approach has higher BLEU score and higher human preference than the baseline. Qualitative analysis of our approach shows that choices made by model are consistent across the document.
摘要：经过培训，并在句子层面进行评估时，神经机器翻译（NMT）可以说是已经达到人类的水平奇偶校验。文档级神经机器翻译已经受到足够的重视和滞后的语句级对应。大多数提出的文档级的方法探讨几个源或目标的句子来捕获文档上下文调理的方法模型。这些方法需要从头并行文档级语料训练专业NMT模型。我们建议，不需要培训并行文档级语料库一个专业模特，并在解码时应用到训练的句子级NMT模型的方法。我们处理文档从左至右多次和自我训练的句级模型上对源句子和产生的翻译。我们的做法加强该模型做出的选择，从而使得它更可能是同样的选择将在文档中的其他句子进行。我们评估我们的三个文件级数据集的方法：NIST中国 - 英语，WMT'19中国 - 英语和英语OpenSubtitles俄罗斯。我们证明我们的方法具有较高的BLEU得分和更高的人的偏好比基线。我们的做法表明定性分析，通过模型做出的选择是在文档一致。
Elman Mansimov, Gábor Melis, Lei Yu
Abstract: Neural machine translation (NMT) has arguably achieved human level parity when trained and evaluated at the sentence-level. Document-level neural machine translation has received less attention and lags behind its sentence-level counterpart. The majority of the proposed document-level approaches investigate ways of conditioning the model on several source or target sentences to capture document context. These approaches require training a specialized NMT model from scratch on parallel document-level corpora. We propose an approach that doesn't require training a specialized model on parallel document-level corpora and is applied to a trained sentence-level NMT model at decoding time. We process the document from left to right multiple times and self-train the sentence-level model on pairs of source sentences and generated translations. Our approach reinforces the choices made by the model, thus making it more likely that the same choices will be made in other sentences in the document. We evaluate our approach on three document-level datasets: NIST Chinese-English, WMT'19 Chinese-English and OpenSubtitles English-Russian. We demonstrate that our approach has higher BLEU score and higher human preference than the baseline. Qualitative analysis of our approach shows that choices made by model are consistent across the document.
摘要：经过培训，并在句子层面进行评估时，神经机器翻译（NMT）可以说是已经达到人类的水平奇偶校验。文档级神经机器翻译已经受到足够的重视和滞后的语句级对应。大多数提出的文档级的方法探讨几个源或目标的句子来捕获文档上下文调理的方法模型。这些方法需要从头并行文档级语料训练专业NMT模型。我们建议，不需要培训并行文档级语料库一个专业模特，并在解码时应用到训练的句子级NMT模型的方法。我们处理文档从左至右多次和自我训练的句级模型上对源句子和产生的翻译。我们的做法加强该模型做出的选择，从而使得它更可能是同样的选择将在文档中的其他句子进行。我们评估我们的三个文件级数据集的方法：NIST中国 - 英语，WMT'19中国 - 英语和英语OpenSubtitles俄罗斯。我们证明我们的方法具有较高的BLEU得分和更高的人的偏好比基线。我们的做法表明定性分析，通过模型做出的选择是在文档一致。
3. Vector symbolic architectures for context-free grammars [PDF] 返回目录
Peter beim Graben, Markus Huber, Werner Meyer, Ronald Römer, Constanze Tschöpe, Matthias Wolff
Abstract: Background / introduction. Vector symbolic architectures (VSA) are a viable approach for the hyperdimensional representation of symbolic data, such as documents, syntactic structures, or semantic frames. Methods. We present a rigorous mathematical framework for the representation of phrase structure trees and parse-trees of context-free grammars (CFG) in Fock space, i.e. infinite-dimensional Hilbert space as being used in quantum field theory. We define a novel normal form for CFG by means of term algebras. Using a recently developed software toolbox, called FockBox, we construct Fock space representations for the trees built up by a CFG left-corner (LC) parser. Results. We prove a universal representation theorem for CFG term algebras in Fock space and illustrate our findings through a low-dimensional principal component projection of the LC parser states. Conclusions. Our approach could leverage the development of VSA for explainable artificial intelligence (XAI) by means of hyperdimensional deep neural computation. It could be of significance for the improvement of cognitive user interfaces and other applications of VSA in machine learning.
摘要：背景/介绍。矢量象征性体系结构（VSA）是符号数据，如文档，句法结构，或语义帧的超维度表示一种可行的方法。方法。我们提出了一个严格的数学框架的短语结构树和在福克空间，即无限维Hilbert空间作为量子场论正在使用上下文无关文法（CFG）的分析树表示。我们通过术语代数的装置限定了新颖的正常形式CFG。使用最近开发的软件工具箱，称为FockBox，我们构建福克空间表示了由CFG建立起来的树木左拐角（LC）解析器。结果。我们证明了在福克空间CFG长期代数通用的表示定理，并通过LC解析器状态的低维主成分投影说明我们的调查结果。结论。我们的方法可以利用VSA对解释的人工智能（XAI）的超维度深层神经计算手段的发展。这可能是认知用户界面和机器学习VSA其他应用程序的改善具有重要意义。
Peter beim Graben, Markus Huber, Werner Meyer, Ronald Römer, Constanze Tschöpe, Matthias Wolff
Abstract: Background / introduction. Vector symbolic architectures (VSA) are a viable approach for the hyperdimensional representation of symbolic data, such as documents, syntactic structures, or semantic frames. Methods. We present a rigorous mathematical framework for the representation of phrase structure trees and parse-trees of context-free grammars (CFG) in Fock space, i.e. infinite-dimensional Hilbert space as being used in quantum field theory. We define a novel normal form for CFG by means of term algebras. Using a recently developed software toolbox, called FockBox, we construct Fock space representations for the trees built up by a CFG left-corner (LC) parser. Results. We prove a universal representation theorem for CFG term algebras in Fock space and illustrate our findings through a low-dimensional principal component projection of the LC parser states. Conclusions. Our approach could leverage the development of VSA for explainable artificial intelligence (XAI) by means of hyperdimensional deep neural computation. It could be of significance for the improvement of cognitive user interfaces and other applications of VSA in machine learning.
摘要：背景/介绍。矢量象征性体系结构（VSA）是符号数据，如文档，句法结构，或语义帧的超维度表示一种可行的方法。方法。我们提出了一个严格的数学框架的短语结构树和在福克空间，即无限维Hilbert空间作为量子场论正在使用上下文无关文法（CFG）的分析树表示。我们通过术语代数的装置限定了新颖的正常形式CFG。使用最近开发的软件工具箱，称为FockBox，我们构建福克空间表示了由CFG建立起来的树木左拐角（LC）解析器。结果。我们证明了在福克空间CFG长期代数通用的表示定理，并通过LC解析器状态的低维主成分投影说明我们的调查结果。结论。我们的方法可以利用VSA对解释的人工智能（XAI）的超维度深层神经计算手段的发展。这可能是认知用户界面和机器学习VSA其他应用程序的改善具有重要意义。
4. A Benchmark for Systematic Generalization in Grounded Language Understanding [PDF] 返回目录
Laura Ruis, Jacob Andreas, Marco Baroni, Diane Bouchacourt, Brenden M. Lake
Abstract: Human language users easily interpret expressions that describe unfamiliar situations composed from familiar parts ("greet the pink brontosaurus by the ferris wheel"). Modern neural networks, by contrast, struggle to interpret compositions unseen in training. In this paper, we introduce a new benchmark, gSCAN, for evaluating compositional generalization in models of situated language understanding. We take inspiration from standard models of meaning composition in formal linguistics. Going beyond an earlier related benchmark that focused on syntactic aspects of generalization, gSCAN defines a language grounded in the states of a grid world. This allows us to build novel generalization tasks that probe the acquisition of linguistically motivated rules. For example, agents must understand how adjectives such as 'small' are interpreted relative to the current world state or how adverbs such as 'cautiously' combine with new verbs. We test a strong multi-modal baseline model and a state-of-the-art compositional method finding that, in most cases, they fail dramatically when generalization requires systematic compositional rules.
摘要：人类语言的用户轻松地解释，描述从熟悉的部分组成不熟悉的情况表述（“迎接由摩天轮的粉红色雷龙”）。现代神经网络，相比之下，斗争来解释训练看不见的成分。在本文中，我们引入了新的标杆，gSCAN，为的情境语言理解模型评估的组成概括。我们采取的灵感来自于正规的语言学意义成分的规格型号。超越是重点推广的句法方面较早的相关基准，gSCAN定义网格世界各国接地的语言。这使我们能够建立一个探讨收购的动机语言规则的新的推广任务。例如，代理商必须了解的形容词，如“小”是相对于当前世界国家如何副词，如“谨慎”与新的动词结合解释。我们测试一个强大的多模态基准模型和一个国家的最先进的成分的方法发现，在大多数情况下，他们在推广需要系统的组成规则，极大地失败。
Laura Ruis, Jacob Andreas, Marco Baroni, Diane Bouchacourt, Brenden M. Lake
Abstract: Human language users easily interpret expressions that describe unfamiliar situations composed from familiar parts ("greet the pink brontosaurus by the ferris wheel"). Modern neural networks, by contrast, struggle to interpret compositions unseen in training. In this paper, we introduce a new benchmark, gSCAN, for evaluating compositional generalization in models of situated language understanding. We take inspiration from standard models of meaning composition in formal linguistics. Going beyond an earlier related benchmark that focused on syntactic aspects of generalization, gSCAN defines a language grounded in the states of a grid world. This allows us to build novel generalization tasks that probe the acquisition of linguistically motivated rules. For example, agents must understand how adjectives such as 'small' are interpreted relative to the current world state or how adverbs such as 'cautiously' combine with new verbs. We test a strong multi-modal baseline model and a state-of-the-art compositional method finding that, in most cases, they fail dramatically when generalization requires systematic compositional rules.
摘要：人类语言的用户轻松地解释，描述从熟悉的部分组成不熟悉的情况表述（“迎接由摩天轮的粉红色雷龙”）。现代神经网络，相比之下，斗争来解释训练看不见的成分。在本文中，我们引入了新的标杆，gSCAN，为的情境语言理解模型评估的组成概括。我们采取的灵感来自于正规的语言学意义成分的规格型号。超越是重点推广的句法方面较早的相关基准，gSCAN定义网格世界各国接地的语言。这使我们能够建立一个探讨收购的动机语言规则的新的推广任务。例如，代理商必须了解的形容词，如“小”是相对于当前世界国家如何副词，如“谨慎”与新的动词结合解释。我们测试一个强大的多模态基准模型和一个国家的最先进的成分的方法发现，在大多数情况下，他们在推广需要系统的组成规则，极大地失败。
5. TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages [PDF] 返回目录
Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, Jennimaria Palomaki
Abstract: Confidently making progress on multilingual modeling requires challenging, trustworthy evaluations. We present TyDi QA---a question answering dataset covering 11 typologically diverse languages with 204K question-answer pairs. The languages of TyDi QA are diverse with regard to their typology---the set of linguistic features each language expresses---such that we expect models performing well on this set to generalize across a large number of the world's languages. We present a quantitative analysis of the data quality and example-level qualitative linguistic analyses of observed language phenomena that would not be found in English-only corpora. To provide a realistic information-seeking task and avoid priming effects, questions are written by people who want to know the answer, but don't know the answer yet, and the data is collected directly in each language without the use of translation.
摘要：自信使多语种建模进步需要挑战的，值得信赖的评价。我们提出泰迪QA ---一个问题回答的数据集涵盖11种不同的类型学与语言204K的问答配对。泰迪QA的语言是不同的关于他们的类型学---设定语言的功能每种语言都表达---这样，我们期望在这组表现良好的模型在大量世界语言的概括。我们目前观察到的语言现象的数据质量和例子级定性语言的分析，不会在纯英语语料库中找到的定量分析。为了提供一个现实的信息搜索任务，避免启动效应，问题是谁想要知道答案，但不知道答案但人写的，而数据在不使用翻译的各种语言直接采集。
Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, Jennimaria Palomaki
Abstract: Confidently making progress on multilingual modeling requires challenging, trustworthy evaluations. We present TyDi QA---a question answering dataset covering 11 typologically diverse languages with 204K question-answer pairs. The languages of TyDi QA are diverse with regard to their typology---the set of linguistic features each language expresses---such that we expect models performing well on this set to generalize across a large number of the world's languages. We present a quantitative analysis of the data quality and example-level qualitative linguistic analyses of observed language phenomena that would not be found in English-only corpora. To provide a realistic information-seeking task and avoid priming effects, questions are written by people who want to know the answer, but don't know the answer yet, and the data is collected directly in each language without the use of translation.
摘要：自信使多语种建模进步需要挑战的，值得信赖的评价。我们提出泰迪QA ---一个问题回答的数据集涵盖11种不同的类型学与语言204K的问答配对。泰迪QA的语言是不同的关于他们的类型学---设定语言的功能每种语言都表达---这样，我们期望在这组表现良好的模型在大量世界语言的概括。我们目前观察到的语言现象的数据质量和例子级定性语言的分析，不会在纯英语语料库中找到的定量分析。为了提供一个现实的信息搜索任务，避免启动效应，问题是谁想要知道答案，但不知道答案但人写的，而数据在不使用翻译的各种语言直接采集。
6. Toward Interpretability of Dual-Encoder Models for Dialogue Response Suggestions [PDF] 返回目录
Yitong Li, Dianqi Li, Sushant Prakash, Peng Wang
Abstract: This work shows how to improve and interpret the commonly used dual encoder model for response suggestion in dialogue. We present an attentive dual encoder model that includes an attention mechanism on top of the extracted word-level features from two encoders, one for context and one for label respectively. To improve the interpretability in the dual encoder models, we design a novel regularization loss to minimize the mutual information between unimportant words and desired labels, in addition to the original attention method, so that important words are emphasized while unimportant words are de-emphasized. This can help not only with model interpretability, but can also further improve model accuracy. We propose an approximation method that uses a neural network to calculate the mutual information. Furthermore, by adding a residual layer between raw word embeddings and the final encoded context feature, word-level interpretability is preserved at the final prediction of the model. We compare the proposed model with existing methods for the dialogue response task on two public datasets (Persona and Ubuntu). The experiments demonstrate the effectiveness of the proposed model in terms of better Recall@1 accuracy and visualized interpretability.
摘要：该作品展示了如何改善和解读对话响应建议常用的双编码器模式。我们提出了一种双周到编码器模型，其包括来自两个编码器，一个用于上下文和分别为一个标签上的所提取的词级别特征顶部的注意机制。为了提高在双编码器模型的可解释性，我们设计了一种新的正则损失减少不重要的词和期望的标签之间的相互信息，除了原来的关注方法，使重要的话被强调，而不重要的词去强调。这不仅可以帮助与模型解释性，而且还可以进一步提高模型的准确性。我们建议采用神经网络计算相互信息的近似方法。此外，通过添加生的嵌入字和最终的编码上下文特征之间残留层，词级解释性在该模型的最终预测保留。我们比较了该模型与两个公共数据集（假面和Ubuntu）的对话响应任务的现有方法。实验证明更好的召回@ 1准确性和可视化解释性方面所提出的模型的有效性。
Yitong Li, Dianqi Li, Sushant Prakash, Peng Wang
Abstract: This work shows how to improve and interpret the commonly used dual encoder model for response suggestion in dialogue. We present an attentive dual encoder model that includes an attention mechanism on top of the extracted word-level features from two encoders, one for context and one for label respectively. To improve the interpretability in the dual encoder models, we design a novel regularization loss to minimize the mutual information between unimportant words and desired labels, in addition to the original attention method, so that important words are emphasized while unimportant words are de-emphasized. This can help not only with model interpretability, but can also further improve model accuracy. We propose an approximation method that uses a neural network to calculate the mutual information. Furthermore, by adding a residual layer between raw word embeddings and the final encoded context feature, word-level interpretability is preserved at the final prediction of the model. We compare the proposed model with existing methods for the dialogue response task on two public datasets (Persona and Ubuntu). The experiments demonstrate the effectiveness of the proposed model in terms of better Recall@1 accuracy and visualized interpretability.
摘要：该作品展示了如何改善和解读对话响应建议常用的双编码器模式。我们提出了一种双周到编码器模型，其包括来自两个编码器，一个用于上下文和分别为一个标签上的所提取的词级别特征顶部的注意机制。为了提高在双编码器模型的可解释性，我们设计了一种新的正则损失减少不重要的词和期望的标签之间的相互信息，除了原来的关注方法，使重要的话被强调，而不重要的词去强调。这不仅可以帮助与模型解释性，而且还可以进一步提高模型的准确性。我们建议采用神经网络计算相互信息的近似方法。此外，通过添加生的嵌入字和最终的编码上下文特征之间残留层，词级解释性在该模型的最终预测保留。我们比较了该模型与两个公共数据集（假面和Ubuntu）的对话响应任务的现有方法。实验证明更好的召回@ 1准确性和可视化解释性方面所提出的模型的有效性。
7. GASP! Generating Abstracts of Scientific Papers from Abstracts of Cited Papers [PDF] 返回目录
Fabio Massimo Zanzotto, Viviana Bono, Paola Vocca, Andrea Santilli, Danilo Croce, Giorgio Gambosi, Roberto Basili
Abstract: Creativity is one of the driving forces of human kind as it allows to break current understanding to envision new ideas, which may revolutionize entire fields of knowledge. Scientific research offers a challenging environment where to learn a model for the creative process. In fact, scientific research is a creative act in the formal settings of the scientific method and this creative act is described in articles. In this paper, we dare to introduce the novel, scientifically and philosophically challenging task of Generating Abstracts of Scientific Papers from abstracts of cited papers (GASP) as a text-to-text task to investigate scientific creativity, To foster research in this novel, challenging task, we prepared a dataset by using services where that solve the problem of copyright and, hence, the dataset is public available with its standard split. Finally, we experimented with two vanilla summarization systems to start the analysis of the complexity of the GASP task.
摘要：创新是人类的驱动力之一，因为它可以打破目前的了解设想新的想法，这可能会带来革命性的知识全部领域。科学研究提供一个具有挑战性的环境中去学习创作过程的模型。事实上，科学研究是科学方法的正式设置的创意行为，这种创造性的行为在文章中描述。在本文中，我们敢于引进新的，科学和哲学从被引论文（GASP）作为文本到文本任务的摘要挑战性的科学论文中生成摘要的任务是调查科学创新，促进研究在这部小说中，具有挑战性的任务，我们通过使用如该解决版权的问题，因此，该数据集可公开获得其标准分部门编制的数据集。最后，我们有两个香草摘要系统试验开始的GASP任务的复杂性分析。
Fabio Massimo Zanzotto, Viviana Bono, Paola Vocca, Andrea Santilli, Danilo Croce, Giorgio Gambosi, Roberto Basili
Abstract: Creativity is one of the driving forces of human kind as it allows to break current understanding to envision new ideas, which may revolutionize entire fields of knowledge. Scientific research offers a challenging environment where to learn a model for the creative process. In fact, scientific research is a creative act in the formal settings of the scientific method and this creative act is described in articles. In this paper, we dare to introduce the novel, scientifically and philosophically challenging task of Generating Abstracts of Scientific Papers from abstracts of cited papers (GASP) as a text-to-text task to investigate scientific creativity, To foster research in this novel, challenging task, we prepared a dataset by using services where that solve the problem of copyright and, hence, the dataset is public available with its standard split. Finally, we experimented with two vanilla summarization systems to start the analysis of the complexity of the GASP task.
摘要：创新是人类的驱动力之一，因为它可以打破目前的了解设想新的想法，这可能会带来革命性的知识全部领域。科学研究提供一个具有挑战性的环境中去学习创作过程的模型。事实上，科学研究是科学方法的正式设置的创意行为，这种创造性的行为在文章中描述。在本文中，我们敢于引进新的，科学和哲学从被引论文（GASP）作为文本到文本任务的摘要挑战性的科学论文中生成摘要的任务是调查科学创新，促进研究在这部小说中，具有挑战性的任务，我们通过使用如该解决版权的问题，因此，该数据集可公开获得其标准分部门编制的数据集。最后，我们有两个香草摘要系统试验开始的GASP任务的复杂性分析。
8. Masking Orchestration: Multi-task Pretraining for Multi-role Dialogue Representation Learning [PDF] 返回目录
Tianyi Wang, Yating Zhang, Xiaozhong Liu, Changlong Sun, Qiong Zhang
Abstract: Multi-role dialogue understanding comprises a wide range of diverse tasks such as question answering, act classification, dialogue summarization etc. While dialogue corpora are abundantly available, labeled data, for specific learning tasks, can be highly scarce and expensive. In this work, we investigate dialogue context representation learning with various types unsupervised pretraining tasks where the training objectives are given naturally according to the nature of the utterance and the structure of the multi-role conversation. Meanwhile, in order to locate essential information for dialogue summarization/extraction, the pretraining process enables external knowledge integration. The proposed fine-tuned pretraining mechanism is comprehensively evaluated via three different dialogue datasets along with a number of downstream dialogue-mining tasks. Result shows that the proposed pretraining mechanism significantly contributes to all the downstream tasks without discrimination to different encoders.
摘要：多角色对话的理解包括范围广泛的各种任务，如问题解答，行为分类，总结对话等对话虽然语料来源丰富，标签数据，为特定的学习任务，可以是高度稀缺和昂贵。在这项工作中，我们研究与训练前无人监管的地方培养目标是根据话语的性质和多角色对话的结构赋予自然的任务的各类对话上下文表示学习。同时，为了找到对话汇总/提取必要的信息，在训练前过程使外部知识的整合。所提出的微调预训练机制，通过三个不同的数据集对话的综合评价与多家下游的对话，挖掘任务一起。结果表明，所提出的训练前的机制显著有助于所有下游任务，不受歧视地不同的编码器。
Tianyi Wang, Yating Zhang, Xiaozhong Liu, Changlong Sun, Qiong Zhang
Abstract: Multi-role dialogue understanding comprises a wide range of diverse tasks such as question answering, act classification, dialogue summarization etc. While dialogue corpora are abundantly available, labeled data, for specific learning tasks, can be highly scarce and expensive. In this work, we investigate dialogue context representation learning with various types unsupervised pretraining tasks where the training objectives are given naturally according to the nature of the utterance and the structure of the multi-role conversation. Meanwhile, in order to locate essential information for dialogue summarization/extraction, the pretraining process enables external knowledge integration. The proposed fine-tuned pretraining mechanism is comprehensively evaluated via three different dialogue datasets along with a number of downstream dialogue-mining tasks. Result shows that the proposed pretraining mechanism significantly contributes to all the downstream tasks without discrimination to different encoders.
摘要：多角色对话的理解包括范围广泛的各种任务，如问题解答，行为分类，总结对话等对话虽然语料来源丰富，标签数据，为特定的学习任务，可以是高度稀缺和昂贵。在这项工作中，我们研究与训练前无人监管的地方培养目标是根据话语的性质和多角色对话的结构赋予自然的任务的各类对话上下文表示学习。同时，为了找到对话汇总/提取必要的信息，在训练前过程使外部知识的整合。所提出的微调预训练机制，通过三个不同的数据集对话的综合评价与多家下游的对话，挖掘任务一起。结果表明，所提出的训练前的机制显著有助于所有下游任务，不受歧视地不同的编码器。
9. Learning to mirror speaking styles incrementally [PDF] 返回目录
Siyi Liu, Ziang Leng, Derry Wijaya
Abstract: Mirroring is the behavior in which one person subconsciously imitates the gesture, speech pattern, or attitude of another. In conversations, mirroring often signals the speakers enjoyment and engagement in their communication. In chatbots, methods have been proposed to add personas to the chatbots and to train them to speak or to shift their dialogue style to that of the personas. However, they often require a large dataset consisting of dialogues of the target personalities to train. In this work, we explore a method that can learn to mirror the speaking styles of a person incrementally. Our method extracts ngrams that capture a persons speaking styles and uses the ngrams to create patterns for transforming sentences to the persons speaking styles. Our experiments show that our method is able to capture patterns of speaking style that can be used to transform regular sentences into sentences with the target style.
摘要：镜像是其中一个人下意识的模仿的姿态，说话方式，或者其他的态度行为。在交谈中，往往镜像信号在其通信扬声器享受和参与。在聊天机器人，方法已经被提出来的人物角色添加到聊天机器人，并训练他们发言或他们的对话风格转向的人物角色。然而，他们往往需要大量的数据集，包括定位的个性，以列车的对话。在这项工作中，我们探讨的是可以学习增量镜像讲话风格的人的方法。我们的方法提取的n-gram，捕捉一个人的说话风格，并使用n元语法来转化句话来讲风格的人创建模式。我们的实验表明，我们的方法是能够捕捉到说话，可用于常规的句子转换成与目标风格的句子风格的图案。
Siyi Liu, Ziang Leng, Derry Wijaya
Abstract: Mirroring is the behavior in which one person subconsciously imitates the gesture, speech pattern, or attitude of another. In conversations, mirroring often signals the speakers enjoyment and engagement in their communication. In chatbots, methods have been proposed to add personas to the chatbots and to train them to speak or to shift their dialogue style to that of the personas. However, they often require a large dataset consisting of dialogues of the target personalities to train. In this work, we explore a method that can learn to mirror the speaking styles of a person incrementally. Our method extracts ngrams that capture a persons speaking styles and uses the ngrams to create patterns for transforming sentences to the persons speaking styles. Our experiments show that our method is able to capture patterns of speaking style that can be used to transform regular sentences into sentences with the target style.
摘要：镜像是其中一个人下意识的模仿的姿态，说话方式，或者其他的态度行为。在交谈中，往往镜像信号在其通信扬声器享受和参与。在聊天机器人，方法已经被提出来的人物角色添加到聊天机器人，并训练他们发言或他们的对话风格转向的人物角色。然而，他们往往需要大量的数据集，包括定位的个性，以列车的对话。在这项工作中，我们探讨的是可以学习增量镜像讲话风格的人的方法。我们的方法提取的n-gram，捕捉一个人的说话风格，并使用n元语法来转化句话来讲风格的人创建模式。我们的实验表明，我们的方法是能够捕捉到说话，可用于常规的句子转换成与目标风格的句子风格的图案。
10. Multi-task Learning with Multi-head Attention for Multi-choice Reading Comprehension [PDF] 返回目录
Hui Wan
Abstract: Multiple-choice Machine Reading Comprehension (MRC) is an important and challenging Natural Language Understanding (NLU) task, in which a machine must choose the answer to a question from a set of choices, with the question placed in context of text passages or dialog. In the last a couple of years the NLU field has been revolutionized with the advent of models based on the Transformer architecture, which are pretrained on massive amounts of unsupervised data and then fine-tuned for various supervised learning NLU tasks. Transformer models have come to dominate a wide variety of leader-boards in the NLU field; in the area of MRC, the current state-of-the-art model on the DREAM dataset (see[Sunet al., 2019]) fine tunes Albert, a large pretrained Transformer-based model, and addition-ally combines it with an extra layer of multi-head attention between context and question-answer[Zhuet al., 2020].The purpose of this note is to document a new state-of-the-art result in the DREAM task, which is accomplished by, additionally, performing multi-task learning on two MRC multi-choice reading comprehension tasks (RACE and DREAM).
摘要：多选机阅读理解（MRC）是一个重要而具有挑战性的自然语言理解（NLU）任务，在其中一台机器必须选择的答案，从一组的选择问题，放置在文本段落的背景问题或对话框。在过去的几年中NLU领域发生了革命性的具有基于变压器的架构模型，这是预先训练上大量的无监督的数据，然后微调各种监督学习NLU任务的到来。变压器型号有来主宰各种各样的自然语言理解领域的佼佼者，板;在MRC的区域中，当前状态的最先进的模型上DREAM数据集（见[SUNET人，2019]）精细调谐阿尔伯特，大预训练的变压器为基础的模型，并加入烯丙基与结合它的上下文和问答之间多头注意额外层[Zhuet人，2020]本说明的目的.The在DREAM任务，这是通过以下步骤完成记录一个新的国家的最先进的结果，另外，两个MRC多选择阅读理解任务执行多任务学习（RACE和DREAM）。
Hui Wan
Abstract: Multiple-choice Machine Reading Comprehension (MRC) is an important and challenging Natural Language Understanding (NLU) task, in which a machine must choose the answer to a question from a set of choices, with the question placed in context of text passages or dialog. In the last a couple of years the NLU field has been revolutionized with the advent of models based on the Transformer architecture, which are pretrained on massive amounts of unsupervised data and then fine-tuned for various supervised learning NLU tasks. Transformer models have come to dominate a wide variety of leader-boards in the NLU field; in the area of MRC, the current state-of-the-art model on the DREAM dataset (see[Sunet al., 2019]) fine tunes Albert, a large pretrained Transformer-based model, and addition-ally combines it with an extra layer of multi-head attention between context and question-answer[Zhuet al., 2020].The purpose of this note is to document a new state-of-the-art result in the DREAM task, which is accomplished by, additionally, performing multi-task learning on two MRC multi-choice reading comprehension tasks (RACE and DREAM).
摘要：多选机阅读理解（MRC）是一个重要而具有挑战性的自然语言理解（NLU）任务，在其中一台机器必须选择的答案，从一组的选择问题，放置在文本段落的背景问题或对话框。在过去的几年中NLU领域发生了革命性的具有基于变压器的架构模型，这是预先训练上大量的无监督的数据，然后微调各种监督学习NLU任务的到来。变压器型号有来主宰各种各样的自然语言理解领域的佼佼者，板;在MRC的区域中，当前状态的最先进的模型上DREAM数据集（见[SUNET人，2019]）精细调谐阿尔伯特，大预训练的变压器为基础的模型，并加入烯丙基与结合它的上下文和问答之间多头注意额外层[Zhuet人，2020]本说明的目的.The在DREAM任务，这是通过以下步骤完成记录一个新的国家的最先进的结果，另外，两个MRC多选择阅读理解任务执行多任务学习（RACE和DREAM）。
11. Unsupervised and Interpretable Domain Adaptation to Rapidly Filter Social Web Data for Emergency Services [PDF] 返回目录
Jitin Krishnan, Hemant Purohit, Huzefa Rangwala
Abstract: During the onset of a disaster event, filtering relevant information from the social web data is challenging due to its sparse availability and practical limitations in labeling datasets of an ongoing crisis. In this paper, we show that unsupervised domain adaptation through multi-task learning can be a useful framework to leverage data from past crisis events, as well as exploit additional web resources for training efficient information filtering models during an ongoing crisis. We present a novel method to classify relevant tweets during an ongoing crisis without seeing any new examples, using the publicly available dataset of TREC incident streams that provides labeled tweets with 4 relevant classes across 10 different crisis events. Additionally, our method addresses a crucial but missing component from current research in web science for crisis data filtering models: interpretability. Specifically, we first identify a standard single-task attention-based neural network architecture and then construct a customized multi-task architecture for the crisis domain: Multi-Task Domain Adversarial Attention Network. This model consists of dedicated attention layers for each task and a domain classifier for gradient reversal. Evaluation of domain adaptation for crisis events is performed by choosing a target event as the test set and training on the rest. Our results show that the multi-task model outperformed its single-task counterpart and also, training with additional web-resources showed further performance boost. Furthermore, we show that the attention layer can be used as a guide to explain the model predictions by showcasing the words in a tweet that are deemed important in the classification process. Our research aims to pave the way towards a fully unsupervised and interpretable domain adaptation of low-resource crisis web data to aid emergency responders quickly and effectively.
摘要：在灾难事件的发生，过滤来自社交网络的数据相关的信息，由于其稀疏的可用性和贴标的持续危机的数据集的实际限制挑战。在本文中，我们表明，通过多任务学习无监督领域适应性可以从过去的危机事件利用数据，一个有用的框架，以及开发更多的网络资源为持续危机中培养高效的信息过滤模型。我们正在进行的金融危机期间提出了一种新的方法来分类相关的微博没有看到任何新的例子，使用提供了跨10个不同的危机事件标记的鸣叫与4相关的类TREC事件流的可公开获得的数据集。此外，我们的方法解决了在网络科技目前的研究危机数据过滤模型的关键，但缺少的组成部分：可解释性。具体而言，我们首先确定一个基于标准的关注单任务的神经网络结构，然后构建危机域定制的多任务体系结构：多任务域对抗性关注网络。该模型由专用的关注层为每个任务和梯度反转中域分类的。领域适应性危机事件的评估进行选择目标事件作为试验组和其余的训练中表现。我们的研究结果表明，多任务模式优于其单任务的对应也，培训额外的网络资源，进一步表现出的性能提升。此外，我们表明，关注层可以作为一个导游展示被认为重要的，在分类过程中鸣叫的话来解释的模型预测。我们的研究旨在铺平道路向低资源危机Web数据的完全无监督和解释域适应援助紧急救援人员迅速和有效。
Jitin Krishnan, Hemant Purohit, Huzefa Rangwala
Abstract: During the onset of a disaster event, filtering relevant information from the social web data is challenging due to its sparse availability and practical limitations in labeling datasets of an ongoing crisis. In this paper, we show that unsupervised domain adaptation through multi-task learning can be a useful framework to leverage data from past crisis events, as well as exploit additional web resources for training efficient information filtering models during an ongoing crisis. We present a novel method to classify relevant tweets during an ongoing crisis without seeing any new examples, using the publicly available dataset of TREC incident streams that provides labeled tweets with 4 relevant classes across 10 different crisis events. Additionally, our method addresses a crucial but missing component from current research in web science for crisis data filtering models: interpretability. Specifically, we first identify a standard single-task attention-based neural network architecture and then construct a customized multi-task architecture for the crisis domain: Multi-Task Domain Adversarial Attention Network. This model consists of dedicated attention layers for each task and a domain classifier for gradient reversal. Evaluation of domain adaptation for crisis events is performed by choosing a target event as the test set and training on the rest. Our results show that the multi-task model outperformed its single-task counterpart and also, training with additional web-resources showed further performance boost. Furthermore, we show that the attention layer can be used as a guide to explain the model predictions by showcasing the words in a tweet that are deemed important in the classification process. Our research aims to pave the way towards a fully unsupervised and interpretable domain adaptation of low-resource crisis web data to aid emergency responders quickly and effectively.
摘要：在灾难事件的发生，过滤来自社交网络的数据相关的信息，由于其稀疏的可用性和贴标的持续危机的数据集的实际限制挑战。在本文中，我们表明，通过多任务学习无监督领域适应性可以从过去的危机事件利用数据，一个有用的框架，以及开发更多的网络资源为持续危机中培养高效的信息过滤模型。我们正在进行的金融危机期间提出了一种新的方法来分类相关的微博没有看到任何新的例子，使用提供了跨10个不同的危机事件标记的鸣叫与4相关的类TREC事件流的可公开获得的数据集。此外，我们的方法解决了在网络科技目前的研究危机数据过滤模型的关键，但缺少的组成部分：可解释性。具体而言，我们首先确定一个基于标准的关注单任务的神经网络结构，然后构建危机域定制的多任务体系结构：多任务域对抗性关注网络。该模型由专用的关注层为每个任务和梯度反转中域分类的。领域适应性危机事件的评估进行选择目标事件作为试验组和其余的训练中表现。我们的研究结果表明，多任务模式优于其单任务的对应也，培训额外的网络资源，进一步表现出的性能提升。此外，我们表明，关注层可以作为一个导游展示被认为重要的，在分类过程中鸣叫的话来解释的模型预测。我们的研究旨在铺平道路向低资源危机Web数据的完全无监督和解释域适应援助紧急救援人员迅速和有效。
12. ScopeIt: Scoping Task Relevant Sentences in Documents [PDF] 返回目录
Vishwas Suryanarayanan, Barun Patra, Pamela Bhattacharya, Chala Fufa, Charles Lee
Abstract: Intelligent assistants like Cortana, Siri, Alexa, and Google Assistant are trained to parse information when the conversation is synchronous and short; however, for email-based conversational agents, the communication is asynchronous, and often contains information irrelevant to the assistant. This makes it harder for the system to accurately detect intents, extract entities relevant to those intents and thereby perform the desired action. We present a neural model for scoping relevant information for the agent from a large query. We show that when used as a preprocessing step, the model improves performance of both intent detection and entity extraction tasks. We demonstrate the model's impact on Scheduler (Cortana is the persona of the agent, while Scheduler is the name of the service. We use them interchangeably in the context of this paper.) - a virtual conversational meeting scheduling assistant that interacts asynchronously with users through email. The model helps the entity extraction and intent detection tasks requisite by Scheduler achieve an average gain of 35% in precision without any drop in recall. Additionally, we demonstrate that the same approach can be used for component level analysis in large documents, such as signature block identification.
摘要：柯塔娜一样，Siri的，Alexa的，和谷歌智能助理助理进行培训，以当会话是同步的，短解析信息;然而，对于基于电子邮件的会话代理，通信是异步的，而且往往包含无关的辅助信息。这使得难以对系统准确地检测意图应符合那些意图提取实体和从而执行所需的操作。我们提出了从大型查询范围界定为代理相关信息的神经网络模型。我们表明，作为前工序中使用时，该模型提高了双方的意图检测和实体提取任务中的表现。我们证明该模型对调度的影响（柯塔娜是代理的角色，而计划是服务的名称，我们在本文的背景下可以互换使用。） - 虚拟对话会议安排助理进行交互以异步方式与用户通过电子邮件。该模型有助于实体提取和意图检测任务由调度必要实现精密35％没有召回任何下降，平均升幅。此外，我们证明了同样的方法可在大的文档，诸如签名块识别被用于组件级分析。
Vishwas Suryanarayanan, Barun Patra, Pamela Bhattacharya, Chala Fufa, Charles Lee
Abstract: Intelligent assistants like Cortana, Siri, Alexa, and Google Assistant are trained to parse information when the conversation is synchronous and short; however, for email-based conversational agents, the communication is asynchronous, and often contains information irrelevant to the assistant. This makes it harder for the system to accurately detect intents, extract entities relevant to those intents and thereby perform the desired action. We present a neural model for scoping relevant information for the agent from a large query. We show that when used as a preprocessing step, the model improves performance of both intent detection and entity extraction tasks. We demonstrate the model's impact on Scheduler (Cortana is the persona of the agent, while Scheduler is the name of the service. We use them interchangeably in the context of this paper.) - a virtual conversational meeting scheduling assistant that interacts asynchronously with users through email. The model helps the entity extraction and intent detection tasks requisite by Scheduler achieve an average gain of 35% in precision without any drop in recall. Additionally, we demonstrate that the same approach can be used for component level analysis in large documents, such as signature block identification.
摘要：柯塔娜一样，Siri的，Alexa的，和谷歌智能助理助理进行培训，以当会话是同步的，短解析信息;然而，对于基于电子邮件的会话代理，通信是异步的，而且往往包含无关的辅助信息。这使得难以对系统准确地检测意图应符合那些意图提取实体和从而执行所需的操作。我们提出了从大型查询范围界定为代理相关信息的神经网络模型。我们表明，作为前工序中使用时，该模型提高了双方的意图检测和实体提取任务中的表现。我们证明该模型对调度的影响（柯塔娜是代理的角色，而计划是服务的名称，我们在本文的背景下可以互换使用。） - 虚拟对话会议安排助理进行交互以异步方式与用户通过电子邮件。该模型有助于实体提取和意图检测任务由调度必要实现精密35％没有召回任何下降，平均升幅。此外，我们证明了同样的方法可在大的文档，诸如签名块识别被用于组件级分析。
13. A Financial Service Chatbot based on Deep Bidirectional Transformers [PDF] 返回目录
Shi Yu, Yuxin Chen, Hussain Zaidi
Abstract: We develop a chatbot using Deep Bidirectional Transformer models (BERT) to handle client questions in financial investment customer service. The bot can recognize 381 intents, and decides when to say "I don't know" and escalates irrelevant/uncertain questions to human operators. Our main novel contribution is the discussion about uncertainty measure for BERT, where three different approaches are systematically compared on real problems. We investigated two uncertainty metrics, information entropy and variance of dropout sampling in BERT, followed by mixed-integer programming to optimize decision thresholds. Another novel contribution is the usage of BERT as a language model in automatic spelling correction. Inputs with accidental spelling errors can significantly decrease intent classification performance. The proposed approach combines probabilities from masked language model and word edit distances to find the best corrections for misspelled words. The chatbot and the entire conversational AI system are developed using open-source tools, and deployed within our company's intranet. The proposed approach can be useful for industries seeking similar in-house solutions in their specific business domains. We share all our code and a sample chatbot built on a public dataset on Github.
摘要：我们使用Deep双向Transformer模型（BERT）在金融投资客户服务处理客户的问题制定一个聊天机器人。该机器人可以识别381和意图，并决定何时说“我不知道”，并升级为人工操作无关的/不确定性的问题。我们的主要小说的贡献大约是不确定性度量BERT，在三种不同的方法进行了系统的实际问题相比较的讨论。我们研究2度的不确定性的度量，信息熵和在BERT差采样方差，接着混合整数规划来优化决策阈值。另一个新颖的贡献是BERT的使用如在自动拼写校正的语言模型。有偶然的拼写错误输入可以显著降低意图分类性能。从蒙面语言模型和文字编辑距离所提出的方法结合概率找到拼错的单词最好的更正。该聊天机器人和整个对话的AI系统正在使用开源工具的发展，以及我们公司的内部网中部署。所提出的方法可以为寻求在其特定的业务领域类似的内部解决方案的行业非常有用。我们分享我们所有的代码，并建立在Github上的公开数据集的样本聊天机器人。
Shi Yu, Yuxin Chen, Hussain Zaidi
Abstract: We develop a chatbot using Deep Bidirectional Transformer models (BERT) to handle client questions in financial investment customer service. The bot can recognize 381 intents, and decides when to say "I don't know" and escalates irrelevant/uncertain questions to human operators. Our main novel contribution is the discussion about uncertainty measure for BERT, where three different approaches are systematically compared on real problems. We investigated two uncertainty metrics, information entropy and variance of dropout sampling in BERT, followed by mixed-integer programming to optimize decision thresholds. Another novel contribution is the usage of BERT as a language model in automatic spelling correction. Inputs with accidental spelling errors can significantly decrease intent classification performance. The proposed approach combines probabilities from masked language model and word edit distances to find the best corrections for misspelled words. The chatbot and the entire conversational AI system are developed using open-source tools, and deployed within our company's intranet. The proposed approach can be useful for industries seeking similar in-house solutions in their specific business domains. We share all our code and a sample chatbot built on a public dataset on Github.
摘要：我们使用Deep双向Transformer模型（BERT）在金融投资客户服务处理客户的问题制定一个聊天机器人。该机器人可以识别381和意图，并决定何时说“我不知道”，并升级为人工操作无关的/不确定性的问题。我们的主要小说的贡献大约是不确定性度量BERT，在三种不同的方法进行了系统的实际问题相比较的讨论。我们研究2度的不确定性的度量，信息熵和在BERT差采样方差，接着混合整数规划来优化决策阈值。另一个新颖的贡献是BERT的使用如在自动拼写校正的语言模型。有偶然的拼写错误输入可以显著降低意图分类性能。从蒙面语言模型和文字编辑距离所提出的方法结合概率找到拼错的单词最好的更正。该聊天机器人和整个对话的AI系统正在使用开源工具的发展，以及我们公司的内部网中部署。所提出的方法可以为寻求在其特定的业务领域类似的内部解决方案的行业非常有用。我们分享我们所有的代码，并建立在Github上的公开数据集的样本聊天机器人。
14. Investigating an approach for low resource language dataset creation, curation and classification: Setswana and Sepedi [PDF] 返回目录
Vukosi Marivate, Tshephisho Sefara, Vongani Chabalala, Keamogetswe Makhaya, Tumisho Mokgonyane, Rethabile Mokoena, Abiodun Modupe
Abstract: The recent advances in Natural Language Processing have been a boon for well-represented languages in terms of available curated data and research resources. One of the challenges for low-resourced languages is clear guidelines on the collection, curation and preparation of datasets for different use-cases. In this work, we take on the task of creation of two datasets that are focused on news headlines (i.e short text) for Setswana and Sepedi and creation of a news topic classification task. We document our work and also present baselines for classification. We investigate an approach on data augmentation, better suited to low resource languages, to improve the performance of the classifiers
摘要：在自然语言处理的最新进展已经在策划提供的数据和研究资源方面一直是很好的体现语言的福音。一个低资源语言所面临的挑战是在收集，策展和准备数据集用于不同的使用情况的明确的指导方针。在这项工作中，我们采取的创造，专注于新闻标题两个数据集的对茨瓦纳语和塞佩蒂语的新闻主题分类任务的任务（即短文本）和创造。我们记录我们的工作，也存在分类基准。我们调查的数据隆胸的方法，更适合于低资源语言，以提高分类器的性能
Vukosi Marivate, Tshephisho Sefara, Vongani Chabalala, Keamogetswe Makhaya, Tumisho Mokgonyane, Rethabile Mokoena, Abiodun Modupe
Abstract: The recent advances in Natural Language Processing have been a boon for well-represented languages in terms of available curated data and research resources. One of the challenges for low-resourced languages is clear guidelines on the collection, curation and preparation of datasets for different use-cases. In this work, we take on the task of creation of two datasets that are focused on news headlines (i.e short text) for Setswana and Sepedi and creation of a news topic classification task. We document our work and also present baselines for classification. We investigate an approach on data augmentation, better suited to low resource languages, to improve the performance of the classifiers
摘要：在自然语言处理的最新进展已经在策划提供的数据和研究资源方面一直是很好的体现语言的福音。一个低资源语言所面临的挑战是在收集，策展和准备数据集用于不同的使用情况的明确的指导方针。在这项工作中，我们采取的创造，专注于新闻标题两个数据集的对茨瓦纳语和塞佩蒂语的新闻主题分类任务的任务（即短文本）和创造。我们记录我们的工作，也存在分类基准。我们调查的数据隆胸的方法，更适合于低资源语言，以提高分类器的性能
15. Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT [PDF] 返回目录
Lichao Sun, Kazuma Hashimoto, Wenpeng Yin, Akari Asai, Jia Li, Philip Yu, Caiming Xiong
Abstract: There is an increasing amount of literature that claims the brittleness of deep neural networks in dealing with adversarial examples that are created maliciously. It is unclear, however, how the models will perform in realistic scenarios where \textit{natural rather than malicious} adversarial instances often exist. This work systematically explores the robustness of BERT, the state-of-the-art Transformer-style model in NLP, in dealing with noisy data, particularly mistakes in typing the keyboard, that occur inadvertently. Intensive experiments on sentiment analysis and question answering benchmarks indicate that: (i) Typos in various words of a sentence do not influence equally. The typos in informative words make severer damages; (ii) Mistype is the most damaging factor, compared with inserting, deleting, etc.; (iii) Humans and machines have different focuses on recognizing adversarial attacks.
摘要：是要求深神经网络的脆性在处理被恶意创建对抗性例子文献的量增加。目前还不清楚，但是，该机型将如何在现实场景中\ textit {自然，而不是恶意}对抗性的情况下，往往存在执行。这项工作系统探讨BERT，在NLP的国家的最先进的变压器式模型的鲁棒性，在处理噪声数据，在敲击键盘特别的错误，无意之中发生。在情感分析和答疑基准密集的实验表明：（i）在句子的各个单词拼写错误不同样影响。在翔实的文字的错别字让严厉的赔偿金; （ⅱ）输错是最具破坏性的因素，插入，删除等;相比（三）人类和机器有不同的侧重点上承认对抗性攻击。
Lichao Sun, Kazuma Hashimoto, Wenpeng Yin, Akari Asai, Jia Li, Philip Yu, Caiming Xiong
Abstract: There is an increasing amount of literature that claims the brittleness of deep neural networks in dealing with adversarial examples that are created maliciously. It is unclear, however, how the models will perform in realistic scenarios where \textit{natural rather than malicious} adversarial instances often exist. This work systematically explores the robustness of BERT, the state-of-the-art Transformer-style model in NLP, in dealing with noisy data, particularly mistakes in typing the keyboard, that occur inadvertently. Intensive experiments on sentiment analysis and question answering benchmarks indicate that: (i) Typos in various words of a sentence do not influence equally. The typos in informative words make severer damages; (ii) Mistype is the most damaging factor, compared with inserting, deleting, etc.; (iii) Humans and machines have different focuses on recognizing adversarial attacks.
摘要：是要求深神经网络的脆性在处理被恶意创建对抗性例子文献的量增加。目前还不清楚，但是，该机型将如何在现实场景中\ textit {自然，而不是恶意}对抗性的情况下，往往存在执行。这项工作系统探讨BERT，在NLP的国家的最先进的变压器式模型的鲁棒性，在处理噪声数据，在敲击键盘特别的错误，无意之中发生。在情感分析和答疑基准密集的实验表明：（i）在句子的各个单词拼写错误不同样影响。在翔实的文字的错别字让严厉的赔偿金; （ⅱ）输错是最具破坏性的因素，插入，删除等;相比（三）人类和机器有不同的侧重点上承认对抗性攻击。
16. Understanding the Downstream Instability of Word Embeddings [PDF] 返回目录
Megan Leszczynski, Avner May, Jian Zhang, Sen Wu, Christopher R. Aberger, Christopher Ré
Abstract: Many industrial machine learning (ML) systems require frequent retraining to keep up-to-date with constantly changing data. This retraining exacerbates a large challenge facing ML systems today: model training is unstable, i.e., small changes in training data can cause significant changes in the model's predictions. In this paper, we work on developing a deeper understanding of this instability, with a focus on how a core building block of modern natural language processing (NLP) pipelines---pre-trained word embeddings---affects the instability of downstream NLP models. We first empirically reveal a tradeoff between stability and memory: increasing the embedding memory 2x can reduce the disagreement in predictions due to small changes in training data by 5% to 37% (relative). To theoretically explain this tradeoff, we introduce a new measure of embedding instability---the eigenspace instability measure---which we prove bounds the disagreement in downstream predictions introduced by the change in word embeddings. Practically, we show that the eigenspace instability measure can be a cost-effective way to choose embedding parameters to minimize instability without training downstream models, outperforming other embedding distance measures and performing competitively with a nearest neighbor-based measure. Finally, we demonstrate that the observed stability-memory tradeoffs extend to other types of embeddings as well, including knowledge graph and contextual word embeddings.
摘要：许多工业机器学习（ML）系统需要频繁的再培训，以跟上最新与不断变化的数据。这再培训加剧了面临ML系统大挑战今天：模型训练是不稳定的，即，在训练数据的微小变化会导致模型的预测显著的变化。在本文中，我们开发这个不稳定的一个更深入的了解工作，重点是如何现代自然语言处理（NLP）管道的核心组成部分---预先训练字的嵌入---影响下游NLP的不稳定性楷模。我们首先凭经验揭示稳定性和存储器之间的折衷：增加嵌入存储器2X可以降低预测不一致由于在训练数据的小变化由5％至37％（相对）。从理论上解释了这种权衡，我们引入嵌入一个新的不稳定措施---固有空间不稳定的措施---这证明边界在由字的嵌入的变化引入下游的预测分歧。实际上，我们证明了固有空间不稳定的措施可以是一个具有成本效益的方式来选择嵌入参数没有训练下游模型，优于其他嵌入距离措施，并具有最近基于邻居措施进行竞争减少不稳定。最后，我们证明，所观察到的稳定性存储器权衡延伸到其它类型的嵌入的为好，包括知识图表和上下文字的嵌入。
Megan Leszczynski, Avner May, Jian Zhang, Sen Wu, Christopher R. Aberger, Christopher Ré
Abstract: Many industrial machine learning (ML) systems require frequent retraining to keep up-to-date with constantly changing data. This retraining exacerbates a large challenge facing ML systems today: model training is unstable, i.e., small changes in training data can cause significant changes in the model's predictions. In this paper, we work on developing a deeper understanding of this instability, with a focus on how a core building block of modern natural language processing (NLP) pipelines---pre-trained word embeddings---affects the instability of downstream NLP models. We first empirically reveal a tradeoff between stability and memory: increasing the embedding memory 2x can reduce the disagreement in predictions due to small changes in training data by 5% to 37% (relative). To theoretically explain this tradeoff, we introduce a new measure of embedding instability---the eigenspace instability measure---which we prove bounds the disagreement in downstream predictions introduced by the change in word embeddings. Practically, we show that the eigenspace instability measure can be a cost-effective way to choose embedding parameters to minimize instability without training downstream models, outperforming other embedding distance measures and performing competitively with a nearest neighbor-based measure. Finally, we demonstrate that the observed stability-memory tradeoffs extend to other types of embeddings as well, including knowledge graph and contextual word embeddings.
摘要：许多工业机器学习（ML）系统需要频繁的再培训，以跟上最新与不断变化的数据。这再培训加剧了面临ML系统大挑战今天：模型训练是不稳定的，即，在训练数据的微小变化会导致模型的预测显著的变化。在本文中，我们开发这个不稳定的一个更深入的了解工作，重点是如何现代自然语言处理（NLP）管道的核心组成部分---预先训练字的嵌入---影响下游NLP的不稳定性楷模。我们首先凭经验揭示稳定性和存储器之间的折衷：增加嵌入存储器2X可以降低预测不一致由于在训练数据的小变化由5％至37％（相对）。从理论上解释了这种权衡，我们引入嵌入一个新的不稳定措施---固有空间不稳定的措施---这证明边界在由字的嵌入的变化引入下游的预测分歧。实际上，我们证明了固有空间不稳定的措施可以是一个具有成本效益的方式来选择嵌入参数没有训练下游模型，优于其他嵌入距离措施，并具有最近基于邻居措施进行竞争减少不稳定。最后，我们证明，所观察到的稳定性存储器权衡延伸到其它类型的嵌入的为好，包括知识图表和上下文字的嵌入。
17. SAFE: Similarity-Aware Multi-Modal Fake News Detection [PDF] 返回目录
Xinyi Zhou, Jindi Wu, Reza Zafarani
Abstract: Effective detection of fake news has recently attracted significant attention. Current studies have made significant contributions to predicting fake news with less focus on exploiting the relationship (similarity) between the textual and visual information in news articles. Attaching importance to such similarity helps identify fake news stories that, for example, attempt to use irrelevant images to attract readers' attention. In this work, we propose a $\mathsf{S}$imilarity-$\mathsf{A}$ware $\mathsf{F}$ak$\mathsf{E}$ news detection method ($\mathsf{SAFE}$) which investigates multi-modal (textual and visual) information of news articles. First, neural networks are adopted to separately extract textual and visual features for news representation. We further investigate the relationship between the extracted features across modalities. Such representations of news textual and visual information along with their relationship are jointly learned and used to predict fake news. The proposed method facilitates recognizing the falsity of news articles based on their text, images, or their "mismatches." We conduct extensive experiments on large-scale real-world data, which demonstrate the effectiveness of the proposed method.
摘要：假新闻有效检测近来也备受关注显著。目前的研究已经取得对利用新闻文章的文字和视觉信息之间的关系（相似）预测的假新闻，较少关注显著的贡献。附加到这种相似的重要性有助于识别假新闻，例如，尝试使用不相关的图片来吸引读者的注意力。在这项工作中，我们提出了$ \ mathsf {S} $ imilarity - $ \ mathsf {A} $洁具$ \ mathsf {F} $ AK $ \ mathsf {E} $消息的检测方法（$ \ mathsf {安全} $ ），负责调查的多模式（文本和新闻文章的视觉）的信息。首先，神经网络采用单独提取文本和视觉特征的新闻表示。我们进一步研究横跨方式所提取的特征之间的关系。与他们的关系以及消息的文本和视觉信息的申述共同学习和使用预测的假新闻。该方法有利于认识新闻报道的基础上他们的文本，图像，或他们是虚假的“错配”。我们开展的大型真实世界的数据，这证明了该方法的有效性大量的实验。
Xinyi Zhou, Jindi Wu, Reza Zafarani
Abstract: Effective detection of fake news has recently attracted significant attention. Current studies have made significant contributions to predicting fake news with less focus on exploiting the relationship (similarity) between the textual and visual information in news articles. Attaching importance to such similarity helps identify fake news stories that, for example, attempt to use irrelevant images to attract readers' attention. In this work, we propose a $\mathsf{S}$imilarity-$\mathsf{A}$ware $\mathsf{F}$ak$\mathsf{E}$ news detection method ($\mathsf{SAFE}$) which investigates multi-modal (textual and visual) information of news articles. First, neural networks are adopted to separately extract textual and visual features for news representation. We further investigate the relationship between the extracted features across modalities. Such representations of news textual and visual information along with their relationship are jointly learned and used to predict fake news. The proposed method facilitates recognizing the falsity of news articles based on their text, images, or their "mismatches." We conduct extensive experiments on large-scale real-world data, which demonstrate the effectiveness of the proposed method.
摘要：假新闻有效检测近来也备受关注显著。目前的研究已经取得对利用新闻文章的文字和视觉信息之间的关系（相似）预测的假新闻，较少关注显著的贡献。附加到这种相似的重要性有助于识别假新闻，例如，尝试使用不相关的图片来吸引读者的注意力。在这项工作中，我们提出了$ \ mathsf {S} $ imilarity - $ \ mathsf {A} $洁具$ \ mathsf {F} $ AK $ \ mathsf {E} $消息的检测方法（$ \ mathsf {安全} $ ），负责调查的多模式（文本和新闻文章的视觉）的信息。首先，神经网络采用单独提取文本和视觉特征的新闻表示。我们进一步研究横跨方式所提取的特征之间的关系。与他们的关系以及消息的文本和视觉信息的申述共同学习和使用预测的假新闻。该方法有利于认识新闻报道的基础上他们的文本，图像，或他们是虚假的“错配”。我们开展的大型真实世界的数据，这证明了该方法的有效性大量的实验。
18. Improving Reliability of Latent Dirichlet Allocation by Assessing Its Stability Using Clustering Techniques on Replicated Runs [PDF] 返回目录
Jonas Rieger, Lars Koppers, Carsten Jentsch, Jörg Rahnenführer
Abstract: For organizing large text corpora topic modeling provides useful tools. A widely used method is Latent Dirichlet Allocation (LDA), a generative probabilistic model which models single texts in a collection of texts as mixtures of latent topics. The assignments of words to topics rely on initial values such that generally the outcome of LDA is not fully reproducible. In addition, the reassignment via Gibbs Sampling is based on conditional distributions, leading to different results in replicated runs on the same text data. This fact is often neglected in everyday practice. We aim to improve the reliability of LDA results. Therefore, we study the stability of LDA by comparing assignments from replicated runs. We propose to quantify the similarity of two generated topics by a modified Jaccard coefficient. Using such similarities, topics can be clustered. A new pruning algorithm for hierarchical clustering results based on the idea that two LDA runs create pairs of similar topics is proposed. This approach leads to the new measure S-CLOP ({\bf S}imilarity of multiple sets by {\bf C}lustering with {\bf LO}cal {\bf P}runing) for quantifying the stability of LDA models. We discuss some characteristics of this measure and illustrate it with an application to real data consisting of newspaper articles from \textit{USA Today}. Our results show that the measure S-CLOP is useful for assessing the stability of LDA models or any other topic modeling procedure that characterize its topics by word distributions. Based on the newly proposed measure for LDA stability, we propose a method to increase the reliability and hence to improve the reproducibility of empirical findings based on topic modeling. This increase in reliability is obtained by running the LDA several times and taking as prototype the most representative run, that is the LDA run with highest average similarity to all other runs.
摘要：对于举办大型语料库主题建模提供了有用的工具。一种广泛使用的方法是隐含狄利克雷分布（LDA），一种生成概率模型哪些文本的集合作为潜在主题的混合物在模型单个文本。词的主题的分配依赖于初始值，使得一般LDA的结果是不完全重复的。此外，通过Gibbs抽样重新分配是基于条件分布，导致重复运行不同的结果对同一文本数据。这一事实往往被忽视，在日常实践中。我们的目标是改善LDA结果的可靠性。因此，我们通过复制运行比较作业学习LDA的稳定性。我们建议通过修改杰卡德系数量化生成的两个话题的相似性。使用这样的相似，主题可以被集群。基于这个想法有两个LDA运行创造对类似主题的聚类结果的新的修剪算法。这种做法导致新措施S-CLOP（{\ BF S} {由\ BFÇ}与上光{\ BF LO} {CAL \ BF P}乳宁多套imilarity）量化模型LDA的稳定性。我们讨论这项措施的一些特点，并说明它与一个应用程序从\ {textit今日美国}由报纸文章的真实数据。我们的研究结果表明，该措施S-CLOP是评估LDA模型或通过字分布表征其主题其他任何主题建模过程的稳定性。基于对LDA稳定性新近提出的措施，我们建议提高可靠性，从而改善基于主题建模实证结果的可重复性的方法。这增加了可靠性，通过运行LDA几次，并采取为原型的最具代表性的运行，即LDA运行具有最高的平均相似的所有其他运行中获得的。
Jonas Rieger, Lars Koppers, Carsten Jentsch, Jörg Rahnenführer
Abstract: For organizing large text corpora topic modeling provides useful tools. A widely used method is Latent Dirichlet Allocation (LDA), a generative probabilistic model which models single texts in a collection of texts as mixtures of latent topics. The assignments of words to topics rely on initial values such that generally the outcome of LDA is not fully reproducible. In addition, the reassignment via Gibbs Sampling is based on conditional distributions, leading to different results in replicated runs on the same text data. This fact is often neglected in everyday practice. We aim to improve the reliability of LDA results. Therefore, we study the stability of LDA by comparing assignments from replicated runs. We propose to quantify the similarity of two generated topics by a modified Jaccard coefficient. Using such similarities, topics can be clustered. A new pruning algorithm for hierarchical clustering results based on the idea that two LDA runs create pairs of similar topics is proposed. This approach leads to the new measure S-CLOP ({\bf S}imilarity of multiple sets by {\bf C}lustering with {\bf LO}cal {\bf P}runing) for quantifying the stability of LDA models. We discuss some characteristics of this measure and illustrate it with an application to real data consisting of newspaper articles from \textit{USA Today}. Our results show that the measure S-CLOP is useful for assessing the stability of LDA models or any other topic modeling procedure that characterize its topics by word distributions. Based on the newly proposed measure for LDA stability, we propose a method to increase the reliability and hence to improve the reproducibility of empirical findings based on topic modeling. This increase in reliability is obtained by running the LDA several times and taking as prototype the most representative run, that is the LDA run with highest average similarity to all other runs.
摘要：对于举办大型语料库主题建模提供了有用的工具。一种广泛使用的方法是隐含狄利克雷分布（LDA），一种生成概率模型哪些文本的集合作为潜在主题的混合物在模型单个文本。词的主题的分配依赖于初始值，使得一般LDA的结果是不完全重复的。此外，通过Gibbs抽样重新分配是基于条件分布，导致重复运行不同的结果对同一文本数据。这一事实往往被忽视，在日常实践中。我们的目标是改善LDA结果的可靠性。因此，我们通过复制运行比较作业学习LDA的稳定性。我们建议通过修改杰卡德系数量化生成的两个话题的相似性。使用这样的相似，主题可以被集群。基于这个想法有两个LDA运行创造对类似主题的聚类结果的新的修剪算法。这种做法导致新措施S-CLOP（{\ BF S} {由\ BFÇ}与上光{\ BF LO} {CAL \ BF P}乳宁多套imilarity）量化模型LDA的稳定性。我们讨论这项措施的一些特点，并说明它与一个应用程序从\ {textit今日美国}由报纸文章的真实数据。我们的研究结果表明，该措施S-CLOP是评估LDA模型或通过字分布表征其主题其他任何主题建模过程的稳定性。基于对LDA稳定性新近提出的措施，我们建议提高可靠性，从而改善基于主题建模实证结果的可重复性的方法。这增加了可靠性，通过运行LDA几次，并采取为原型的最具代表性的运行，即LDA运行具有最高的平均相似的所有其他运行中获得的。
19. Fake News Detection with Different Models [PDF] 返回目录
Sairamvinay Vijayaraghavan, Ye Wang, Zhiyuan Guo, John Voong, Wenda Xu, Armand Nasseri, Jiaru Cai, Linda Li, Kevin Vuong, Eshan Wadhwa
Abstract: This is a paper for exploring various different models aiming at developing fake news detection models and we had used certain machine learning algorithms and we had used pretrained algorithms such as TFIDF and CV and W2V as features for processing textual data.
摘要：这是一个探索各种不同型号旨在开发假新闻检测模型一张纸，我们已经使用了一定的机器学习算法和我们使用预训练的算法，如TFIDF和CV和W2V的功能，用于处理文本数据。
Sairamvinay Vijayaraghavan, Ye Wang, Zhiyuan Guo, John Voong, Wenda Xu, Armand Nasseri, Jiaru Cai, Linda Li, Kevin Vuong, Eshan Wadhwa
Abstract: This is a paper for exploring various different models aiming at developing fake news detection models and we had used certain machine learning algorithms and we had used pretrained algorithms such as TFIDF and CV and W2V as features for processing textual data.
摘要：这是一个探索各种不同型号旨在开发假新闻检测模型一张纸，我们已经使用了一定的机器学习算法和我们使用预训练的算法，如TFIDF和CV和W2V的功能，用于处理文本数据。
20. Mask & Focus: Conversation Modelling by Learning Concepts [PDF] 返回目录
Gaurav Pandey, Dinesh Raghu, Sachindra Joshi
Abstract: Sequence to sequence models attempt to capture the correlation between all the words in the input and output sequences. While this is quite useful for machine translation where the correlation among the words is indeed quite strong, it becomes problematic for conversation modelling where the correlation is often at a much abstract level. In contrast, humans tend to focus on the essential concepts discussed in the conversation context and generate responses accordingly. In this paper, we attempt to mimic this response generating mechanism by learning the essential concepts in the context and response in an unsupervised manner. The proposed model, referred to as Mask \& Focus maps the input context to a sequence of concepts which are then used to generate the response concepts. Together, the context and the response concepts generate the final response. In order to learn context concepts from the training data automatically, we \emph{mask} words in the input and observe the effect of masking on response generation. We train our model to learn those response concepts that have high mutual information with respect to the context concepts, thereby guiding the model to \emph{focus} on the context concepts. Mask \& Focus achieves significant improvement over the existing baselines in several established metrics for dialogues.
摘要：序列到序列模型试图捕捉在输入和输出序列的所有字之间的相关性。虽然这是机器翻译非常有用的哪里话之间的相关性确实是相当强的，它成为谈话造型，其中的相关性往往是在一个更抽象的层面上有问题。相比之下，人类往往把重点放在对话上下文中讨论的基本概念，并相应地产生反应。在本文中，我们试图通过学习在上下文和响应的基本概念以无监督方式模仿此响应产生机构。所提出的模型中，被称为掩码\＆聚焦映射输入上下文的概念，其然后用于产生响应的概念的序列。总之，上下文和响应的概念生成最终的响应。为了自动地从训练数据学习上下文的概念，我们\ EMPH在输入{掩模}单词和观察掩蔽上响应产生的效果。我们培训我们的模型，了解有关于上下文的概念高交互信息的响应的概念，从而模型引导到\ {EMPH重点}上下文的概念。面膜\＆焦点实现在现有基准显著的改善在几个对话建立的指标。
Gaurav Pandey, Dinesh Raghu, Sachindra Joshi
Abstract: Sequence to sequence models attempt to capture the correlation between all the words in the input and output sequences. While this is quite useful for machine translation where the correlation among the words is indeed quite strong, it becomes problematic for conversation modelling where the correlation is often at a much abstract level. In contrast, humans tend to focus on the essential concepts discussed in the conversation context and generate responses accordingly. In this paper, we attempt to mimic this response generating mechanism by learning the essential concepts in the context and response in an unsupervised manner. The proposed model, referred to as Mask \& Focus maps the input context to a sequence of concepts which are then used to generate the response concepts. Together, the context and the response concepts generate the final response. In order to learn context concepts from the training data automatically, we \emph{mask} words in the input and observe the effect of masking on response generation. We train our model to learn those response concepts that have high mutual information with respect to the context concepts, thereby guiding the model to \emph{focus} on the context concepts. Mask \& Focus achieves significant improvement over the existing baselines in several established metrics for dialogues.
摘要：序列到序列模型试图捕捉在输入和输出序列的所有字之间的相关性。虽然这是机器翻译非常有用的哪里话之间的相关性确实是相当强的，它成为谈话造型，其中的相关性往往是在一个更抽象的层面上有问题。相比之下，人类往往把重点放在对话上下文中讨论的基本概念，并相应地产生反应。在本文中，我们试图通过学习在上下文和响应的基本概念以无监督方式模仿此响应产生机构。所提出的模型中，被称为掩码\＆聚焦映射输入上下文的概念，其然后用于产生响应的概念的序列。总之，上下文和响应的概念生成最终的响应。为了自动地从训练数据学习上下文的概念，我们\ EMPH在输入{掩模}单词和观察掩蔽上响应产生的效果。我们培训我们的模型，了解有关于上下文的概念高交互信息的响应的概念，从而模型引导到\ {EMPH重点}上下文的概念。面膜\＆焦点实现在现有基准显著的改善在几个对话建立的指标。
21. A Computational Investigation on Denominalization [PDF] 返回目录
Zahra Shekarchi, Yang Xu
Abstract: Language has been a dynamic system and word meanings always have been changed over times. Every time a novel concept or sense is introduced, we need to assign it a word to express it. Also, some changes have happened because the result of a change can be more desirable for humans, or cognitively easier to be used by humans. Finding the patterns of these changes is interesting and can reveal some facts about human cognitive evolution. As we have enough resources for studying this problem, it is a good idea to work on the problem through computational modeling, and that can make the work easier and possible to be studied on large scale. In this work, we want to study the nouns which have been used as verbs after some years of their emergence as nouns and find some commonalities among these nouns. In other words, we are interested in finding what potential requirements are essential for this change.
摘要：语言一直是一个动态的系统和词义总是已经改变了时间。一个新的概念或意识引入每一次，我们需要为它分配一个字来表达它。此外，一些变化已经发生，因为变化的结果可能是对人类更理想，或认知更容易被人类使用。发现这些变化的模式很有趣，而且可以揭示人类认知进化的一些事实。因为我们有足够的资源来研究这个问题，这是一个好主意，通过计算模型对这个问题的工作，并可以使工作更容易，可能在大规模地进行研究。在这项工作中，我们要研究已经若干年后它们的出现为名词用作动词名词和发现这些名词中的一些共性。换句话说，我们有兴趣在寻找什么潜在需求是这种变化至关重要。
Zahra Shekarchi, Yang Xu
Abstract: Language has been a dynamic system and word meanings always have been changed over times. Every time a novel concept or sense is introduced, we need to assign it a word to express it. Also, some changes have happened because the result of a change can be more desirable for humans, or cognitively easier to be used by humans. Finding the patterns of these changes is interesting and can reveal some facts about human cognitive evolution. As we have enough resources for studying this problem, it is a good idea to work on the problem through computational modeling, and that can make the work easier and possible to be studied on large scale. In this work, we want to study the nouns which have been used as verbs after some years of their emergence as nouns and find some commonalities among these nouns. In other words, we are interested in finding what potential requirements are essential for this change.
摘要：语言一直是一个动态的系统和词义总是已经改变了时间。一个新的概念或意识引入每一次，我们需要为它分配一个字来表达它。此外，一些变化已经发生，因为变化的结果可能是对人类更理想，或认知更容易被人类使用。发现这些变化的模式很有趣，而且可以揭示人类认知进化的一些事实。因为我们有足够的资源来研究这个问题，这是一个好主意，通过计算模型对这个问题的工作，并可以使工作更容易，可能在大规模地进行研究。在这项工作中，我们要研究已经若干年后它们的出现为名词用作动词名词和发现这些名词中的一些共性。换句话说，我们有兴趣在寻找什么潜在需求是这种变化至关重要。
22. Transformer++ [PDF] 返回目录
Prakhar Thapak, Prodip Hore
Abstract: Recent advancements in attention mechanisms have replaced recurrent neural networks and its variants for machine translation tasks. Transformer using attention mechanism solely achieved state-of-the-art results in sequence modeling. Neural machine translation based on the attention mechanism is parallelizable and addresses the problem of handling long-range dependencies among words in sentences more effectively than recurrent neural networks. One of the key concepts in attention is to learn three matrices, query, key, and value, where global dependencies among words are learned through linearly projecting word embeddings through these matrices. Multiple query, key, value matrices can be learned simultaneously focusing on a different subspace of the embedded dimension, which is called multi-head in Transformer. We argue that certain dependencies among words could be learned better through an intermediate context than directly modeling word-word dependencies. This could happen due to the nature of certain dependencies or lack of patterns that lend them difficult to be modeled globally using multi-head self-attention. In this work, we propose a new way of learning dependencies through a context in multi-head using convolution. This new form of multi-head attention along with the traditional form achieves better results than Transformer on the WMT 2014 English-to-German and English-to-French translation tasks. We also introduce a framework to learn POS tagging and NER information during the training of encoder which further improves results achieving a new state-of-the-art of 32.1 BLEU, better than existing best by 1.4 BLEU, on the WMT 2014 English-to-German and 44.6 BLEU, better than existing best by 1.1 BLEU, on the WMT 2014 English-to-French translation tasks. We call this Transformer++.
摘要：关注机制的最新进展已经取代递归神经网络及其在机器翻译任务的变种。变压器使用注意机制仅在序列建模实现国家的最先进的成果。基于注意机制神经机器翻译是并行和地址的处理词中长距离的依赖关系的句子更有效地比回归神经网络的问题。一个注意力的关键概念是学习三个矩阵，查询键，和值，字间的全球依赖性穿过这些矩阵以直线状突出的嵌入词学。多个查询，关键字，值矩阵可以同时得知聚焦在不同的子空间的嵌入维数，这是所谓的多磁头在变压器。我们认为，单词之间一定的依赖性可以通过不是直接建模字字依赖中间上下文中了解到更好。这种情况可能发生由于某些依赖关系的性质或者缺乏的是借给他们难以用多头自重视在全球范围内模拟模式。在这项工作中，我们提出通过使用卷积在多头上下文学习依赖关系的新途径。这种新的多头注意形式与传统的形式一起实现对WMT 2014比变压器更好的效果英语到德语和英语到法语翻译任务。我们还引进了一个框架，学习编码器的培训，进一步提高成绩达到32.1 BLEU的新的国家的最先进的，比现有的1.4 BLEU最好的，在WMT 2014英语到在POS标记和NER信息 - 德国和44.6 BLEU，比现有的1.1 BLEU最好的，在WMT 2014英语对法语翻译任务更好。我们称这种变压器++。
Prakhar Thapak, Prodip Hore
Abstract: Recent advancements in attention mechanisms have replaced recurrent neural networks and its variants for machine translation tasks. Transformer using attention mechanism solely achieved state-of-the-art results in sequence modeling. Neural machine translation based on the attention mechanism is parallelizable and addresses the problem of handling long-range dependencies among words in sentences more effectively than recurrent neural networks. One of the key concepts in attention is to learn three matrices, query, key, and value, where global dependencies among words are learned through linearly projecting word embeddings through these matrices. Multiple query, key, value matrices can be learned simultaneously focusing on a different subspace of the embedded dimension, which is called multi-head in Transformer. We argue that certain dependencies among words could be learned better through an intermediate context than directly modeling word-word dependencies. This could happen due to the nature of certain dependencies or lack of patterns that lend them difficult to be modeled globally using multi-head self-attention. In this work, we propose a new way of learning dependencies through a context in multi-head using convolution. This new form of multi-head attention along with the traditional form achieves better results than Transformer on the WMT 2014 English-to-German and English-to-French translation tasks. We also introduce a framework to learn POS tagging and NER information during the training of encoder which further improves results achieving a new state-of-the-art of 32.1 BLEU, better than existing best by 1.4 BLEU, on the WMT 2014 English-to-German and 44.6 BLEU, better than existing best by 1.1 BLEU, on the WMT 2014 English-to-French translation tasks. We call this Transformer++.
摘要：关注机制的最新进展已经取代递归神经网络及其在机器翻译任务的变种。变压器使用注意机制仅在序列建模实现国家的最先进的成果。基于注意机制神经机器翻译是并行和地址的处理词中长距离的依赖关系的句子更有效地比回归神经网络的问题。一个注意力的关键概念是学习三个矩阵，查询键，和值，字间的全球依赖性穿过这些矩阵以直线状突出的嵌入词学。多个查询，关键字，值矩阵可以同时得知聚焦在不同的子空间的嵌入维数，这是所谓的多磁头在变压器。我们认为，单词之间一定的依赖性可以通过不是直接建模字字依赖中间上下文中了解到更好。这种情况可能发生由于某些依赖关系的性质或者缺乏的是借给他们难以用多头自重视在全球范围内模拟模式。在这项工作中，我们提出通过使用卷积在多头上下文学习依赖关系的新途径。这种新的多头注意形式与传统的形式一起实现对WMT 2014比变压器更好的效果英语到德语和英语到法语翻译任务。我们还引进了一个框架，学习编码器的培训，进一步提高成绩达到32.1 BLEU的新的国家的最先进的，比现有的1.4 BLEU最好的，在WMT 2014英语到在POS标记和NER信息 - 德国和44.6 BLEU，比现有的1.1 BLEU最好的，在WMT 2014英语对法语翻译任务更好。我们称这种变压器++。
23. Localized Flood DetectionWith Minimal Labeled Social Media Data Using Transfer Learning [PDF] 返回目录
Neha Singh, Nirmalya Roy, Aryya Gangopadhyay
Abstract: Social media generates an enormous amount of data on a daily basis but it is very challenging to effectively utilize the data without annotating or labeling it according to the target application. We investigate the problem of localized flood detection using the social sensing model (Twitter) in order to provide an efficient, reliable and accurate flood text classification model with minimal labeled data. This study is important since it can immensely help in providing the flood-related updates and notifications to the city officials for emergency decision making, rescue operations, and early warnings, etc. We propose to perform the text classification using the inductive transfer learning method i.e pre-trained language model ULMFiT and fine-tune it in order to effectively classify the flood-related feeds in any new location. Finally, we show that using very little new labeled data in the target domain we can successfully build an efficient and high performing model for flood detection and analysis with human-generated facts and observations from Twitter.
摘要：社交媒体生成数据的每天的基础上大量的但它是非常具有挑战性的有效利用数据而无需标注或根据目标应用程序标记它。我们调查的局部洪涝检测，以提供高效，可靠和准确的洪水文本分类模型以最小的标记数据使用社交感知模型（Twitter）上的问题。这项研究很重要，因为它可以提供应急决策，救援行动和预警等洪水相关的更新和通知给市政府官员帮助极大，我们建议使用感应传输学习方法即进行文本分类预先训练语言模型ULMFiT，为了在任何新的位置有效分类洪水相关的饲料微调它。最后，我们表明，在目标域中使用很少的新标记数据，我们可以成功打造洪水检测和分析来自Twitter的人产生的事实和意见的高效，高性能的机型。
Neha Singh, Nirmalya Roy, Aryya Gangopadhyay
Abstract: Social media generates an enormous amount of data on a daily basis but it is very challenging to effectively utilize the data without annotating or labeling it according to the target application. We investigate the problem of localized flood detection using the social sensing model (Twitter) in order to provide an efficient, reliable and accurate flood text classification model with minimal labeled data. This study is important since it can immensely help in providing the flood-related updates and notifications to the city officials for emergency decision making, rescue operations, and early warnings, etc. We propose to perform the text classification using the inductive transfer learning method i.e pre-trained language model ULMFiT and fine-tune it in order to effectively classify the flood-related feeds in any new location. Finally, we show that using very little new labeled data in the target domain we can successfully build an efficient and high performing model for flood detection and analysis with human-generated facts and observations from Twitter.
摘要：社交媒体生成数据的每天的基础上大量的但它是非常具有挑战性的有效利用数据而无需标注或根据目标应用程序标记它。我们调查的局部洪涝检测，以提供高效，可靠和准确的洪水文本分类模型以最小的标记数据使用社交感知模型（Twitter）上的问题。这项研究很重要，因为它可以提供应急决策，救援行动和预警等洪水相关的更新和通知给市政府官员帮助极大，我们建议使用感应传输学习方法即进行文本分类预先训练语言模型ULMFiT，为了在任何新的位置有效分类洪水相关的饲料微调它。最后，我们表明，在目标域中使用很少的新标记数据，我们可以成功打造洪水检测和分析来自Twitter的人产生的事实和意见的高效，高性能的机型。
24. A Comparative Study of Sequence Classification Models for Privacy Policy Coverage Analysis [PDF] 返回目录
Zachary Lindner
Abstract: Privacy policies are legal documents that describe how a website will collect, use, and distribute a user's data. Unfortunately, such documents are often overly complicated and filled with legal jargon; making it difficult for users to fully grasp what exactly is being collected and why. Our solution to this problem is to provide users with a coverage analysis of a given website's privacy policy using a wide range of classical machine learning and deep learning techniques. Given a website's privacy policy, the classifier identifies the associated data practice for each logical segment. These data practices/labels are taken directly from the OPP-115 corpus. For example, the data practice "Data Retention" refers to how long a website stores a user's information. The coverage analysis allows users to determine how many of the ten possible data practices are covered, along with identifying the sections that correspond to the data practices of particular interest.
摘要：隐私政策是描述一个网站将如何收集，使用和分发用户数据的法律文件。不幸的是，这样的文件往往过于复杂，充满了法律术语;使得用户难以完全掌握确切正在收集什么，以及为什么。我们对这个问题的解决方案是为用户提供使用范围广，经典的机器学习和深入学习技术的特定网站的隐私政策的覆盖分析。鉴于网站的隐私政策，分类识别每个逻辑段相关数据的做法。这些数据做法/标签直接从OPP-115语料库截取。例如，数据的做法“数据保留”是指用户的信息多久网站专卖店。该报道分析，使用户能够确定有多少十个可能的数据实践的覆盖，以识别部分沿着对应于特定兴趣的惯例。
Zachary Lindner
Abstract: Privacy policies are legal documents that describe how a website will collect, use, and distribute a user's data. Unfortunately, such documents are often overly complicated and filled with legal jargon; making it difficult for users to fully grasp what exactly is being collected and why. Our solution to this problem is to provide users with a coverage analysis of a given website's privacy policy using a wide range of classical machine learning and deep learning techniques. Given a website's privacy policy, the classifier identifies the associated data practice for each logical segment. These data practices/labels are taken directly from the OPP-115 corpus. For example, the data practice "Data Retention" refers to how long a website stores a user's information. The coverage analysis allows users to determine how many of the ten possible data practices are covered, along with identifying the sections that correspond to the data practices of particular interest.
摘要：隐私政策是描述一个网站将如何收集，使用和分发用户数据的法律文件。不幸的是，这样的文件往往过于复杂，充满了法律术语;使得用户难以完全掌握确切正在收集什么，以及为什么。我们对这个问题的解决方案是为用户提供使用范围广，经典的机器学习和深入学习技术的特定网站的隐私政策的覆盖分析。鉴于网站的隐私政策，分类识别每个逻辑段相关数据的做法。这些数据做法/标签直接从OPP-115语料库截取。例如，数据的做法“数据保留”是指用户的信息多久网站专卖店。该报道分析，使用户能够确定有多少十个可能的数据实践的覆盖，以识别部分沿着对应于特定兴趣的惯例。
25. A Dataset Independent Set of Baselines for Relation Prediction in Argument Mining [PDF] 返回目录
Oana Cocarascu, Elena Cabrio, Serena Villata, Francesca Toni
Abstract: Argument Mining is the research area which aims at extracting argument components and predicting argumentative relations (i.e.,support and attack) from text. In particular, numerous approaches have been proposed in the literature to predict the relations holding between the arguments, and application-specific annotated resources were built for this purpose. Despite the fact that these resources have been created to experiment on the same task, the definition of a single relation prediction method to be successfully applied to a significant portion of these datasets is an open research problem in Argument Mining. This means that none of the methods proposed in the literature can be easily ported from one resource to another. In this paper, we address this problem by proposing a set of dataset independent strong neural baselines which obtain homogeneous results on all the datasets proposed in the literature for the argumentative relation prediction task. Thus, our baselines can be employed by the Argument Mining community to compare more effectively how well a method performs on the argumentative relation prediction task.
摘要：参数矿业是研究领域，其目的是提取参数部件和预测从文本议论的关系（即，支持和攻击）。特别是，许多方法已经提出在文献中预测参数之间的控股关系，以及具体的应用注解资源专为这一目的。尽管这些资源已创建实验在同一任务的事实，单一的关系预测方法的定义被成功地应用于这些数据集的显著部分是论证挖掘一个开放的研究问题。这意味着，没有在文献中提出的方法可以很容易地从一个资源到另一个移植。在本文中，我们通过提出一组获得的文献议论关系预测任务提出的所有数据集同质结果集自主神经强劲基线的解决这个问题。因此，我们的基线可以通过参数矿业界被用来比较更有效地如何在议论关系预测任务的方法进行。
Oana Cocarascu, Elena Cabrio, Serena Villata, Francesca Toni
Abstract: Argument Mining is the research area which aims at extracting argument components and predicting argumentative relations (i.e.,support and attack) from text. In particular, numerous approaches have been proposed in the literature to predict the relations holding between the arguments, and application-specific annotated resources were built for this purpose. Despite the fact that these resources have been created to experiment on the same task, the definition of a single relation prediction method to be successfully applied to a significant portion of these datasets is an open research problem in Argument Mining. This means that none of the methods proposed in the literature can be easily ported from one resource to another. In this paper, we address this problem by proposing a set of dataset independent strong neural baselines which obtain homogeneous results on all the datasets proposed in the literature for the argumentative relation prediction task. Thus, our baselines can be employed by the Argument Mining community to compare more effectively how well a method performs on the argumentative relation prediction task.
摘要：参数矿业是研究领域，其目的是提取参数部件和预测从文本议论的关系（即，支持和攻击）。特别是，许多方法已经提出在文献中预测参数之间的控股关系，以及具体的应用注解资源专为这一目的。尽管这些资源已创建实验在同一任务的事实，单一的关系预测方法的定义被成功地应用于这些数据集的显著部分是论证挖掘一个开放的研究问题。这意味着，没有在文献中提出的方法可以很容易地从一个资源到另一个移植。在本文中，我们通过提出一组获得的文献议论关系预测任务提出的所有数据集同质结果集自主神经强劲基线的解决这个问题。因此，我们的基线可以通过参数矿业界被用来比较更有效地如何在议论关系预测任务的方法进行。
26. Aspect Term Extraction using Graph-based Semi-Supervised Learning [PDF] 返回目录
Gunjan Ansari, Chandni Saxena, Tanvir Ahmad, M.N.Doja
Abstract: Aspect based Sentiment Analysis is a major subarea of sentiment analysis. Many supervised and unsupervised approaches have been proposed in the past for detecting and analyzing the sentiment of aspect terms. In this paper, a graph-based semi-supervised learning approach for aspect term extraction is proposed. In this approach, every identified token in the review document is classified as aspect or non-aspect term from a small set of labeled tokens using label spreading algorithm. The k-Nearest Neighbor (kNN) for graph sparsification is employed in the proposed approach to make it more time and memory efficient. The proposed work is further extended to determine the polarity of the opinion words associated with the identified aspect terms in review sentence to generate visual aspect-based summary of review documents. The experimental study is conducted on benchmark and crawled datasets of restaurant and laptop domains with varying value of labeled instances. The results depict that the proposed approach could achieve good result in terms of Precision, Recall and Accuracy with limited availability of labeled data.
摘要：基于看点情感分析是情感分析的主要分区。许多监督和无监督的方法在过去已经提出了检测和分析方面方面的情绪。在本文中，对术语方面提取基于图的半监督学习方法建议。在这种方法中，每一个审查文件中确定令牌被分类为从一小组使用标签扩展算法标记的令牌的方面或非方面的术语。对于图形稀疏化的k最近邻（KNN）中所提出的方法，采用使之更多的时间和存储器高效。拟议的工作进一步延伸，以确定与识别方面的术语在评述语句关联生成审查文件的基于视觉方面，总结观点词的极性。的实验研究是在基准进行爬餐厅和膝上型域的数据集具有不同标记的实例的值。结果描绘，该方法可以在准确率，召回方面取得良好的结果，并与标记的数据的可用性有限精度。
Gunjan Ansari, Chandni Saxena, Tanvir Ahmad, M.N.Doja
Abstract: Aspect based Sentiment Analysis is a major subarea of sentiment analysis. Many supervised and unsupervised approaches have been proposed in the past for detecting and analyzing the sentiment of aspect terms. In this paper, a graph-based semi-supervised learning approach for aspect term extraction is proposed. In this approach, every identified token in the review document is classified as aspect or non-aspect term from a small set of labeled tokens using label spreading algorithm. The k-Nearest Neighbor (kNN) for graph sparsification is employed in the proposed approach to make it more time and memory efficient. The proposed work is further extended to determine the polarity of the opinion words associated with the identified aspect terms in review sentence to generate visual aspect-based summary of review documents. The experimental study is conducted on benchmark and crawled datasets of restaurant and laptop domains with varying value of labeled instances. The results depict that the proposed approach could achieve good result in terms of Precision, Recall and Accuracy with limited availability of labeled data.
摘要：基于看点情感分析是情感分析的主要分区。许多监督和无监督的方法在过去已经提出了检测和分析方面方面的情绪。在本文中，对术语方面提取基于图的半监督学习方法建议。在这种方法中，每一个审查文件中确定令牌被分类为从一小组使用标签扩展算法标记的令牌的方面或非方面的术语。对于图形稀疏化的k最近邻（KNN）中所提出的方法，采用使之更多的时间和存储器高效。拟议的工作进一步延伸，以确定与识别方面的术语在评述语句关联生成审查文件的基于视觉方面，总结观点词的极性。的实验研究是在基准进行爬餐厅和膝上型域的数据集具有不同标记的实例的值。结果描绘，该方法可以在准确率，召回方面取得良好的结果，并与标记的数据的可用性有限精度。
27. KryptoOracle: A Real-Time Cryptocurrency Price Prediction Platform Using Twitter Sentiments [PDF] 返回目录
Shubhankar Mohapatra, Nauman Ahmed, Paulo Alencar
Abstract: Cryptocurrencies, such as Bitcoin, are becoming increasingly popular, having been widely used as an exchange medium in areas such as financial transaction and asset transfer verification. However, there has been a lack of solutions that can support real-time price prediction to cope with high currency volatility, handle massive heterogeneous data volumes, including social media sentiments, while supporting fault tolerance and persistence in real time, and provide real-time adaptation of learning algorithms to cope with new price and sentiment data. In this paper we introduce KryptoOracle, a novel real-time and adaptive cryptocurrency price prediction platform based on Twitter sentiments. The integrative and modular platform is based on (i) a Spark-based architecture which handles the large volume of incoming data in a persistent and fault tolerant way; (ii) an approach that supports sentiment analysis which can respond to large amounts of natural language processing queries in real time; and (iii) a predictive method grounded on online learning in which a model adapts its weights to cope with new prices and sentiments. Besides providing an architectural design, the paper also describes the KryptoOracle platform implementation and experimental evaluation. Overall, the proposed platform can help accelerate decision-making, uncover new opportunities and provide more timely insights based on the available and ever-larger financial data volume and variety.
摘要：Cryptocurrencies，如比特币，正变得越来越流行，已被广泛用作领域的交流媒介，诸如金融交易和资产转移验证。但是，一直缺乏能够支持实时价格预测，以应对高汇率波动，处理海量异构数据卷，包括社交媒体情绪的解决方案，同时支持实时容错性和持久性，并提供实时的的学习算法调整，以应付新的价格和情绪数据。在本文中，我们介绍了KryptoOracle，根据Twitter上的情绪一种新型的实时性和适应性cryptocurrency价格预测平台。一体化的模块化平台是基于（i）基于火花架构，其处理的大量输入数据的在持久性和容错方式; （ⅱ）一种方法，它支持情绪分析可将大量的实时自然语言处理响应查询;及（iii）的预测方法接地在线学习其中一种模式调整其权重，以应付新的价格和情绪。除了提供建筑设计，本文还介绍了KryptoOracle平台实现与实验评价。总体而言，拟议的平台可以帮助加快决策，发掘新的机遇，并提供基于现有的和越来越大的财务数据的数量和种类上更及时的见解。
Shubhankar Mohapatra, Nauman Ahmed, Paulo Alencar
Abstract: Cryptocurrencies, such as Bitcoin, are becoming increasingly popular, having been widely used as an exchange medium in areas such as financial transaction and asset transfer verification. However, there has been a lack of solutions that can support real-time price prediction to cope with high currency volatility, handle massive heterogeneous data volumes, including social media sentiments, while supporting fault tolerance and persistence in real time, and provide real-time adaptation of learning algorithms to cope with new price and sentiment data. In this paper we introduce KryptoOracle, a novel real-time and adaptive cryptocurrency price prediction platform based on Twitter sentiments. The integrative and modular platform is based on (i) a Spark-based architecture which handles the large volume of incoming data in a persistent and fault tolerant way; (ii) an approach that supports sentiment analysis which can respond to large amounts of natural language processing queries in real time; and (iii) a predictive method grounded on online learning in which a model adapts its weights to cope with new prices and sentiments. Besides providing an architectural design, the paper also describes the KryptoOracle platform implementation and experimental evaluation. Overall, the proposed platform can help accelerate decision-making, uncover new opportunities and provide more timely insights based on the available and ever-larger financial data volume and variety.
摘要：Cryptocurrencies，如比特币，正变得越来越流行，已被广泛用作领域的交流媒介，诸如金融交易和资产转移验证。但是，一直缺乏能够支持实时价格预测，以应对高汇率波动，处理海量异构数据卷，包括社交媒体情绪的解决方案，同时支持实时容错性和持久性，并提供实时的的学习算法调整，以应付新的价格和情绪数据。在本文中，我们介绍了KryptoOracle，根据Twitter上的情绪一种新型的实时性和适应性cryptocurrency价格预测平台。一体化的模块化平台是基于（i）基于火花架构，其处理的大量输入数据的在持久性和容错方式; （ⅱ）一种方法，它支持情绪分析可将大量的实时自然语言处理响应查询;及（iii）的预测方法接地在线学习其中一种模式调整其权重，以应付新的价格和情绪。除了提供建筑设计，本文还介绍了KryptoOracle平台实现与实验评价。总体而言，拟议的平台可以帮助加快决策，发掘新的机遇，并提供基于现有的和越来越大的财务数据的数量和种类上更及时的见解。
28. Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning [PDF] 返回目录
Zhiyuan Fang, Tejas Gokhale, Pratyay Banerjee, Chitta Baral, Yezhou Yang
Abstract: Captioning is a crucial and challenging task for video understanding. In videos that involve active agents such as humans, the agent's actions can bring about myriad changes in the scene. These changes can be observable, such as movements, manipulations, and transformations of the objects in the scene - these are reflected in conventional video captioning. However, unlike images, actions in videos are also inherently linked to social and commonsense aspects such as intentions (why the action is taking place), attributes (such as who is doing the action, on whom, where, using what etc.) and effects (how the world changes due to the action, the effect of the action on other agents). Thus for video understanding, such as when captioning videos or when answering question about videos, one must have an understanding of these commonsense aspects. We present the first work on generating \textit{commonsense} captions directly from videos, in order to describe latent aspects such as intentions, attributes, and effects. We present a new dataset "Video-to-Commonsense (V2C)" that contains 9k videos of human agents performing various actions, annotated with 3 types of commonsense descriptions. Additionally we explore the use of open-ended video-based commonsense question answering (V2C-QA) as a way to enrich our captions. We finetune our commonsense generation models on the V2C-QA task where we ask questions about the latent aspects in the video. Both the generation task and the QA task can be used to enrich video captions.
摘要：字幕是视频理解的关键和艰巨的任务。在涉及活性剂，如人体的视频，代理人的行为可以带来在现场无数的变化。这些变化可以是可观察到的，如移动，操纵，并在场景中的对象的变换 - 这些都反映在常规的视频字幕。然而，与图像，视频动作也是内在的联系社会和常识等方面的意图（为什么动作正在发生），属性（比如是谁做的动作，上谁，在哪里，使用的是什么等）和效果（世界如何变化，由于行动，对其他药物的作用效果）。因此，对于视频的理解，这样的视频添加字幕或回答有关影片的问题时作为时，必须有这些常识方面的理解。我们目前的发电\ textit {}常识字幕第一部作品直接从视频中，为了描述潜在的方面，如意图，属性和效果。我们提出了一个新的数据集“视频到常识（V2C）”包含执行各种行动，3种类型的常识描述注释的人类代理9K视频。此外，我们还探索利用开放式的基于视频的常识问答（V2C-QA）的，以此来丰富我们的字幕。我们微调对V2C-QA任务，我们询问在视频中的潜在问题方面我们常识性的一代车型。无论是生成任务和QA任务可以用来丰富的视频字幕。
Zhiyuan Fang, Tejas Gokhale, Pratyay Banerjee, Chitta Baral, Yezhou Yang
Abstract: Captioning is a crucial and challenging task for video understanding. In videos that involve active agents such as humans, the agent's actions can bring about myriad changes in the scene. These changes can be observable, such as movements, manipulations, and transformations of the objects in the scene - these are reflected in conventional video captioning. However, unlike images, actions in videos are also inherently linked to social and commonsense aspects such as intentions (why the action is taking place), attributes (such as who is doing the action, on whom, where, using what etc.) and effects (how the world changes due to the action, the effect of the action on other agents). Thus for video understanding, such as when captioning videos or when answering question about videos, one must have an understanding of these commonsense aspects. We present the first work on generating \textit{commonsense} captions directly from videos, in order to describe latent aspects such as intentions, attributes, and effects. We present a new dataset "Video-to-Commonsense (V2C)" that contains 9k videos of human agents performing various actions, annotated with 3 types of commonsense descriptions. Additionally we explore the use of open-ended video-based commonsense question answering (V2C-QA) as a way to enrich our captions. We finetune our commonsense generation models on the V2C-QA task where we ask questions about the latent aspects in the video. Both the generation task and the QA task can be used to enrich video captions.
摘要：字幕是视频理解的关键和艰巨的任务。在涉及活性剂，如人体的视频，代理人的行为可以带来在现场无数的变化。这些变化可以是可观察到的，如移动，操纵，并在场景中的对象的变换 - 这些都反映在常规的视频字幕。然而，与图像，视频动作也是内在的联系社会和常识等方面的意图（为什么动作正在发生），属性（比如是谁做的动作，上谁，在哪里，使用的是什么等）和效果（世界如何变化，由于行动，对其他药物的作用效果）。因此，对于视频的理解，这样的视频添加字幕或回答有关影片的问题时作为时，必须有这些常识方面的理解。我们目前的发电\ textit {}常识字幕第一部作品直接从视频中，为了描述潜在的方面，如意图，属性和效果。我们提出了一个新的数据集“视频到常识（V2C）”包含执行各种行动，3种类型的常识描述注释的人类代理9K视频。此外，我们还探索利用开放式的基于视频的常识问答（V2C-QA）的，以此来丰富我们的字幕。我们微调对V2C-QA任务，我们询问在视频中的潜在问题方面我们常识性的一代车型。无论是生成任务和QA任务可以用来丰富的视频字幕。
29. Visual Grounding in Video for Unsupervised Word Translation [PDF] 返回目录
Gunnar A. Sigurdsson, Jean-Baptiste Alayrac, Aida Nematzadeh, Lucas Smaira, Mateusz Malinowski, João Carreira, Phil Blunsom, Andrew Zisserman
Abstract: There are thousands of actively spoken languages on Earth, but a single visual world. Grounding in this visual world has the potential to bridge the gap between all these languages. Our goal is to use visual grounding to improve unsupervised word mapping between languages. The key idea is to establish a common visual representation between two languages by learning embeddings from unpaired instructional videos narrated in the native language. Given this shared embedding we demonstrate that (i) we can map words between the languages, particularly the 'visual' words; (ii) that the shared embedding provides a good initialization for existing unsupervised text-based word translation techniques, forming the basis for our proposed hybrid visual-text mapping algorithm, MUVE; and (iii) our approach achieves superior performance by addressing the shortcomings of text-based methods -- it is more robust, handles datasets with less commonality, and is applicable to low-resource languages. We apply these methods to translate words from English to French, Korean, and Japanese -- all without any parallel corpora and simply by watching many videos of people speaking while doing things.
摘要：有数以千计的积极使用的语言在地球上，但单一的视觉世界。在这个视觉世界具有接地弥合所有这些语言之间的差距的潜力。我们的目标是利用视觉接地，以提高语言之间的无监督单词映射。其核心思想是通过学习从母语讲述配对的教学视频的嵌入建立两种语言之间共同的视觉表示。鉴于这种共享嵌入我们表明，（I），我们可以映射语言，特别是“视觉”字之间的话; （ii）该共享嵌入提供对现有的无监督的基于文本的字翻译技术，形成用于我们提出的混合视觉文本映射算法，MUVE基础良好的初始化; （三）我们的方法通过解决基于文本的方法的缺点实现卓越的性能 - 这是更强大的，把手用更少的共性的数据集，并适用于低资源语言。我们应用这些方法从英语译成法语，韩语和日语的话 - 无需任何平行语料库，只是通过看的人而做的事情讲许多影片。
Gunnar A. Sigurdsson, Jean-Baptiste Alayrac, Aida Nematzadeh, Lucas Smaira, Mateusz Malinowski, João Carreira, Phil Blunsom, Andrew Zisserman
Abstract: There are thousands of actively spoken languages on Earth, but a single visual world. Grounding in this visual world has the potential to bridge the gap between all these languages. Our goal is to use visual grounding to improve unsupervised word mapping between languages. The key idea is to establish a common visual representation between two languages by learning embeddings from unpaired instructional videos narrated in the native language. Given this shared embedding we demonstrate that (i) we can map words between the languages, particularly the 'visual' words; (ii) that the shared embedding provides a good initialization for existing unsupervised text-based word translation techniques, forming the basis for our proposed hybrid visual-text mapping algorithm, MUVE; and (iii) our approach achieves superior performance by addressing the shortcomings of text-based methods -- it is more robust, handles datasets with less commonality, and is applicable to low-resource languages. We apply these methods to translate words from English to French, Korean, and Japanese -- all without any parallel corpora and simply by watching many videos of people speaking while doing things.
摘要：有数以千计的积极使用的语言在地球上，但单一的视觉世界。在这个视觉世界具有接地弥合所有这些语言之间的差距的潜力。我们的目标是利用视觉接地，以提高语言之间的无监督单词映射。其核心思想是通过学习从母语讲述配对的教学视频的嵌入建立两种语言之间共同的视觉表示。鉴于这种共享嵌入我们表明，（I），我们可以映射语言，特别是“视觉”字之间的话; （ii）该共享嵌入提供对现有的无监督的基于文本的字翻译技术，形成用于我们提出的混合视觉文本映射算法，MUVE基础良好的初始化; （三）我们的方法通过解决基于文本的方法的缺点实现卓越的性能 - 这是更强大的，把手用更少的共性的数据集，并适用于低资源语言。我们应用这些方法从英语译成法语，韩语和日语的话 - 无需任何平行语料库，只是通过看的人而做的事情讲许多影片。
注：中文为机器翻译结果！