Published as a conference paper at ICLR 2015 NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE Dzmitry Bahdanau Jacobs University Bremen, - 中英文对照

专业知识 · 40-References/Papers/attention - Attention/02_bilingual.md

translated: 2026-07-16

title: "Published as a conference paper at ICLR 2015 NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE Dzmitry Bahdanau Jacobs University Bremen," aliases: - "Attention" - "arXiv:1409.0473" source: "https://arxiv.org/abs/1409.0473" arxiv: "1409.0473" created: 2026-07-16 type: paper-translation status: translated tags: - paper - ml - deep-learning - nlp

Published as a conference paper at ICLR 2015 NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE Dzmitry Bahdanau Jacobs University Bremen, - 中英文对照

中英文对照

<a id="S0001"></a> Source: p.1 S0001

Original: Published as a conference paper at ICLR 2015 NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE Dzmitry Bahdanau Jacobs University Bremen, Germany KyungHyun Cho Yoshua Bengio∗ Universite´ de Montre´al ABSTRACT Neural machine translation is a recently proposed approach to machine translation.

中文: 作为会议论文发表于德国不来梅大学2015 NEURAL MACHINE TRATION LIGN and TranalLATE Dzmitry Bahdanau Jacobs University Bremen CyungHyun Cho Yoshua Bengio * Montre大学 神经机能翻译是最近提出的机器翻译方法.

<a id="S0002"></a> Source: p.1 S0002

Original: Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance.

中文: 与传统的统计机器翻译不同,神经机器翻译旨在建立一个单一的神经网络,可以联合调制来使翻译性能最大化.

<a id="S0003"></a> Source: p.1 S0003

Original: The models proposed recently for neural machine translation often belong to a family of encoder–decoders and encode a source sentence into a fixed-length vector from which a decoder generates a translation.

中文: 最近提出的神经机器翻译模型往往属于编码器-解码器的家族,并将源句编码为固定长向量,从中解码器产生翻译.

<a id="S0004"></a> Source: p.1 S0004

Original: In this paper, we conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder–decoder architecture, and propose to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly.

中文: 在本文中,我们推测使用固定长向量是改善这种基本编码-解码器架构性能的瓶颈,并提议通过允许一个模型自动(软)地搜索源句中与预测目标词相关的部分,而不必明确地将这些部分形成硬段来扩展.

<a id="S0005"></a> Source: p.1 S0005

Original: With this new approach, we achieve a translation performance comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation.

中文: 有了这一新的方法,我们实现了与现有最先进的词组系统相当的关于英语到法语翻译任务的翻译性能.

<a id="S0006"></a> Source: p.1 S0006

Original: Furthermore, qualitative analysis reveals that the (soft-)alignments found by the model agree well with our intuition. 1 INTRODUCTION Neural machine translation is a newly emerging approach to machine translation, recently proposed by Kalchbrenner and Blunsom (2013), Sutskever et al. (2014) and Cho et al. (2014b).

中文: 此外,定性分析表明,模型所发现的(软的)排列与我们的直觉完全一致。 导言 神经机翻译是一种新兴的机器翻译方法,最近由Kalchbrenner和Blunsom(2013年),Sutskever等人(2014年)和Cho等人(2014年b)提出.

<a id="S0007"></a> Source: p.1 S0007

Original: Unlike the traditional phrase-based translation system (see, e.g., Koehn et al., 2003) which consists of many small sub-components that are tuned separately, neural machine translation attempts to build and train a single, large neural network that reads a sentence and outputs a correct translation.

中文: 与传统的以短语为主的翻译系统(例如见Koehn等,2003年)不同,该系统由许多分别调制的小子组件组成,神经机翻译试图构建并训练一个能读出句子并输出出正确翻译的单一而大型的神经网络.

<a id="S0008"></a> Source: p.1 S0008

Original: Most of the proposed neural machine translation models belong to a family of encoder– decoders (Sutskever et al., 2014; Cho et al., 2014a), with an encoder and a decoder for each language, or involve a language-specific encoder applied to each sentence whose outputs are then compared (Hermann and Blunsom, 2014).

中文: 所拟议的神经机翻译模型大多属于编码器-解码器家族(Sutskever等,2014;Cho等,2014a),每种语言都有编码器和解码器,或者涉及一个语言特有编码器适用于每个句子,然后对输出进行比较(Hermann和Blunsom,2014年)。

<a id="S0009"></a> Source: p.1 S0009

Original: An encoder neural network reads and encodes a source sentence into a fixed-length vector. A decoder then outputs a translation from the encoded vector.

中文: 一个编码器神经网络将源句读取并编码为固定长向量. 一个解码器然后输出编码向量的翻译。

<a id="S0010"></a> Source: p.1 S0010

Original: The whole encoder–decoder system, which consists of the encoder and the decoder for a language pair, is jointly trained to maximize the probability of a correct translation given a source sentence. A potential issue with this encoder–decoder approach is that a neural network needs to be able to compress all the necessary information of a source sentence into a fixed-length vector.

中文: 整个编码器-解码器系统由一对语言的编码器和解码器组成,共同训练,以尽可能地使给出源句的正确翻译概率最大化. 这种编码器–解码器方法的一个潜在问题是神经网络需要能够将源句所有必要信息压缩成固定长向量.

<a id="S0011"></a> Source: p.1 S0011

Original: This may make it difficult for the neural network to cope with long sentences, especially those that are longer than the sentences in the training corpus.

中文: 这可能使神经网络难以应对长刑期,尤其是比训练大纲中的刑期长的刑期.

<a id="S0012"></a> Source: p.1 S0012

Original: Cho et al. (2014b) showed that indeed the performance of a basic encoder–decoder deteriorates rapidly as the length of an input sentence increases.

中文: Cho等人(2014年b)表明,随着输入句子长度的增加,一个基本编码器-解码器的性能确实会迅速恶化.

<a id="S0013"></a> Source: p.1 S0013

Original: In order to address this issue, we introduce an extension to the encoder–decoder model which learns to align and translate jointly.

中文: 为了解决这一问题,我们引入了编码器-解码器模型的扩展,该模型学习联合对齐和翻译。

<a id="S0014"></a> Source: p.1 S0014

Original: Each time the proposed model generates a word in a translation, it (soft-)searches for a set of positions in a source sentence where the most relevant information is concentrated.

中文: 每次拟议模型在翻译中生成一个单词,它(soft-)搜索源句中最相关信息集中的一组位置.

<a id="S0015"></a> Source: p.1 S0015

Original: The model then predicts a target word based on the context vectors associated with these source positions and all the previous generated target words. ∗CIFAR Senior Fellow 1 6102 yaM 91 ]LC.sc[ 7v3740.9041:viXra

中文: 该模型随后根据与这些源位置相关的上下文向量和之前所有生成的目标词预测出一个目标词. * 科协高级研究员 1 6102 yaM 91]LC.sc [7v3740.9041:viXra

<a id="S0016"></a> Source: p.2 S0016

Original: Published as a conference paper at ICLR 2015 The most important distinguishing feature of this approach from the basic encoder–decoder is that it does not attempt to encode a whole input sentence into a single fixed-length vector.

中文: 作为会议论文发表于2015年国际法学研究中心 这种方法与基本编码器-解码器最重要的区别特征是,它并不试图将整个输入句编码为单个固定长向量.

<a id="S0017"></a> Source: p.2 S0017

Original: Instead, it encodes the input sentence into a sequence of vectors and chooses a subset of these vectors adaptively while decoding the translation.

中文: 相反地,它将输入句编码为向量的序列,并在解码翻译时根据需要选择这些向量的子集.

<a id="S0018"></a> Source: p.2 S0018

Original: This frees a neural translation model from having to squash all the information of a source sentence, regardless of its length, into a fixed-length vector.

中文: 这使得神经翻译模型不必将源句的所有信息,无论长度如何,都挤压为固定长向量.

<a id="S0019"></a> Source: p.2 S0019

Original: We show this allows a model to cope better with long sentences.

中文: 我们显示这可以让模型更好地应对长句.

<a id="S0020"></a> Source: p.2 S0020

Original: In this paper, we show that the proposed approach of jointly learning to align and translate achieves significantly improved translation performance over the basic encoder–decoder approach.

中文: 在本文中,我们表明,拟议的联合学习以调整和翻译的方法比基本编码器-编码器方法大大提高了翻译绩效。

<a id="S0021"></a> Source: p.2 S0021

Original: The improvement is more apparent with longer sentences, but can be observed with sentences of any length.

中文: 随着刑期的延长,这种改善更为明显,但无论刑期长短,都可以观察到。

<a id="S0022"></a> Source: p.2 S0022

Original: On the task of English-to-French translation, the proposed approach achieves, with a single model, a translation performance comparable, or close, to the conventional phrase-based system.

中文: 关于从英文到法文的翻译任务,提议的方法实现了单一模式的翻译性能与传统的以短语为基础的系统相当或接近。

<a id="S0023"></a> Source: p.2 S0023

Original: Furthermore, qualitative analysis reveals that the proposed model finds a linguistically plausible (soft-)alignment between a source sentence and the corresponding target sentence. 2 BACKGROUND: NEURAL MACHINE TRANSLATION From a probabilistic perspective, translation is equivalent to finding a target sentence y that maximizes the conditional probability of y given a source sentence x, i.e., arg max p(y | x).

中文: 此外,定性分析显示,拟议模型发现源句与相应目标句在语言上是可信的(软的)相接. 2 背景:新机器传输 从概率学的角度来看,翻译相当于找到一个目标句y,使y给定源句x的条件概率最大化,即arg max p(y QX x).

<a id="S0024"></a> Source: p.2 S0024

Original: In y neural machine translation, we fit a parameterized model to maximize the conditional probability of sentence pairs using a parallel training corpus.

中文: 在y神经机理翻译中,我们搭配了参数化模型,利用平行的训练本体来使句子对的有条件概率最大化.

<a id="S0025"></a> Source: p.2 S0025

Original: Once the conditional distribution is learned by a translation model, given a source sentence a corresponding translation can be generated by searching for the sentence that maximizes the conditional probability.

中文: 一旦一个翻译模型得知了有条件的分布,给出了源句,就可以通过搜索可以使条件概率最大化的句子来生成相应的翻译.

<a id="S0026"></a> Source: p.2 S0026

Original: Recently, a number of papers have proposed the use of neural networks to directly learn this conditional distribution (see, e.g., Kalchbrenner and Blunsom, 2013; Cho et al., 2014a; Sutskever et al., 2014; Cho et al., 2014b; Forcada and N˜ eco, 1997).

中文: 最近,有多篇论文提议使用神经网络直接学习这种有条件的分布(如:Kalchbrenner和Blunsom,2013;Cho等,2014a;Sutskever等,2014;Cho等,2014b;Forcada和N-Q-生态,1997年)。

<a id="S0027"></a> Source: p.2 S0027

Original: This neural machine translation approach typically consists of two components, the first of which encodes a source sentence x and the second decodes to a target sentence y.

中文: 这种神经机翻译方法一般由两个部分组成,第一个部分编码出源句x,第二个部分解码出目标句y.

<a id="S0028"></a> Source: p.2 S0028

Original: For instance, two recurrent neural networks (RNN) were used by (Cho et al., 2014a) and (Sutskever et al., 2014) to encode a variable-length source sentence into a fixed-length vector and to decode the vector into a variable-length target sentence.

中文: 例如,(Cho等人,2014年a)和(Sutskever等人,2014年)使用了两个经常性神经网络(RNN)将可变长源句编码为固定长向量并解码为可变长目标句.

<a id="S0029"></a> Source: p.2 S0029

Original: Despite being a quite new approach, neural machine translation has already shown promising results.

中文: 尽管是一种相当新的方法,神经机翻译已经显示出有希望的结果.

<a id="S0030"></a> Source: p.2 S0030

Original: Sutskever et al. (2014) reported that the neural machine translation based on RNNs with long shortterm memory (LSTM) units achieves close to the state-of-the-art performance of the conventional phrase-based machine translation system on an English-to-French translation task.1 Adding neural components to existing translation systems, for instance, to score the phrase pairs in the phrase table (Cho et al., 2014a) or to re-rank candidate translations (Sutskever et al., 2014), has allowed to surpass the previous state-of-the-art performance level. 2.1 RNN ENCODER–DECODER Here, we describe briefly the underlying framework, called RNN Encoder–Decoder, proposed by Cho et al. (2014a) and Sutskever et al. (2014) upon which we build a novel architecture that learns to align and translate simultaneously.

中文: Sutskever等人(2014年)报告说,基于具有长期内存(LSTM)单元的RNNs的神经机翻译在英语至法语翻译任务上接近于传统短语机翻译系统的最新性能。 例如,在现有翻译系统中添加神经元,以对短语表中的词组对分(Cho等人,2014年a),或对候选翻译进行重新排序(Sutskever等人,2014年),使得该词可以超越之前最先进的性能水平. 2.1 连胜 在此,我们简要地描述了由Cho等人(2014年a)和Sutskever等人(2014年)提出的名为RNN Encoder-Decoder的基本框架,我们在此基础上构建了一种新的结构,学会同步地对齐和翻译。

<a id="S0031"></a> Source: p.2 S0031

Original: In the Encoder–Decoder framework, an encoder reads the input sentence, a sequence of vectors x = (x , · · · , x ), into a vector c.2 The most common approach is to use an RNN such that 1 Tx h = f (x , h ) (1) t t t−1 and c = q ({h , · · · , h }) , 1 Tx where h ∈ Rn is a hidden state at time t, and c is a vector generated from the sequence of the t hidden states. f and q are some nonlinear functions.

中文: 在 Encoder-Decoder 框架中,一个编码器将输入句,一个向量x=(x,-,-,x)的序列读取为向量c.2. 最常见的方法是使用RNN,这样1Tx h = f (x, h) (1)t t - 1和c = q ({h, · , h}),1Tx,其中h ∈ Rn在时间t是一个隐藏状态,而c是由t隐藏状态的序列产生的矢量. f和q是一些非线性函数.

<a id="S0032"></a> Source: p.2 S0032

Original: Sutskever et al. (2014) used an LSTM as f and q ({h , · · · , h }) = h , for instance. 1 T T 1 We mean by the state-of-the-art performance, the performance of the conventional phrase-based system without using any neural network-based component. 2 Although most of the previous works (see, e.g., Cho et al., 2014a; Sutskever et al., 2014; Kalchbrenner and Blunsom, 2013) used to encode a variable-length input sentence into a fixed-length vector, it is not necessary, and even it may be beneficial to have a variable-length vector, as we will show later. 2

中文: Sutskever等人(2014年)使用LSTM作为f和q({h, ·,h})=h,例如. 1 T T 1 我们指最先进的性能,即常规的以短语为基础的系统的性能而不使用任何以神经网络为基础的组件. 2个 虽然之前的大部分作品(如Cho等,2014a;Sutskever等,2014;Kalchbrenner和Blunsom,2013)曾将可变长的输入句编码为固定长向量,但没有必要,即使有可变长向量也可能有益,我们稍后会显示这一点. 2个

<a id="S0033"></a> Source: p.3 S0033

Original: Published as a conference paper at ICLR 2015 The decoder is often trained to predict the next word y given the context vector c and all the t(cid:48) previously predicted words {y , · · · , y }.

中文: 作为会议论文发表于2015年国际法学研究中心 考虑到上下文矢量c和所有t(cid:48)之前预言的单词{y, ·,y},解码器经常被训练来预测下个单词y.

<a id="S0034"></a> Source: p.3 S0034

Original: In other words, the decoder defines a probability over 1 t(cid:48)−1 the translation y by decomposing the joint probability into the ordered conditionals: T (cid:89) p(y) = p(y | {y , · · · , y } , c), (2) t 1 t−1 t=1 (cid:0) (cid:1) where y = y , · · · , y .

中文: 换句话说,解码器定义了1t(cid:48)−1的概率,通过将连通概率分解为所命令的条件来表示: T(cid:89) p(y) = p(y = {y, ·,y},c), (2)t 1t−1t= 1 (cid:0)(cid:1) 其中y=y,·,y.

<a id="S0035"></a> Source: p.3 S0035

Original: With an RNN, each conditional probability is modeled as 1 Ty p(y | {y , · · · , y } , c) = g(y , s , c), (3) t 1 t−1 t−1 t where g is a nonlinear, potentially multi-layered, function that outputs the probability of y , and s is t t the hidden state of the RNN.

中文: 使用 RNN,每个有条件的概率被模型化为1 Ty p(y | {y, · } }, y }, c } = g (y, s, c)), (3)t 1t−1t−1t,其中g是一个非线性,可能具有多层,输出 y 概率的函数,而 s 是 t t 所隐藏的 RNN 状态.

<a id="S0036"></a> Source: p.3 S0036

Original: It should be noted that other architectures such as a hybrid of an RNN and a de-convolutional neural network can be used (Kalchbrenner and Blunsom, 2013). 3 LEARNING TO ALIGN AND TRANSLATE In this section, we propose a novel architecture for neural machine translation.

中文: 需要注意的是,其他建筑如RNN和去革命神经网络的混合体可以被使用(Kalchbrenner和Blunsom,2013年). 3 学习文学和艺术 在这一节中,我们提出神经机能翻译的新建筑.

<a id="S0037"></a> Source: p.3 S0037

Original: The new architecture consists of a bidirectional RNN as an encoder (Sec. 3.2) and a decoder that emulates searching through a source sentence during decoding a translation (Sec. 3.1). 3.1 DECODER: GENERAL DESCRIPTION In a new model architecture, we define each conditional probability in Eq. (2) as: y y t-1 t p(y |y , . . . , y , x) = g(y , s , c ), (4) i 1 i−1 i−1 i i s s t-1 t where s is an RNN hidden state for time i, computed by i s = f (s , y , c ). + i i−1 i−1 i It should be noted that unlike the existing encoder–decoder ap- α t,1 αt,T proach (see Eq. (2)), here the probability is conditioned on a distinct αt,2 αt,3 context vector c for each target word y . i i h h h h 1 2 3 T The context vector c depends on a sequence of annotations i (h , · · · , h ) to which an encoder maps the input sentence.

中文: 新架构由双向RNN作为编码器(Sec. 3.2)和解码器组成,在解码翻译(Sec. 3.1)时通过源句来模拟搜索. 3.1 争端:一般解释 在新的模型架构中,我们在 Eq. (2) 中将每个条件概率定义为:y y t-1 t p (y |y,. y, x) = g (y, s, c.), (4) i 1 i− 1 i− 1 i i s s t-1 t 在时间i中为 RNN 隐藏状态,由i s = f (s, y, c. ) 计算 + i− 1 i− 1 i i i 与现有的编码器–decoder a- αt, 1 αt, T prach (参见 Eq. (2))不同,这里的概率取决于每个目标词 y 的不同 的 αt 2, αt, 3 上下文向量 c i h h 1, 3 T 上下文向量 c 取决于一个注释的序列 i (h, ·-, h), i 上下文向量 c 绘制输入句.

<a id="S0038"></a> Source: p.3 S0038

Original: Each ann 1 otation T h x i contains information about the whole input sequence h 1 h 2 h 3 h T with a strong focus on the parts surrounding the i-th word of the x x x x input sequence.

中文: 每个ann 1 otation T h x i 包含关于整个输入序列h 1 h 2 h 3 h T 的信息,并强烈关注x x x x 输入序列的i-th字周围的部分.

<a id="S0039"></a> Source: p.3 S0039

Original: We explain in detail how the annotations are com- 1 2 3 T puted in the next section.

中文: 我们详细解释说明是怎样放在下一节的。

<a id="S0040"></a> Source: p.3 S0040

Original: Figure 1: The graphical illus- The context vector c i is, then, computed as a weighted sum of these tration of the proposed model annotations h i : trying to generate the t-th target word y given a source c i = (cid:88) Tx α ij h j . (5) sentence (x 1 t , x 2 , . . . , x T ). j=1 The weight α of each annotation h is computed by ij j exp (e ) α = ij , (6) ij (cid:80)Tx exp (e ) k=1 ik where e = a(s , h ) ij i−1 j is an alignment model which scores how well the inputs around position j and the output at position i match.

中文: 图1:图1 因此,上下文矢量ci是作为拟议模型说明hi的这些tration的加权和来计算的:试图生成 t-th 目标词y给一个源ci=(cid:88) Tx α ij h j (5)句(x 1t,x 2,.,x T.). j=1. 每个注h的重α由ij j exp (e) α = ij , (6) ij (cid:80) Tx exp (e) k = 1 ik 其中 e = a(s), h) ij i - 1 j 是一个对齐模型,该模型对位置 j 周围的输入和位置 i 相匹配的输出进行分分多好.

<a id="S0041"></a> Source: p.3 S0041

Original: The score is based on the RNN hidden state s (just before emitting y , Eq. (4)) and the i−1 i j-th annotation h of the input sentence. j We parametrize the alignment model a as a feedforward neural network which is jointly trained with all the other components of the proposed system.

中文: 分数取自RNN隐藏状态 s (正前出y, Eq. (4))和输入句的i−1 i-th 注释 h. j (中文(简体) ). 我们把对接模式作为向导神经网络,与拟议系统的所有其他组成部分共同培训。

<a id="S0042"></a> Source: p.3 S0042

Original: Note that unlike in traditional machine translation, 3

中文: 请注意,与传统机器翻译不同,3

<a id="S0043"></a> Source: p.4 S0043

Original: Published as a conference paper at ICLR 2015 the alignment is not considered to be a latent variable.

中文: 作为会议文件发表在2015年国际林业研究中心,调整不被认为是潜在的变量。

<a id="S0044"></a> Source: p.4 S0044

Original: Instead, the alignment model directly computes a soft alignment, which allows the gradient of the cost function to be backpropagated through.

中文: 相反,对接模式直接计算出软对接,这使得成本函数的梯度被反向传播通过.

<a id="S0045"></a> Source: p.4 S0045

Original: This gradient can be used to train the alignment model as well as the whole translation model jointly.

中文: 这种梯度可用于联合训练对接模式以及整个翻译模式.

<a id="S0046"></a> Source: p.4 S0046

Original: We can understand the approach of taking a weighted sum of all the annotations as computing an expected annotation, where the expectation is over possible alignments.

中文: 我们可以理解的是,如果期望大于可能的对齐,那么将所有说明的加权总和作为计算预期注释的方法。

<a id="S0047"></a> Source: p.4 S0047

Original: Let α be a probability that ij the target word y is aligned to, or translated from, a source word x .

中文: 让α成为ij目标词y与源词x对齐或从源词x翻译的概率.

<a id="S0048"></a> Source: p.4 S0048

Original: Then, the i-th context vector i j c is the expected annotation over all the annotations with probabilities α . i ij The probability α , or its associated energy e , reflects the importance of the annotation h with ij ij j respect to the previous hidden state s in deciding the next state s and generating y .

中文: 然后, i-th 上下文向量 i j c 是所有注释的预期注释, 并带有概率 α. ij 概率 α 或 其相关能 e , 反映了 ij ij j 对前一个隐藏状态 s 在决定下一个状态 s 和 生成 y 时的注释 h 的重要性.

<a id="S0049"></a> Source: p.4 S0049

Original: Intuitively, i−1 i i this implements a mechanism of attention in the decoder.

中文: 直观地,i−1i i 这个在解码器中执行一种注意机制.

<a id="S0050"></a> Source: p.4 S0050

Original: The decoder decides parts of the source sentence to pay attention to.

中文: 解码器决定部分源句注意.

<a id="S0051"></a> Source: p.4 S0051

Original: By letting the decoder have an attention mechanism, we relieve the encoder from the burden of having to encode all information in the source sentence into a fixedlength vector.

中文: 通过让解码器具有注意机制,我们免除了解码器必须将源句中的所有信息编码成固定长向量的负担.

<a id="S0052"></a> Source: p.4 S0052

Original: With this new approach the information can be spread throughout the sequence of annotations, which can be selectively retrieved by the decoder accordingly. 3.2 ENCODER: BIDIRECTIONAL RNN FOR ANNOTATING SEQUENCES The usual RNN, described in Eq. (1), reads an input sequence x in order starting from the first symbol x to the last one x .

中文: 有了这种新的方法,信息就可以被分散在注释的序列中,由解码器据此有选择地取出. 3.2 国家:宣布安全情况的基本政策 Eq. (1)中描述的通常的RNN读取一个输入序列x,顺序从第一个符号x到最后一个符号x.

<a id="S0053"></a> Source: p.4 S0053

Original: However, in the proposed scheme, we would like the annotation 1 Tx of each word to summarize not only the preceding words, but also the following words.

中文: 然而,在拟议的办法中,我们希望每个单词的注解1 Tx不仅概括前面的单词,而且概括以下单词.

<a id="S0054"></a> Source: p.4 S0054

Original: Hence, we propose to use a bidirectional RNN (BiRNN, Schuster and Paliwal, 1997), which has been successfully used recently in speech recognition (see, e.g., Graves et al., 2013). →− A BiRNN consists of forward and backward RNN’s.

中文: 因此,我们提议采用双向RNN(BIRN、Shuster和Paliwal,1997年),该NN最近成功地用于语音识别(例如见Graves等人,2013年)。 BirNN由前向和后向的RNN组成.

<a id="S0055"></a> Source: p.4 S0055

Original: The forward RNN f reads the input sequence →− →− as it is ordered (from x to x ) and calculates a sequence of forward hidden states ( h , · · · , h ). ←− 1 Tx 1 Tx The backward RNN f reads the sequence in the reverse order (from x to x ), resulting in a ←− ←− Tx 1 sequence of backward hidden states ( h , · · · , h ). 1 Tx →− We obtain an annotation for each word x by concatenating the forward hidden state h and the j j ←− (cid:104)→− ←− (cid:105)(cid:62) backward one h , i.e., h = h (cid:62); h (cid:62) .

中文: 前置 RNN f 读取了按顺序排列的输入序列 QQ (从 x 到 x) , 并计算出前置隐藏状态的序列 ( h, ~ , h ). QX 1 Tx 1 Tx ). 后 RNN f 读取顺序为倒序(从x到x),导致后向隐藏状态(h,-,-,-,-,h.)的QX 1 序列; 通过对前向隐藏状态(h)和j j j (cid:104) → (cid:105 (cid:62)) 后向 1 h,即 h = h (cid:62); h (cid:62) 获得每个单词的注释.

<a id="S0056"></a> Source: p.4 S0056

Original: In this way, the annotation h contains the summaries j j j j j of both the preceding words and the following words.

中文: 这样,注解h就包含了以上词和以下词的简表jjj.

<a id="S0057"></a> Source: p.4 S0057

Original: Due to the tendency of RNNs to better represent recent inputs, the annotation h will be focused on the words around x .

中文: 由于RNNs倾向于更好地代表最近的投入,注h将集中在x周围的单词上.

<a id="S0058"></a> Source: p.4 S0058

Original: This sequence j j of annotations is used by the decoder and the alignment model later to compute the context vector (Eqs. (5)–(6)).

中文: 注释的这个序列jj被后来的解码器和对齐模型用来计算上下文向量(Eqs. (5)-(6)).

<a id="S0059"></a> Source: p.4 S0059

Original: See Fig. 1 for the graphical illustration of the proposed model. 4 EXPERIMENT SETTINGS We evaluate the proposed approach on the task of English-to-French translation.

中文: 提议的模型图示见图1。 4 警告设置 我们评价拟议的英文至法文翻译办法。

<a id="S0060"></a> Source: p.4 S0060

Original: We use the bilingual, parallel corpora provided by ACL WMT ’14.3 As a comparison, we also report the performance of an RNN Encoder–Decoder which was proposed recently by Cho et al. (2014a).

中文: 我们使用ACL WMT ' 14.3提供的双语并列体. 作为比较,我们还报告Cho等人最近提议的RNN编码器-Decoder的性能(2014年a)。

<a id="S0061"></a> Source: p.4 S0061

Original: We use the same training procedures and the same dataset for both models.4 4.1 DATASET WMT ’14 contains the following English-French parallel corpora: Europarl (61M words), news commentary (5.5M), UN (421M) and two crawled corpora of 90M and 272.5M words respectively, totaling 850M words.

中文: 4 4.1 DATASET WMT ' 14载有下列英法平行的corpora:Europal(61M字)、新闻评论(5.5M)、UN(421M)和两个分别由90M和272.5M字组成的爬行corpora,共计850M字。

<a id="S0062"></a> Source: p.4 S0062

Original: Following the procedure described in Cho et al. (2014a), we reduce the size of the combined corpus to have 348M words using the data selection method by Axelrod et al. (2011).5 We do not use any monolingual data other than the mentioned parallel corpora, although it may be possible to use a much larger monolingual corpus to pretrain an encoder.

中文: 按照Cho等人(2014年a)所述程序,我们利用Axelrod等人(2011年)的数据选择方法,将合并体的字数减少到348M个字。 我们不使用任何单一语言的数据,除了上述平行的corpora,尽管可能使用一个更大的单一语言体来预演编码器.

<a id="S0063"></a> Source: p.4 S0063

Original: We concatenate news-test- 3 http://www.statmt.org/wmt14/translation-task.html 4 Implementations are available at https://github.com/lisa-groundhog/GroundHog. 5 Available online at http://www-lium.univ-lemans.fr/˜schwenk/cslm_joint_paper/. 4

中文: http://www.statmt.org/wmt14/translation-task.html 互联网档案馆的存檔,存档日期2011-12-21. 4 执行情况见https://github.com/lisa-groundhog/GroundHog。 5 可上网查阅:http://www-lium.univ-lemans.fr/ schwenk/cslm joint paper/. 4

<a id="S0064"></a> Source: p.5 S0064

Original: Published as a conference paper at ICLR 2015 30 25 20 15 10 5 0 0 10 20 30 40 50 60 Sentence length erocsUELB Figure 2: The BLEU scores of the generated translations on the test set with respect to the lengths of the sen- RNNsearch-50 tences.

中文: 作为会议论文发表于ICLR 2015 30 25 20 15 10 5 0 10 20 40 50 60 句子长度 erocsUELB 图2:测试集中生成的与sen-RNNsearch-50 tents长度相关的BLEU分数.

<a id="S0065"></a> Source: p.5 S0065

Original: The results are on RNNsearch-30 the full test set which in- RNNenc-50 cludes sentences having un- RNNenc-30 known words to the models. 2012 and news-test-2013 to make a development (validation) set, and evaluate the models on the test set (news-test-2014) from WMT ’14, which consists of 3003 sentences not present in the training data.

中文: 结果是在RNNsearch-30上的全测试集上,该集在-RNNenc-50中将有un-RNNenc-30已知词的句子与模型相隔. 2012年和"新闻-测试-2013"(News-test-2013)制作开发(验证)集,并评价WMT'14的测试集(news-test-2014)上的模型,其中包含3003句没有在培训数据中出现.

<a id="S0066"></a> Source: p.5 S0066

Original: After a usual tokenization6, we use a shortlist of 30,000 most frequent words in each language to train our models.

中文: 在通常的标志化6之后,我们使用每种语言中最常见的3万个字的短名单来训练我们的模型.

<a id="S0067"></a> Source: p.5 S0067

Original: Any word not included in the shortlist is mapped to a special token ([UNK]).

中文: 未列入入围名单的任何字被绘制为特别符号([UNK])。

<a id="S0068"></a> Source: p.5 S0068

Original: We do not apply any other special preprocessing, such as lowercasing or stemming, to the data. 4.2 MODELS We train two types of models.

中文: 我们对数据不采用任何其他特殊预处理,如降低或终止。 4.2 国防部 我们训练两种模式。

<a id="S0069"></a> Source: p.5 S0069

Original: The first one is an RNN Encoder–Decoder (RNNencdec, Cho et al., 2014a), and the other is the proposed model, to which we refer as RNNsearch.

中文: 第一个是RNN编码器-Decoder(RNNCdec, Cho等,2014年a),另一个是拟议模型,我们称之为RNNsearch.

<a id="S0070"></a> Source: p.5 S0070

Original: We train each model twice: first with the sentences of length up to 30 words (RNNencdec-30, RNNsearch-30) and then with the sentences of length up to 50 word (RNNencdec-50, RNNsearch-50).

中文: 我们训练了每个型号两次:先用长度可达30个字的句子(RNNcdec-30,RNNsearch-30),再用长度可达50个字的句子(RNNCdec-50,RNNsearch-50).

<a id="S0071"></a> Source: p.5 S0071

Original: The encoder and decoder of the RNNencdec have 1000 hidden units each.7 The encoder of the RNNsearch consists of forward and backward recurrent neural networks (RNN) each having 1000 hidden units.

中文: RNNencdec的编码和解码器每个有1000个隐藏单元. RNNsearch的编码器由前向和后向的经常性神经网络(RNN)组成,每个神经网络都有1000个隐藏单元.

<a id="S0072"></a> Source: p.5 S0072

Original: In both cases, we use a multilayer network with a single maxout (Goodfellow et al., 2013) hidden layer to compute the conditional probability of each target word (Pascanu et al., 2014).

中文: 在这两种情况下,我们使用一个具有单一最大输出的多层网络(Goodfellow等,2013年)隐藏层来计算每个目标词的有条件概率(Pascanu等,2014年)。

<a id="S0073"></a> Source: p.5 S0073

Original: We use a minibatch stochastic gradient descent (SGD) algorithm together with Adadelta (Zeiler, 2012) to train each model.

中文: 我们与Adadelta(Zeiler,2012年)一起使用一个小批量分层梯度下降(SGD)算法来训练每个模型.

<a id="S0074"></a> Source: p.5 S0074

Original: Each SGD update direction is computed using a minibatch of 80 sentences.

中文: 每个 SGD 更新方向都是用80个句子的小批量来计算.

<a id="S0075"></a> Source: p.5 S0075

Original: We trained each model for approximately 5 days.

中文: 我们训练每个模特大约5天

<a id="S0076"></a> Source: p.5 S0076

Original: Once a model is trained, we use a beam search to find a translation that approximately maximizes the conditional probability (see, e.g., Graves, 2012; Boulanger-Lewandowski et al., 2013).

中文: 一旦一个模型经过了训练,我们就会使用光束搜索来找到一个能大致最大化条件概率的翻译(例如见Graves,2012;Boulanger-Lewandowski等,2013年)。

<a id="S0077"></a> Source: p.5 S0077

Original: Sutskever et al. (2014) used this approach to generate translations from their neural machine translation model.

中文: Sutskever等人(2014年)使用这种方法从他们的神经机翻译模型中生成翻译.

<a id="S0078"></a> Source: p.5 S0078

Original: For more details on the architectures of the models and training procedure used in the experiments, see Appendices A and B. 5 RESULTS 5.1 QUANTITATIVE RESULTS In Table 1, we list the translation performances measured in BLEU score.

中文: 关于试验中所使用的模型和培训程序的结构的更详细情况,见附录A和B. 5 ResSULTS 5.1 QUANTITATIATIATIATIATIATIATIA ResULTS 在表1中,我们列出了以BLEU分数衡量的翻译性能.

<a id="S0079"></a> Source: p.5 S0079

Original: It is clear from the table that in all the cases, the proposed RNNsearch outperforms the conventional RNNencdec.

中文: 从表中可以明显看出,在所有情况下,拟议的RNNsearch都超过了常规的RNNencdec。

<a id="S0080"></a> Source: p.5 S0080

Original: More importantly, the performance of the RNNsearch is as high as that of the conventional phrase-based translation system (Moses), when only the sentences consisting of known words are considered.

中文: 更重要的是,RNNsearch的性能和传统的以短语为基础的翻译系统(摩西)一样高,当时只考虑由已知词组成的句子.

<a id="S0081"></a> Source: p.5 S0081

Original: This is a significant achievement, considering that Moses uses a separate monolingual corpus (418M words) in addition to the parallel corpora we used to train the RNNsearch and RNNencdec. 6 We used the tokenization script from the open-source machine translation package, Moses. 7 In this paper, by a ’hidden unit’, we always mean the gated hidden unit (see Appendix A.1.1). 5

中文: 这是一个重大的成就,考虑到摩西除了我们用来训练RNNSearch和RNenencdec的相平行的Corpora之外,还使用了单独的单一语言体(418M字). 6我们使用了开源机器翻译包中的符号化脚本"摩西". 7 在本文中,由 " 隐蔽单位 " ,我们总是指被屏蔽的隐蔽单位(见附录A.1.1)。 页:1

<a id="S0082"></a> Source: p.6 S0082

Original: Published as a conference paper at ICLR 2015 ehT tnemeerga no eht naeporuE cimonocE aerA saw dengis ni tsuguA 2991 . >dne< L' accord sur la zone économique européenne a été signé en août 1992 . <end> tI dluohs eb deton taht eht eniram tnemnorivne si eht tsael nwonk fo stnemnorivne . >dne< Il convient de noter que l' environnement marin est le moins connu de l' environnement . <end> (a) (b) noitcurtseD fo eht tnempiuqe snaem taht airyS nac on regnol ecudorp wen lacimehc snopaew . >dne< La destruction de l' équipement signifie que la Syrie ne peut plus produire de nouvelles armes chimiques . <end> " sihT lliw egnahc ym erutuf htiw ym ylimaf , " eht nam dias . >dne< " Cela va changer mon avenir avec ma famille " , a dit l' homme . <end> (c) (d) Figure 3: Four sample alignments found by RNNsearch-50.

中文: 作为会议论文发表于ICLR 2015 ehT tnemeerga no eht naeporuE cimonocE erA saw dengis ni tsuguA 2991. > dne < 欧洲经济区协议 a été signé en au-t 1992. > 结束。 tI dluohs eb deton teht eniram tenimnorivne si etht sael nwonk fo stnemnnorivne. > dne < Il convient de noer que l'environment marin est le moins connu de l'environment. > (a) (b) noitcurtseD fo eht tneimpiuqe snaem taht airyS nac on regnol ecudorp wen lacimehc snopaew. >dne < La de de l' équiment signifie que la Syrie ne peut + produire de Newa arms chimiques. > end > " sihT liht egnahc ym erutuf htiw ym ylimaf " , " eht nam dias. > dne < Cela va changer mon avenir avec ma famille " ,一个dit l'human. < end > (c) (d) 图3:RNSearch-50发现的4个样本对齐。

<a id="S0083"></a> Source: p.6 S0083

Original: The x-axis and y-axis of each plot correspond to the words in the source sentence (English) and the generated translation (French), respectively.

中文: 每个情节的x轴和y轴分别对应出源句(英语)和生成的译名(法语)中的单词.

<a id="S0084"></a> Source: p.6 S0084

Original: Each pixel shows the weight α of the annotation of the j-th source word for the i-th ij target word (see Eq. (6)), in grayscale (0: black, 1: white). (a) an arbitrary sentence. (b–d) three randomly selected samples among the sentences without any unknown words and of length between 10 and 20 words from the test set.

中文: 每个像素都显示为i-th ij目标单词的j-th源词注释的重α(参见Eq. (6)),以灰度(0:黑色,1:白色)表示. (a)任意的句子. (b-d)在句子中随机抽取出三个样本,没有任何未知的字,从测试集中抽取出长度为10到20个字.

<a id="S0085"></a> Source: p.6 S0085

Original: One of the motivations behind the proposed approach was the use of a fixed-length context vector in the basic encoder–decoder approach.

中文: 拟议方法的动机之一是在基本编码器-解码器方法中使用固定长度上下文矢量。

<a id="S0086"></a> Source: p.6 S0086

Original: We conjectured that this limitation may make the basic encoder–decoder approach to underperform with long sentences.

中文: 我们猜想,这种限制可能使基本的编码器-解码器方法在长句子的情况下表现不佳。

<a id="S0087"></a> Source: p.6 S0087

Original: In Fig. 2, we see that the performance of RNNencdec dramatically drops as the length of the sentences increases.

中文: 在图2中,我们看到RNNencdec的性能随着刑期的延长而急剧下降.

<a id="S0088"></a> Source: p.6 S0088

Original: On the other hand, both RNNsearch-30 and RNNsearch-50 are more robust to the length of the sentences.

中文: 另一方面,RNNsearch-30和RNNsearch-50都比句子长度更强.

<a id="S0089"></a> Source: p.6 S0089

Original: RNNsearch- 50, especially, shows no performance deterioration even with sentences of length 50 or more.

中文: RNNsearch - 50,特别是,即使长度在50分或50分以上,也没有显示性能恶化.

<a id="S0090"></a> Source: p.6 S0090

Original: This superiority of the proposed model over the basic encoder–decoder is further confirmed by the fact that the RNNsearch-30 even outperforms RNNencdec-50 (see Table 1). 6

中文: 拟议的模型优于基本编码器-解码器进一步证实了RNNsearch-30甚至优于RNNencdec-50(见表1)。 6个

<a id="S0091"></a> Source: p.7 S0091

Original: Published as a conference paper at ICLR 2015 Table 1: BLEU scores of the trained models com- Model All No UNK◦ puted on the test set.

中文: 作为会议文件发表于ICLR 2015 表1:经培训的模型的BLEU分数

<a id="S0092"></a> Source: p.7 S0092

Original: The second and third columns show respectively the scores on all the sentences and, RNNencdec-30 13.93 24.19 on the sentences without any unknown word in them- RNNsearch-30 21.50 31.44 selves and in the reference translations.

中文: 第二栏和第三栏分别列出所有句子的分数,以及无任何未知词的句子的RNNencdec-30 13.93 24.19 -- -- RNNsearch-30 21.50 31.44本身和参考翻译。

<a id="S0093"></a> Source: p.7 S0093

Original: Note that RNNencdec-50 17.82 26.71 RNNsearch-50(cid:63) was trained much longer until the RNNsearch-50 26.75 34.16 performance on the development set stopped improv- RNNsearch-50(cid:63) 28.45 36.15 ing. (◦) We disallowed the models to generate [UNK] Moses 33.30 35.63 tokens when only the sentences having no unknown words were evaluated (last column). 5.2 QUALITATIVE ANALYSIS 5.2.1 ALIGNMENT The proposed approach provides an intuitive way to inspect the (soft-)alignment between the words in a generated translation and those in a source sentence.

中文: 注意RNNencdec-50 17.82 26.71 RNNsearch-50(cid:63) 训练时间要长得多,直到RNNsearch-50 26.75 34.16 开发套装的性能停止即兴-RNNsearch-50(cid:63 28.45 36.15 ing. (◦) 我们拒绝模型生成[UNK]摩西 33.30 35.63 令牌,当时只评价了没有未知词的句子(上一栏). 5.2 定性分析 5.2.1 拟议的方法提供了一种直观的方法,可以检查生成的译文中的单词与源句中的单词之间的(软)相接.

<a id="S0094"></a> Source: p.7 S0094

Original: This is done by visualizing the annotation weights α from Eq. (6), as in Fig. 3.

中文: 如图3所示,这是通过可视化Eq. (6)中α的注释权重来实现的。

<a id="S0095"></a> Source: p.7 S0095

Original: Each row of a matrix in each plot indicates the weights ij associated with the annotations.

中文: 每个图中的一行矩阵都表示与注释相联的权重ij.

<a id="S0096"></a> Source: p.7 S0096

Original: From this we see which positions in the source sentence were considered more important when generating the target word.

中文: 由此可见,在生成目标词时,源句中哪些立场被认为更为重要.

<a id="S0097"></a> Source: p.7 S0097

Original: We can see from the alignments in Fig. 3 that the alignment of words between English and French is largely monotonic.

中文: 从图3中的对接中可以看出,英语和法语的对接基本上是单调的.

<a id="S0098"></a> Source: p.7 S0098

Original: We see strong weights along the diagonal of each matrix.

中文: 我们看到每个基质的对角线上有很强的分量。

<a id="S0099"></a> Source: p.7 S0099

Original: However, we also observe a number of non-trivial, non-monotonic alignments.

中文: 然而,我们也观察到一些非三角性,非莫诺式的对接.

<a id="S0100"></a> Source: p.7 S0100

Original: Adjectives and nouns are typically ordered differently between French and English, and we see an example in Fig. 3 (a).

中文: 形容词和名词一般在法语和英语之间顺序不同,我们在图3(a)中看到一个例子.

<a id="S0101"></a> Source: p.7 S0101

Original: From this figure, we see that the model correctly translates a phrase [European Economic Area] into [zone e´conomique europe´en].

中文: 从这个图中,我们看到模型正确地将一个短语[欧洲经济区]翻译为[zone e'conomique Europe'en].

<a id="S0102"></a> Source: p.7 S0102

Original: The RNNsearch was able to correctly align [zone] with [Area], jumping over the two words ([European] and [Economic]), and then looked one word back at a time to complete the whole phrase [zone e´conomique europe´enne].

中文: RNNsearch能够正确地将[zone]与[Area]对齐,跳出两个单词([欧]和[经济]),然后一次翻出一个单词来完成整个短语[zone e'conomique Europe'enne].

<a id="S0103"></a> Source: p.7 S0103

Original: The strength of the soft-alignment, opposed to a hard-alignment, is evident, for instance, from Fig. 3 (d).

中文: 例如从图3(d)可以看出,软调的强度,反对硬调。

<a id="S0104"></a> Source: p.7 S0104

Original: Consider the source phrase [the man] which was translated into [l’ homme].

中文: 想想被译为 " 人 " 的源词。

<a id="S0105"></a> Source: p.7 S0105

Original: Any hard alignment will map [the] to [l’] and [man] to [homme].

中文: 任何硬调将映射到[我]和[人 。

<a id="S0106"></a> Source: p.7 S0106

Original: This is not helpful for translation, as one must consider the word following [the] to determine whether it should be translated into [le], [la], [les] or [l’].

中文: 这对翻译没有帮助,因为人们必须考虑后面的词,以确定应将其译成[le]、[la]、[les]或[l ' 。

<a id="S0107"></a> Source: p.7 S0107

Original: Our soft-alignment solves this issue naturally by letting the model look at both [the] and [man], and in this example, we see that the model was able to correctly translate [the] into [l’].

中文: 我们的软调自然地解决了这个问题,让模型同时观察[人]和[人],在这个例子中,我们看到模型能够将[人]正确翻译为[l'.

<a id="S0108"></a> Source: p.7 S0108

Original: We observe similar behaviors in all the presented cases in Fig. 3.

中文: 在图3中,我们看到所有提交的案件都有类似行为。

<a id="S0109"></a> Source: p.7 S0109

Original: An additional benefit of the soft alignment is that it naturally deals with source and target phrases of different lengths, without requiring a counter-intuitive way of mapping some words to or from nowhere ([NULL]) (see, e.g., Chapters 4 and 5 of Koehn, 2010). 5.2.2 LONG SENTENCES As clearly visible from Fig. 2 the proposed model (RNNsearch) is much better than the conventional model (RNNencdec) at translating long sentences.

中文: 软对接的另一个好处是,它自然地处理不同长度的源和目标短语,而不需要反直观的方式将一些单词映射入出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出入出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出入出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出出 5.2.2 长期传感器 从图2可明显看出,拟议模型(RNNsearch)在翻译长句时比常规模型(RN Nencdec)要好得多.

<a id="S0110"></a> Source: p.7 S0110

Original: This is likely due to the fact that the RNNsearch does not require encoding a long sentence into a fixed-length vector perfectly, but only accurately encoding the parts of the input sentence that surround a particular word.

中文: 这可能是由于RNNsearch并不要求将长句编码成一整段的固定长向量,而只是精确地编码了输入句中围绕特定词的部分.

<a id="S0111"></a> Source: p.7 S0111

Original: As an example, consider this source sentence from the test set: An admitting privilege is the right of a doctor to admit a patient to a hospital or a medical centre to carry out a diagnosis or a procedure, based on his status as a health care worker at a hospital.

中文: 例如,从测试集中考虑这一源句: 承认的特权是,医生有权根据病人在医院的保健工作者的身份,将病人送入医院或医疗中心进行诊断或程序。

<a id="S0112"></a> Source: p.7 S0112

Original: The RNNencdec-50 translated this sentence into: Un privilege d’admission est le droit d’un me´decin de reconnaˆıtre un patient a l’hoˆpital ou un centre me´dical d’un diagnostic ou de prendre un diagnostic en fonction de son e´tat de sante´. 7

中文: RNNenencdec-50将这句话译为: " 不得优先承认我的权利 " 。 第7条

<a id="S0113"></a> Source: p.8 S0113

Original: Published as a conference paper at ICLR 2015 The RNNencdec-50 correctly translated the source sentence until [a medical center].

中文: 以会议论文发表于ICLR 2015 The RNNencdec-50 正确翻译出源句至[一个医疗中心].

<a id="S0114"></a> Source: p.8 S0114

Original: However, from there on (underlined), it deviated from the original meaning of the source sentence.

中文: 然而,从此(下划线),它偏离了源句的原始含义.

<a id="S0115"></a> Source: p.8 S0115

Original: For instance, it replaced [based on his status as a health care worker at a hospital] in the source sentence with [en fonction de son e´tat de sante´] (“based on his state of health”).

中文: 例如,它[根据他在医院的保健工作者身份]在来源句中取而代之的是[根据他的健康情况]。

<a id="S0116"></a> Source: p.8 S0116

Original: On the other hand, the RNNsearch-50 generated the following correct translation, preserving the whole meaning of the input sentence without omitting any details: Un privilege d’admission est le droit d’un me´decin d’admettre un patient a un hoˆpital ou un centre me´dical pour effectuer un diagnostic ou une proce´dure, selon son statut de travailleur des soins de sante´ a` l’hoˆpital.

中文: 另一方面,"RNNsearch-50"生成了以下正确的翻译,保留了输入句的全部含义而不会省略任何细节: 以我为中心,

<a id="S0117"></a> Source: p.8 S0117

Original: Let us consider another sentence from the test set: This kind of experience is part of Disney’s efforts to ”extend the lifetime of its series and build new relationships with audiences via digital platforms that are becoming ever more important,” he added.

中文: 我们再考虑一下试验集中的一句: 他补充说,这种经验是迪士尼努力“延长其系列的寿命并通过日益重要的数字平台与观众建立新的关系”。

<a id="S0118"></a> Source: p.8 S0118

Original: The translation by the RNNencdec-50 is Ce type d’expe´rience fait partie des initiatives du Disney pour ”prolonger la dure´e de vie de ses nouvelles et de de´velopper des liens avec les lecteurs nume´riques qui deviennent plus complexes.

中文: 由RNNencdec-50翻译为“Dexpe'rance既成部分,

<a id="S0119"></a> Source: p.8 S0119

Original: As with the previous example, the RNNencdec began deviating from the actual meaning of the source sentence after generating approximately 30 words (see the underlined phrase).

中文: 与前一个例子一样,RNNencdec在生成了大约30个单词(见下划线短语)后,开始偏离源句的实际含义.

<a id="S0120"></a> Source: p.8 S0120

Original: After that point, the quality of the translation deteriorates, with basic mistakes such as the lack of a closing quotation mark.

中文: 之后翻译质量恶化,缺少收尾引号等基本错误.

<a id="S0121"></a> Source: p.8 S0121

Original: Again, the RNNsearch-50 was able to translate this long sentence correctly: Ce genre d’expe´rience fait partie des efforts de Disney pour ”prolonger la dure´e de vie de ses se´ries et cre´er de nouvelles relations avec des publics via des plateformes nume´riques de plus en plus importantes”, a-t-il ajoute´.

中文: 同样,RNNsearch-50能够正确翻译出这句长句: 迪士尼的努力“通过板块形式促进公众关系”“一提,

<a id="S0122"></a> Source: p.8 S0122

Original: In conjunction with the quantitative results presented already, these qualitative observations confirm our hypotheses that the RNNsearch architecture enables far more reliable translation of long sentences than the standard RNNencdec model.

中文: 结合已经提出的定量结果,这些定性观测证实了我们的假设,即RNNsearch架构能够比标准RNNencdec模型更可靠地翻译长句.

<a id="S0123"></a> Source: p.8 S0123

Original: In Appendix C, we provide a few more sample translations of long source sentences generated by the RNNencdec-50, RNNsearch-50 and Google Translate along with the reference translations. 6 RELATED WORK 6.1 LEARNING TO ALIGN A similar approach of aligning an output symbol with an input symbol was proposed recently by Graves (2013) in the context of handwriting synthesis.

中文: 在附录C中,我们提供了更多由RNNencdec-50,RNNsearch-50和Google Translate以及参考翻译产生的长源句的样本翻译. 6 相关工作 6.1 学习自治 最近格雷夫斯(2013年)在笔迹合成方面提出了类似的将输出符号与输入符号相匹配的方法.

<a id="S0124"></a> Source: p.8 S0124

Original: Handwriting synthesis is a task where the model is asked to generate handwriting of a given sequence of characters.

中文: 手写合成是要求模型生成给定字符序列的笔迹的任务.

<a id="S0125"></a> Source: p.8 S0125

Original: In his work, he used a mixture of Gaussian kernels to compute the weights of the annotations, where the location, width and mixture coefficient of each kernel was predicted from an alignment model.

中文: 在他的作品中他使用高斯内核的混合物来计算注释的分量,其中每个内核的位置,宽度和混合系数都是从一个对齐模型来预测的.

<a id="S0126"></a> Source: p.8 S0126

Original: More specifically, his alignment was restricted to predict the location such that the location increases monotonically.

中文: 更具体地说,他的对接被限制在预测位置,以至于位置单调地增加.

<a id="S0127"></a> Source: p.8 S0127

Original: The main difference from our approach is that, in (Graves, 2013), the modes of the weights of the annotations only move in one direction.

中文: 与我们的方法的主要区别在于,在(Graves,2013年)中,注释的权重模式只朝一个方向移动.

<a id="S0128"></a> Source: p.8 S0128

Original: In the context of machine translation, this is a severe limitation, as (long-distance) reordering is often needed to generate a grammatically correct translation (for instance, English-to-German).

中文: 在机器翻译方面,这是一个严重的限制,因为(长途)重排经常需要生成语法上正确的翻译(例如英语对德语).

<a id="S0129"></a> Source: p.8 S0129

Original: Our approach, on the other hand, requires computing the annotation weight of every word in the source sentence for each word in the translation.

中文: 另一方面,我们的方法需要计算出源句中每个词对翻译中每个词的注释权重.

<a id="S0130"></a> Source: p.8 S0130

Original: This drawback is not severe with the task of translation in which most of input and output sentences are only 15–40 words.

中文: 这种缺点并不严重,因为翻译任务中大多数输入和输出句子只有15–40个字.

<a id="S0131"></a> Source: p.8 S0131

Original: However, this may limit the applicability of the proposed scheme to other tasks. 8

中文: 然而,这可能会限制拟议办法对其他任务的适用。 第8条

<a id="S0132"></a> Source: p.9 S0132

Original: Published as a conference paper at ICLR 2015 6.2 NEURAL NETWORKS FOR MACHINE TRANSLATION Since Bengio et al. (2003) introduced a neural probabilistic language model which uses a neural network to model the conditional probability of a word given a fixed number of the preceding words, neural networks have widely been used in machine translation.

中文: 作为会议论文发表于ICLR 2015 6.2 NEURAL NETWORKS for MACHINE TRANSLATION Since Bengio等(2003年),引入了神经概率语言模型,使用神经网络对给定了上个词数的词的有条件概率进行模型,神经网络在机器翻译中被广泛使用.

<a id="S0133"></a> Source: p.9 S0133

Original: However, the role of neural networks has been largely limited to simply providing a single feature to an existing statistical machine translation system or to re-rank a list of candidate translations provided by an existing system.

中文: 然而,神经网络的作用基本上仅限于简单地为现有的统计机器翻译系统提供一个单一的功能,或者重新排列现有系统提供的候选翻译名单.

<a id="S0134"></a> Source: p.9 S0134

Original: For instance, Schwenk (2012) proposed using a feedforward neural network to compute the score of a pair of source and target phrases and to use the score as an additional feature in the phrase-based statistical machine translation system.

中文: 例如,Schwenk(2012年)提议使用向上神经网络来计算一对源和目标短语的得分,并将得分作为基于短语的统计机器翻译系统中的附加功能.

<a id="S0135"></a> Source: p.9 S0135

Original: More recently, Kalchbrenner and Blunsom (2013) and Devlin et al. (2014) reported the successful use of the neural networks as a sub-component of the existing translation system.

中文: 最近,Kalchbrenner和Blunsom(2013年)和Devlin等人(2014年)报告说,成功地利用神经网络作为现有翻译系统的分组成部分。

<a id="S0136"></a> Source: p.9 S0136

Original: Traditionally, a neural network trained as a target-side language model has been used to rescore or rerank a list of candidate translations (see, e.g., Schwenk et al., 2006).

中文: 传统上,一个作为目标边语言模型而训练的神经网络被用来重新记录或重新排列候选翻译名单(例如见Schwenk等,2006年)。

<a id="S0137"></a> Source: p.9 S0137

Original: Although the above approaches were shown to improve the translation performance over the stateof-the-art machine translation systems, we are more interested in a more ambitious objective of designing a completely new translation system based on neural networks.

中文: 尽管上述方法显示出来比最先进的机器翻译系统来改进翻译性能,但我们更感兴趣的是更雄心勃勃的目标是设计一个基于神经网络的全新翻译系统.

<a id="S0138"></a> Source: p.9 S0138

Original: The neural machine translation approach we consider in this paper is therefore a radical departure from these earlier works.

中文: 因此,本文中我们所考虑的神经机翻译方法与这些早期的作品是根本的背离.

<a id="S0139"></a> Source: p.9 S0139

Original: Rather than using a neural network as a part of the existing system, our model works on its own and generates a translation from a source sentence directly. 7 CONCLUSION The conventional approach to neural machine translation, called an encoder–decoder approach, encodes a whole input sentence into a fixed-length vector from which a translation will be decoded.

中文: 我们的模型与其使用神经网络作为现有系统的一部分,不如自己工作,直接生成源句的翻译. 7 结论 常规的神经机翻译方法,称为编码器–解码器方法,将整个输入句子编码为固定长向量,从中解码出翻译.

<a id="S0140"></a> Source: p.9 S0140

Original: We conjectured that the use of a fixed-length context vector is problematic for translating long sentences, based on a recent empirical study reported by Cho et al. (2014b) and Pouget-Abadie et al. (2014).

中文: 根据Cho等人(2014年b)和Pouget-Abadie等人(2014年)最近报告的一项经验研究,我们推测使用固定长的上下文向量对翻译长句有问题。

<a id="S0141"></a> Source: p.9 S0141

Original: In this paper, we proposed a novel architecture that addresses this issue.

中文: 在这份文件中,我们提出了一个解决这一问题的新结构。

<a id="S0142"></a> Source: p.9 S0142

Original: We extended the basic encoder–decoder by letting a model (soft-)search for a set of input words, or their annotations computed by an encoder, when generating each target word.

中文: 我们扩展了基本的编码器-解码器,让模型(soft-)在生成每个目标词时搜索一组输入词,或用编码器计算的注释。

<a id="S0143"></a> Source: p.9 S0143

Original: This frees the model from having to encode a whole source sentence into a fixed-length vector, and also lets the model focus only on information relevant to the generation of the next target word.

中文: 这让模型不必将整个源句编码成固定长向量,也让模型只关注与下一个目标词生成有关的信息.

<a id="S0144"></a> Source: p.9 S0144

Original: This has a major positive impact on the ability of the neural machine translation system to yield good results on longer sentences.

中文: 这对神经机器翻译系统在较长的句子上产生良好效果的能力有着重大的积极影响.

<a id="S0145"></a> Source: p.9 S0145

Original: Unlike with the traditional machine translation systems, all of the pieces of the translation system, including the alignment mechanism, are jointly trained towards a better log-probability of producing correct translations.

中文: 与传统机器翻译系统不同的是,翻译系统的所有部分,包括校正机制,都经过了联合培训,以提高生成正确翻译的日志可能性.

<a id="S0146"></a> Source: p.9 S0146

Original: We tested the proposed model, called RNNsearch, on the task of English-to-French translation.

中文: 我们测试了拟议的模型,称为RNNsearch, 以完成英文到法文的翻译任务。

<a id="S0147"></a> Source: p.9 S0147

Original: The experiment revealed that the proposed RNNsearch outperforms the conventional encoder–decoder model (RNNencdec) significantly, regardless of the sentence length and that it is much more robust to the length of a source sentence.

中文: 实验显示,无论句子长度如何,拟议的RNNSearch都大大地超过了常规编码器-解码器模型(RNNCdec),并且对源句长度来说更强.

<a id="S0148"></a> Source: p.9 S0148

Original: From the qualitative analysis where we investigated the (soft-)alignment generated by the RNNsearch, we were able to conclude that the model can correctly align each target word with the relevant words, or their annotations, in the source sentence as it generated a correct translation.

中文: 从我们调查了RNNsearch产生的(软-)对齐的定性分析中,我们得出了这样的结论,即该模型能够正确地将每个目标词与源句中的相关词,或其说明相配合,因为它产生了正确的翻译.

<a id="S0149"></a> Source: p.9 S0149

Original: Perhaps more importantly, the proposed approach achieved a translation performance comparable to the existing phrase-based statistical machine translation.

中文: 也许更重要的是,拟议方法实现了与现有的以短语为基础的统计机器翻译相类似的翻译绩效。

<a id="S0150"></a> Source: p.9 S0150

Original: It is a striking result, considering that the proposed architecture, or the whole family of neural machine translation, has only been proposed as recently as this year.

中文: 考虑到拟议的建筑,或者整个神经机器翻译家族,直到今年才提出,这是一个引人注目的结果.

<a id="S0151"></a> Source: p.9 S0151

Original: We believe the architecture proposed here is a promising step toward better machine translation and a better understanding of natural languages in general.

中文: 我们认为,这里提出的结构是朝着更好的机器翻译和更好地理解一般自然语言迈出的有希望的一步。

<a id="S0152"></a> Source: p.9 S0152

Original: One of challenges left for the future is to better handle unknown, or rare words.

中文: 未来遗留的挑战之一是更好地处理未知或罕见的文字。

<a id="S0153"></a> Source: p.9 S0153

Original: This will be required for the model to be more widely used and to match the performance of current state-of-the-art machine translation systems in all contexts. 9

中文: 这将需要使模型得到更广泛的使用,并匹配当前最先进的机器翻译系统在所有情况下的性能. 第9条

<a id="S0154"></a> Source: p.10 S0154

Original: Published as a conference paper at ICLR 2015 ACKNOWLEDGMENTS The authors would like to thank the developers of Theano (Bergstra et al., 2010; Bastien et al., 2012).

中文: 作为会议论文发表于ICLR 2015 ACKNOWLDGMENTS 提交人感谢Theano的开发者(Bergstra等人,2010年;Bastien等人,2012年)。

<a id="S0155"></a> Source: p.10 S0155

Original: We acknowledge the support of the following agencies for research funding and computing support: NSERC, Calcul Que´bec, Compute Canada, the Canada Research Chairs and CIFAR.

中文: 我们赞赏下列机构对研究资金和计算支持的支持:NSERC、Calcul Que'bec、计算加拿大、加拿大研究主席和CIFAR。

<a id="S0156"></a> Source: p.10 S0156

Original: Bahdanau thanks the support from Planet Intelligent Systems GmbH.

中文: Bahdanau感谢星球智能系统GmbH的支持.

<a id="S0157"></a> Source: p.10 S0157

Original: We also thank Felix Hill, Bart van Merrie´nboer, Jean Pouget-Abadie, Coline Devin and Tae-Ho Kim.

中文: 我们还感谢费利克斯·希尔、巴特·范梅里·恩博尔、让·普盖特-阿巴迪、科琳·德文和泰浩·金。

<a id="S0158"></a> Source: p.10 S0158

Original: REFERENCES Axelrod, A., He, X., and Gao, J. (2011).

中文: 参见Axelrod,A.,He,X.和Gao,J.(2011年).

<a id="S0159"></a> Source: p.10 S0159

Original: Domain adaptation via pseudo in-domain data selection.

中文: 通过伪域内数据选择进行域内修改.

<a id="S0160"></a> Source: p.10 S0160

Original: In Proceedings of the ACL Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 355–362.

中文: ACL自然语言处理经验方法会议记录,第355-362页。

<a id="S0161"></a> Source: p.10 S0161

Original: Association for Computational Linguistics.

中文: 计算语言学协会.

<a id="S0162"></a> Source: p.10 S0162

Original: Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I. J., Bergeron, A., Bouchard, N., and Bengio, Y. (2012).

中文: Bastien, F., Lamblin, P., P., Pascanu, R., Bergstra, J., Goodfellow, I. J., Bergeron, A., Bouchard, N., and Bengio, Y.(2012).

<a id="S0163"></a> Source: p.10 S0163

Original: Theano: new features and speed improvements.

中文: 西亚诺:新特征和速度改进.

<a id="S0164"></a> Source: p.10 S0164

Original: Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop.

中文: 深度学习与无监督的特色学习 NIPS 2012研讨会.

<a id="S0165"></a> Source: p.10 S0165

Original: Bengio, Y., Simard, P., and Frasconi, P. (1994).

中文: Bengio, Y., Simard, P., and Frasconi, P.(1994年).

<a id="S0166"></a> Source: p.10 S0166

Original: Learning long-term dependencies with gradient descent is difficult.

中文: 学习具有梯度下降的长期依赖性是困难的。

<a id="S0167"></a> Source: p.10 S0167

Original: IEEE Transactions on Neural Networks, 5(2), 157–166.

中文: IEEE交易在神经网络,5(2),157–166.

<a id="S0168"></a> Source: p.10 S0168

Original: Bengio, Y., Ducharme, R., Vincent, P., and Janvin, C. (2003). A neural probabilistic language model. J.

中文: Bengio, Y., Ducharme, R., Vincent, P.,和Janvin, C. (2003年). 神经概率语言模型. J.

<a id="S0169"></a> Source: p.10 S0169

Original: Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde- Farley, D., and Bengio, Y. (2010).

中文: Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., Bengio, Y.(2010).

<a id="S0170"></a> Source: p.10 S0170

Original: Theano: a CPU and GPU math expression compiler.

中文: Theano:一个CPU和GPU数学表达式编译器.

<a id="S0171"></a> Source: p.10 S0171

Original: In Proceedings of the Python for Scientific Computing Conference (SciPy).

中文: 在Python科学计算会议的议事录(SciPy)中.

<a id="S0172"></a> Source: p.10 S0172

Original: Boulanger-Lewandowski, N., Bengio, Y., and Vincent, P. (2013).

中文: Boulanger-Lewandowski, N., Bengio, Y., and Vincent, P. (2013年).

<a id="S0173"></a> Source: p.10 S0173

Original: Audio chord recognition with recurrent neural networks.

中文: 音频和弦识别与经常性神经网络.

<a id="S0174"></a> Source: p.10 S0174

Original: Cho, K., van Merrienboer, B., Gulcehre, C., Bougares, F., Schwenk, H., and Bengio, Y. (2014a).

中文: Cho, K., van Merrienboer, B., Gulcehre, C., Bougares, F., Schwenk, H.,和 Bengio, Y. (2014a).

<a id="S0175"></a> Source: p.10 S0175

Original: Learning phrase representations using RNN encoder-decoder for statistical machine translation.

中文: 使用 RNN 编码器-解码器进行统计机器翻译的学习短语表达.

<a id="S0176"></a> Source: p.10 S0176

Original: In Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014). to appear.

中文: 《自然语言处理实验方法记录》(EMNLP,2014年)。 出现。

<a id="S0177"></a> Source: p.10 S0177

Original: Cho, K., van Merrie¨nboer, B., Bahdanau, D., and Bengio, Y. (2014b).

中文: Cho、K.、van Merrie'nboer、B.、Bahdanau、D.和Bengio,Y.(2014年b)。

<a id="S0178"></a> Source: p.10 S0178

Original: On the properties of neural machine translation: Encoder–Decoder approaches.

中文: 关于神经机器翻译的属性: Encoder-Decoder 方法.

<a id="S0179"></a> Source: p.10 S0179

Original: In Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation. to appear.

中文: 第八次统计翻译的语法、语义和结构讲习班。 出现。

<a id="S0180"></a> Source: p.10 S0180

Original: Devlin, J., Zbib, R., Huang, Z., Lamar, T., Schwartz, R., and Makhoul, J. (2014).

中文: 德夫林,J.,兹比布,R.,黄克,Z.,拉马尔,T.,施瓦茨,R.,和马克胡尔,J.(2014).

<a id="S0181"></a> Source: p.10 S0181

Original: Fast and robust neural network joint models for statistical machine translation.

中文: 快速而有力的神经网络联合模型,用于统计机器翻译.

<a id="S0182"></a> Source: p.10 S0182

Original: In Association for Computational Linguistics.

中文: 在计算语言学协会。

<a id="S0183"></a> Source: p.10 S0183

Original: Forcada, M. L. and N˜ eco, R. P. (1997).

中文: Forcada, M. L.和N. 生态,R. P. (1997年)。

<a id="S0184"></a> Source: p.10 S0184

Original: Recursive hetero-associative memories for translation.

中文: 递归异体-关联记忆作翻译.

<a id="S0185"></a> Source: p.10 S0185

Original: Cabestany, editors, Biological and Artificial Computation: From Neuroscience to Technology, volume 1240 of Lecture Notes in Computer Science, pages 453–462.

中文: Cabestany,编辑,生物和人工计算:从神经科学到技术,计算机科学讲座说明 第1240卷,第453-462页.

<a id="S0186"></a> Source: p.10 S0186

Original: Goodfellow, I., Warde-Farley, D., Mirza, M., Courville, A., and Bengio, Y. (2013).

中文: Goodfellow, I., Warde-Farley, D., Mirza, M., Courville, A.,和 Bengio, Y.(2013).

<a id="S0187"></a> Source: p.10 S0187

Original: In Proceedings of The 30th International Conference on Machine Learning, pages 1319– 1327.

中文: 《第三十届机器学习国际会议记录》,第1319-1327页。

<a id="S0188"></a> Source: p.10 S0188

Original: Sequence transduction with recurrent neural networks.

中文: 序列传递与反复出现的神经网络.

<a id="S0189"></a> Source: p.10 S0189

Original: In Proceedings of the 29th International Conference on Machine Learning (ICML 2012).

中文: 第29届机器学习国际会议记录(ICML 2012).

<a id="S0190"></a> Source: p.10 S0190

Original: Generating sequences with recurrent neural networks. arXiv:1308.0850 [cs.NE].

中文: 以反复出现的神经网络生成序列. arXiv:1308.0850 [cs.NE] (中文(中国大陆) ).

<a id="S0191"></a> Source: p.10 S0191

Original: Graves, A., Jaitly, N., and Mohamed, A.-R. (2013).

中文: Graves, A., Jaitly, N., 和Mohamed, A.-R. (2013).

<a id="S0192"></a> Source: p.10 S0192

Original: Hybrid speech recognition with deep bidirectional LSTM.

中文: 具有深双向LSTM的混合语音识别.

<a id="S0193"></a> Source: p.10 S0193

Original: In Automatic Speech Recognition and Understanding (ASRU), 2013 IEEE Workshop on, pages 273–278. 10

中文: 《自动语音识别和理解》(ASRU),2013年,IEEE讲习班,第273-278页。

<a id="S0194"></a> Source: p.11 S0194

Original: Published as a conference paper at ICLR 2015 Hermann, K. and Blunsom, P. (2014).

中文: 作为会议论文发表于ICLR 2015 Hermann, K. and Blunsom, P. (2014).

<a id="S0195"></a> Source: p.11 S0195

Original: Multilingual distributed representations without word alignment.

中文: 多语种分布式表述,不进行字词对接.

<a id="S0196"></a> Source: p.11 S0196

Original: In Proceedings of the Second International Conference on Learning Representations (ICLR 2014).

中文: 《第二届国际学习代表会议记录》(ICLR,2014年)。

<a id="S0197"></a> Source: p.11 S0197

Original: Untersuchungen zu dynamischen neuronalen Netzen.

中文: (原始内容存档于2017-10-21). Untersuchungen zu dynamischen neuralen Netzen.

<a id="S0198"></a> Source: p.11 S0198

Original: Diploma thesis, Institut fu¨r Informatik, Lehrstuhl Prof.

中文: Lehrstuhl教授信息学院文凭论文。

<a id="S0199"></a> Source: p.11 S0199

Original: Brauer, Technische Universita¨t Mu¨nchen.

中文: Brauer, 技术大学 Mu'nchen.

<a id="S0200"></a> Source: p.11 S0200

Original: Hochreiter, S. and Schmidhuber, J. (1997).

中文: Hochreiter, S. and Schmidhuber, J. (1997年).

<a id="S0201"></a> Source: p.11 S0201

Original: Kalchbrenner, N. and Blunsom, P. (2013).

中文: 卡尔克布伦纳,N.和布伦索姆,P.(2013).

<a id="S0202"></a> Source: p.11 S0202

Original: Recurrent continuous translation models.

中文: 经常性连续翻译模式.

<a id="S0203"></a> Source: p.11 S0203

Original: In Proceedings of the ACL Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1700–1709.

中文: ACL关于自然语言处理经验方法的会议记录,第1700-1709页。

<a id="S0204"></a> Source: p.11 S0204

Original: Association for Computational Linguistics.

中文: 计算语言学协会.

<a id="S0205"></a> Source: p.11 S0205

Original: Cambridge University Press, New York, NY, USA.

中文: 美国纽约州纽约剑桥大学出版社.

<a id="S0206"></a> Source: p.11 S0206

Original: Koehn, P., Och, F. J., and Marcu, D. (2003).

中文: Koehn, P., Och, F. J.和Marcu, D.(2003年)。

<a id="S0207"></a> Source: p.11 S0207

Original: In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1, NAACL ’03, pages 48–54, Stroudsburg, PA, USA.

中文: 人类语言技术计算语言学协会北美分会2003年会议记录 -- -- 第1卷,NAACL ' 03, 第48-54页,美国PA Stroudsburg。

<a id="S0208"></a> Source: p.11 S0208

Original: Association for Computational Linguistics.

中文: 计算语言学协会.

<a id="S0209"></a> Source: p.11 S0209

Original: Pascanu, R., Mikolov, T., and Bengio, Y. (2013a).

中文: Pascanu, R., Mikolov, T.,和 Bengio, Y. (2013年a).

<a id="S0210"></a> Source: p.11 S0210

Original: On the difficulty of training recurrent neural networks.

中文: 关于经常性神经网络训练的难度.

<a id="S0211"></a> Source: p.11 S0211

Original: Pascanu, R., Mikolov, T., and Bengio, Y. (2013b).

中文: Pascanu, R., Mikolov, T.,和 Bengio, Y. (2013年b).

<a id="S0212"></a> Source: p.11 S0212

Original: On the difficulty of training recurrent neural networks.

中文: 关于经常性神经网络训练的难度.

<a id="S0213"></a> Source: p.11 S0213

Original: In Proceedings of the 30th International Conference on Machine Learning (ICML 2013).

中文: 第30届机器学习国际会议记录(ICML 2013).

<a id="S0214"></a> Source: p.11 S0214

Original: Pascanu, R., Gulcehre, C., Cho, K., and Bengio, Y. (2014).

中文: Pascanu, R., Gulcehre, C., Cho, K.,和 Bengio, Y. (2014).

<a id="S0215"></a> Source: p.11 S0215

Original: How to construct deep recurrent neural networks.

中文: 如何构建深层反复神经网络.

<a id="S0216"></a> Source: p.11 S0216

Original: In Proceedings of the Second International Conference on Learning Representations (ICLR 2014).

中文: 《第二届国际学习代表会议记录》(ICLR,2014年)。

<a id="S0217"></a> Source: p.11 S0217

Original: Pouget-Abadie, J., Bahdanau, D., van Merrie¨nboer, B., Cho, K., and Bengio, Y. (2014).

中文: Pouget-Abadie、J.、Bahdanau、D.、van Merrie'nboer、B.、Cho、K.和Bengio,Y.(2014)。

<a id="S0218"></a> Source: p.11 S0218

Original: Overcoming the curse of sentence length for neural machine translation using automatic segmentation.

中文: 克服用自动分解方式翻译神经机的句子长度的诅咒.

<a id="S0219"></a> Source: p.11 S0219

Original: In Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation. to appear.

中文: 第八次统计翻译的语法、语义和结构讲习班。 出现。

<a id="S0220"></a> Source: p.11 S0220

Original: Bidirectional recurrent neural networks.

中文: 双向循环神经网络.

<a id="S0221"></a> Source: p.11 S0221

Original: Signal Processing, IEEE Transactions on, 45(11), 2673–2681.

中文: 信号处理,IEEE交易于45(11),2673–2681.

<a id="S0222"></a> Source: p.11 S0222

Original: Continuous space translation models for phrase-based statistical machine translation.

中文: 基于短语的统计机器翻译的连续空间翻译模型.

<a id="S0223"></a> Source: p.11 S0223

Original: Boitet, editors, Proceedings of the 24th International Conference on Computational Linguistics (COLIN), pages 1071–1080.

中文: Boitet编辑,第24届计算语言学国际会议记录,第1071-1080.

<a id="S0224"></a> Source: p.11 S0224

Original: Schwenk, H., Dchelotte, D., and Gauvain, J.-L. (2006).

中文: Schwenk, H., Dchelotte, D., and Gauvain, J.-L.(2006年).

<a id="S0225"></a> Source: p.11 S0225

Original: Continuous space language models for statistical machine translation.

中文: 统计机器翻译的连续空间语言模型.

<a id="S0226"></a> Source: p.11 S0226

Original: In Proceedings of the COLING/ACL on Main conference poster sessions, pages 723–730.

中文: 《关于主要会议海报的会议的会议记录》,第723-730页。

<a id="S0227"></a> Source: p.11 S0227

Original: Association for Computational Linguistics.

中文: 计算语言学协会.

<a id="S0228"></a> Source: p.11 S0228

Original: Sutskever, I., Vinyals, O., and Le, Q. (2014).

中文: Sutskever, I., Vinyals, O., and Le, Q. (2014). (原始内容存档于2014年9月20日).

<a id="S0229"></a> Source: p.11 S0229

Original: Sequence to sequence learning with neural networks.

中文: 与神经网络同步学习的顺序.

<a id="S0230"></a> Source: p.11 S0230

Original: In Advances in Neural Information Processing Systems (NIPS 2014).

中文: 在神经信息处理系统的进步(NIPS 2014).

<a id="S0231"></a> Source: p.11 S0231

Original: ADADELTA: An adaptive learning rate method. arXiv:1212.5701 [cs.LG]. 11

中文: ADADELTA:适应性学习率方法。

<a id="S0232"></a> Source: p.12 S0232

Original: Published as a conference paper at ICLR 2015 A MODEL ARCHITECTURE A.1 ARCHITECTURAL CHOICES The proposed scheme in Section 3 is a general framework where one can freely define, for instance, the activation functions f of recurrent neural networks (RNN) and the alignment model a.

中文: 2015年以会议论文发表 第3节中的拟议办法是一个一般框架,人们可以自由地界定经常神经网络的激活功能f和对齐模式a。

<a id="S0233"></a> Source: p.12 S0233

Original: Here, we describe the choices we made for the experiments in this paper. A.1.1 RECURRENT NEURAL NETWORK For the activation function f of an RNN, we use the gated hidden unit recently proposed by Cho et al. (2014a).

中文: 在这里,我们描述了我们对本文中的实验所作的选择。 A.1.1 新的新网络 对于一个RNN的活化功能f,我们使用Cho等人最近提出的有门的隐藏单元(2014a).

<a id="S0234"></a> Source: p.12 S0234

Original: The gated hidden unit is an alternative to the conventional simple units such as an element-wise tanh.

中文: 入门隐蔽单元是常规简单单元的替代物,如元素取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取出取的取出取出取出取出取出取出取出取出取出取

<a id="S0235"></a> Source: p.12 S0235

Original: This gated unit is similar to a long short-term memory (LSTM) unit proposed earlier by Hochreiter and Schmidhuber (1997), sharing with it the ability to better model and learn long-term dependencies.

中文: 这个有门的单元类似于霍克赖特和施密德胡伯(Schmidhuber (1997))早些时候提出的一个长期短期内存(LSTM)单元,与它分享更好的模型和学习长期依赖的能力.

<a id="S0236"></a> Source: p.12 S0236

Original: This is made possible by having computation paths in the unfolded RNN for which the product of derivatives is close to 1.

中文: 通过在已展开的RNN中拥有计算路径,使衍生物的产物接近于1的计算路径成为可能.

<a id="S0237"></a> Source: p.12 S0237

Original: These paths allow gradients to flow backward easily without suffering too much from the vanishing effect (Hochreiter, 1991; Bengio et al., 1994; Pascanu et al., 2013a).

中文: 这些路径使梯度容易向后流动,而不会因消失效应而承受太多的痛苦(Hochreiter,1991年;Bengio等,1994年;Pascanu等,2013年a)。

<a id="S0238"></a> Source: p.12 S0238

Original: It is therefore possible to use LSTM units instead of the gated hidden unit described here, as was done in a similar context by Sutskever et al. (2014).

中文: 因此,可以使用LSTM单元,而不是本文所描述的有门隐藏单元,正如Sutskever等人(2014年)在类似背景下所做的那样。

<a id="S0239"></a> Source: p.12 S0239

Original: The new state s of the RNN employing n gated hidden units8 is computed by i s = f (s , y , c ) = (1 − z ) ◦ s + z ◦ s˜ , i i−1 i−1 i i i−1 i i where ◦ is an element-wise multiplication, and z is the output of the update gates (see below).

中文: 使用n门隐藏单元8的RNN新状态由i s = f (s, y, c) = (1 - z) = (1 - z) = + z = (1 - 1 i - 1 i - 1 i) i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i s i i i i i i i i i i i i i i s i i i i i i i i i i i i i i i i s i i i i i i i i i i i i i i i i i i i i i i i i i i i i i

<a id="S0240"></a> Source: p.12 S0240

Original: The i proposed updated state s˜ is computed by i s˜ = tanh (W e(y ) + U [r ◦ s ] + Cc ) , i i−1 i i−1 i where e(y ) ∈ Rm is an m-dimensional embedding of a word y , and r is the output of the i−1 i−1 i reset gates (see below).

中文: i建议的更新状态 s 由 i s = tanh (W e (y) + U [r ◦ s] + Cc )计算, i− 1 i− 1 i− 1 i i i i 其中 e (y) → Rm 是一个单词 y 的 m-维嵌入,r 为 i− 1 i− 1 i 重置门的输出(见下文).

<a id="S0241"></a> Source: p.12 S0241

Original: When y is represented as a 1-of-K vector, e(y ) is simply a column of an i i embedding matrix E ∈ Rm×K.

中文: 当y以 1-K 向量表示时,e(y) 仅仅是 i 嵌入矩阵 E → Rm×K 的一列.

<a id="S0242"></a> Source: p.12 S0242

Original: Whenever possible, we omit bias terms to make the equations less cluttered.

中文: 只要有可能,我们都会省略偏差的术语,使等式变得不那么杂乱.

<a id="S0243"></a> Source: p.12 S0243

Original: The update gates z allow each hidden unit to maintain its previous activation, and the reset gates r i i control how much and what information from the previous state should be reset.

中文: 更新门 z 允许每个隐藏单元保持其先前的激活,而重设门 r i 控制了从先前状态中应重置多少和哪些信息.

<a id="S0244"></a> Source: p.12 S0244

Original: We compute them by z = σ (W e(y ) + U s + C c ) , i z i−1 z i−1 z i r = σ (W e(y ) + U s + C c ) , i r i−1 r i−1 r i where σ (·) is a logistic sigmoid function.

中文: 我们以 z = (W e(y) + U s + C c) 计算它们,i z i− 1 → 1 → 1 → r = σ (W e(y) + U s + C C c),i r− 1 r− 1 r i 在 σ 为后勤 sigmoid 函数的地方.

<a id="S0245"></a> Source: p.12 S0245

Original: At each step of the decoder, we compute the output probability (Eq. (4)) as a multi-layered function (Pascanu et al., 2014).

中文: 在解码器的每个步骤上,我们计算出输出概率(Eq. (4))为多层函数(Pascanu等,2014年).

<a id="S0246"></a> Source: p.12 S0246

Original: We use a single hidden layer of maxout units (Goodfellow et al., 2013) and normalize the output probabilities (one for each word) with a softmax function (see Eq. (6)). A.1.2 ALIGNMENT MODEL The alignment model should be designed considering that the model needs to be evaluated T × T x y times for each sentence pair of lengths T and T .

中文: 我们使用一个单层隐藏出最大单位(Goodfellow等,2013年),并将输出概率(每个单词一个)与软最大函数正常化(见Eq.(6)). A.1.2 租赁模式 配对模型的设计应考虑到,需要对每对句子长度T和T的TxTxy乘以评价该模型。

<a id="S0247"></a> Source: p.12 S0247

Original: In order to reduce computation, we use a singlex y layer multilayer perceptron such that a(s , h ) = v(cid:62) tanh (W s + U h ) , i−1 j a a i−1 a j where W ∈ Rn×n, U ∈ Rn×2n and v ∈ Rn are the weight matrices.

中文: 为了减少计算,我们使用一个单x y 层多层受体,使 a(s, h) = v(cid:62) tanh(W s + U h), i− 1 j a i− 1 a j 在 W → Rnxn, U → Rnx2n 和 v → Rn 为权重矩阵.

<a id="S0248"></a> Source: p.12 S0248

Original: Since U h does not a a a a j depend on i, we can pre-compute it in advance to minimize the computational cost. 8 Here, we show the formula of the decoder.

中文: 由于Uh不是一个j依赖于i,我们可以提前预先计算以将计算成本降到最低. 8 在这里,我们显示解码器的公式.

<a id="S0249"></a> Source: p.12 S0249

Original: The same formula can be used in the encoder by simply ignoring the context vector c and the related terms. i 12

中文: 同样的公式可以在编码器中被简单地忽略上下文矢量c和相关术语. i 12个

<a id="S0250"></a> Source: p.13 S0250

Original: Published as a conference paper at ICLR 2015 A.2 DETAILED DESCRIPTION OF THE MODEL A.2.1 ENCODER In this section, we describe in detail the architecture of the proposed model (RNNsearch) used in the experiments (see Sec. 4–5).

中文: 作为会议论文发表于ICLR 2015 A.2 DETAILD DESCION MODEL A.2.1 ENCODER 在本节中,我们详细描述了实验中使用的拟议模型(RNNsearch)的结构(见Sec. 4-5)。

<a id="S0251"></a> Source: p.13 S0251

Original: From here on, we omit all bias terms in order to increase readability.

中文: 从这里开始,我们省略了所有的偏差术语,以提高可读性.

<a id="S0252"></a> Source: p.13 S0252

Original: The model takes a source sentence of 1-of-K coded word vectors as input x = (x , . . . , x ), x ∈ RKx 1 Tx i and outputs a translated sentence of 1-of-K coded word vectors y = (y , . . . , y ), y ∈ RKy , 1 Ty i where K and K are the vocabulary sizes of source and target languages, respectively. T and T x y x y respectively denote the lengths of source and target sentences.

中文: 该模型取出1-of-K编码字向量的源句为输入x=(x,.x),x-Q-RKx 1 Tx i并输出1-of-K编码字向量y=(y,.,.y),y-Q-RKy,1 Ty i分别是源语言和目标语言的词汇大小. T和T xy x y分别表示源句和目标句的长度.

<a id="S0253"></a> Source: p.13 S0253

Original: First, the forward states of the bidirectional recurrent neural network (BiRNN) are computed: (cid:40) →− →− →− →− →− (1 − z ) ◦ h + z ◦ h , if i > 0 h = i i−1 i i i 0 , if i = 0 where →− (cid:16)−→ →− (cid:104)→− →− (cid:105)(cid:17) h = tanh W Ex + U r ◦ h i i i i−1 →− (cid:16)−→ →− →− (cid:17) z =σ W Ex + U h i z i z i−1 →− (cid:16)−→ →− →− (cid:17) r =σ W Ex + U h . i r i r i−1 −→ −→ −→ →− →− →− E ∈ Rm×Kx is the word embedding matrix. W , W z , W r ∈ Rn×m, U , U z , U r ∈ Rn×n are weight matrices. m and n are the word embedding dimensionality and the number of hidden units, respectively. σ(·) is as usual a logistic sigmoid function. ←− ←− The backward states ( h , · · · , h ) are computed similarly.

中文: 首先,双向常年神经网络(BIRN)的前期状态是计算: (cid:40) →−(Cid:40) →−(Cid:40) →− (1-z) ◦(h) + z ◦ (h) 如果i > 0 h (i-1) i-i (i) i (i) → 0 如果i → (cid:16) −→ (cid:104) → →− (cid:105)(cid:17) h = tanh W Ex + U r i i- 1→− (cid:16) −→ (cid:17) z (z) + U hi i i- 1→− (cid:16) → →− (cid:17) r → =σ (i) + U h i- i- 1) → (cid: ) + + + + + → (cid: ) + + + + + + + + + + W, W z, W r ∈ Rn×m, U, U z, U r ∈ Rn×n 是重量矩阵. m和n分别是嵌入维度和隐藏单位的数目。\ (\) 和通常的逻辑 sigmoid 函数一样。 ←− 落后国家(h, )的计算方式类似。

<a id="S0254"></a> Source: p.13 S0254

Original: We share the word embedding matrix 1 Tx E between the forward and backward RNNs, unlike the weight matrices.

中文: 我们分享前向和后向的RNNs之间嵌入矩阵1Tx E的单词,与重量矩阵不同.

<a id="S0255"></a> Source: p.13 S0255

Original: We concatenate the forward and backward states to to obtain the annotations (h , h , · · · , h ), 1 2 Tx where (cid:34) →− (cid:35) h h i = ←− i (7) h i A.2.2 DECODER The hidden state s of the decoder given the annotations from the encoder is computed by i s =(1 − z ) ◦ s + z ◦ s˜ , i i i−1 i i where s˜ = tanh (W Ey + U [r ◦ s ] + Cc ) i i−1 i i−1 i z =σ (W Ey + U s + C c ) i z i−1 z i−1 z i r =σ (W Ey + U s + C c ) i r i−1 r i−1 r i E is the word embedding matrix for the target language. W, W , W ∈ Rn×m, U, U , U ∈ Rn×n, z r z r and C, C , C ∈ Rn×2n are weights.

中文: 我们召集前方和后方各州获得说明(h, h, ), 1 2 Tx 在哪里(cid:34) →− (cid:35) h i = ←− i (7) h A.2.2 DECODER 从编码器中给出的解码器的隐藏状态 s由i s = (1 − z ) ◦ s + z s i i i i i i i i i i i = tanh (W Ey + U [r ◦ s] + Cc ) i- 1 i- 1 i- i- i- i- i i- i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i 克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克克.

<a id="S0256"></a> Source: p.13 S0256

Original: Again, m and n are the word embedding dimensionality z r and the number of hidden units, respectively.

中文: 同样,m和n分别是嵌入维度zr的单词和隐藏单位的数量.

<a id="S0257"></a> Source: p.13 S0257

Original: The initial hidden state s is computed by s = 0 0 (cid:16) ←− (cid:17) tanh W h , where W ∈ Rn×n. s 1 s The context vector c are recomputed at each step by the alignment model: i (cid:88) Tx c = α h , i ij j j=1 13

中文: 初始隐藏状态 s由 s = 0 (cid:16) → (cid:17) tanh Wh 计算出,其中 W → Rn×n. s 1 s 上下文矢量 c 由对齐模型逐步重算 i (cid:88) Tx c = α h, i j j = 1 13

<a id="S0258"></a> Source: p.14 S0258

Original: Published as a conference paper at ICLR 2015 Model Updates (×105) Epochs Hours GPU Train NLL Dev.

中文: 作为会议论文发表于ICLR 2015 Model Updates (x105) Epochs Hours GPU Train NLL Dev.

<a id="S0259"></a> Source: p.14 S0259

Original: NLL RNNenc-30 8.46 6.4 109 TITAN BLACK 28.1 53.0 RNNenc-50 6.00 4.5 108 Quadro K-6000 44.0 43.6 RNNsearch-30 4.71 3.6 113 TITAN BLACK 26.7 47.2 RNNsearch-50 2.88 2.2 111 Quadro K-6000 40.7 38.1 RNNsearch-50(cid:63) 6.67 5.0 252 Quadro K-6000 36.7 35.2 Table 2: Learning statistics and relevant information.

中文: NLL RNenc-30 8.46 6.4 109 TITAN BLACK 28.1 53.0 RNNenc-50 6.00 4.5 108 Quadro K-600 44.0 43.6 RNNSearch-30 4.71 3.6 TITAN BLACK 26.7 47.2 RNSearch-50 2.8 111 Quadro K-600 40.7 38.1 RNNSearch-50(Cid:63) 6.67 5.0 252 Quadro K-600 36.7 35.2 表2:学习统计和相关信息。

<a id="S0260"></a> Source: p.14 S0260

Original: Each update corresponds to updating the parameters once using a single minibatch.

中文: 每次更新都对应一次使用单个小批量来更新参数.

<a id="S0261"></a> Source: p.14 S0261

Original: One epoch is one pass through the training set.

中文: 一个时代就是通过训练

<a id="S0262"></a> Source: p.14 S0262

Original: NLL is the average conditional log-probabilities of the sentences in either the training set or the development set.

中文: NLL是训练集或开发集中句子的平均条件对数概率.

<a id="S0263"></a> Source: p.14 S0263

Original: Note that the lengths of the sentences differ. where exp (e ) α = ij ij (cid:80)Tx exp (e ) k=1 ik e =v(cid:62) tanh (W s + U h ) , ij a a i−1 a j and h is the j-th annotation in the source sentence (see Eq. (7)). v ∈ Rn(cid:48), W ∈ Rn(cid:48)×n and j a a U ∈ Rn(cid:48)×2n are weight matrices.

中文: 注意句子长度不同. 其中exp (e) α = ij ij (cid:80)Tx exp (e) k = 1 ik e =v(cid:62) tanh (W s + U h),ij a a i− 1 a j和h是源句中j-th的注释(见Eq. (7)). v → Rn(cid:48), W → Rn(cid:48)×n和 j a U → Rn(cid:48)×2n 为权重矩阵.

<a id="S0264"></a> Source: p.14 S0264

Original: Note that the model becomes RNN Encoder–Decoder (Cho a →− et al., 2014a), if we fix c to h . i Tx With the decoder state s , the context c and the last generated word y , we define the probability i−1 i i−1 of a target word y as i p(y |s , y , c ) ∝ exp (cid:0) y(cid:62)W t (cid:1) , i i i−1 i i o i where t = (cid:2) max (cid:8) t˜ , t˜ (cid:9)(cid:3)(cid:62) i i,2j−1 i,2j j=1,...,l and t˜ is the k-th element of a vector t˜ which is computed by i,k i t˜ =U s + V Ey + C c . i o i−1 o i−1 o i W o ∈ RKy×l, U o ∈ R2l×n, V o ∈ R2l×m and C o ∈ R2l×2n are weight matrices.

中文: 注意,如果我们将 c 改为 h. i Tx,该模型将成为 RNN Encoder-Decoder(Cho a 等.,2014年a) 以解码器状态 s,上下文 c和最后一个生成的单词 y,我们定义目标单词 y 的概率 i− 1 i− 1 i− 1 作为 i p (y s, y, c) ∝ exp (cid:0) y (cid:62) W t (cid:1), i i− 1 i i i i i o i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i i - i i i - i i i - i i - - i i i → → rky xl i (cid: i i i i

<a id="S0265"></a> Source: p.14 S0265

Original: This can be understood as having a deep output (Pascanu et al., 2014) with a single maxout hidden layer (Goodfellow et al., 2013). A.2.3 MODEL SIZE For all the models used in this paper, the size of a hidden layer n is 1000, the word embedding dimensionality m is 620 and the size of the maxout hidden layer in the deep output l is 500.

中文: 这可被理解为具有深层输出(Pascanu等,2014年),具有单一最大输出隐藏层(Goodfellow等,2013年)。 A.2.3 模德·西泽 对于本文使用的所有模型,隐藏层n的大小为1000个,嵌入维度m的单词为620个,深输出l中最大出隐藏层的大小为500个.

<a id="S0266"></a> Source: p.14 S0266

Original: The number of hidden units in the alignment model n(cid:48) is 1000. B TRAINING PROCEDURE B.1 PARAMETER INITIALIZATION ←− ←− ←− →− →− →− We initialized the recurrent weight matrices U, U , U , U , U , U , U , U and U as random orz r z r z r thogonal matrices.

中文: 对齐模式n(cid:48)中隐藏的单位数为1000. B. 训练程序 B.1 我们初始化了经常重量矩阵 U, U, U, U, U, U, U, U, U 和 U ,作为随机的orz r r r r r r rogonal矩阵.

<a id="S0267"></a> Source: p.14 S0267

Original: For W and U , we initialized them by sampling each element from the Gaussian a a distribution of mean 0 and variance 0.0012.

中文: 对W和U来说,我们从高斯山脉的每个元素取样,初始化了它们。

<a id="S0268"></a> Source: p.14 S0268

Original: All the elements of V and all the bias vectors were inia tialized to zero.

中文: V的所有元素和所有偏差向量都被Inia tial化为零.

<a id="S0269"></a> Source: p.14 S0269

Original: Any other weight matrix was initialized by sampling from the Gaussian distribution of mean 0 and variance 0.012. B.2 TRAINING We used the stochastic gradient descent (SGD) algorithm.

中文: 任何其他重量矩阵都通过从高斯分布的平均值为0和差异为0.012的取样而初始化。 B.2 培训 我们用的是斜向梯度下降算法。

<a id="S0270"></a> Source: p.14 S0270

Original: Adadelta (Zeiler, 2012) was used to automatically adapt the learning rate of each parameter ((cid:15) = 10−6 and ρ = 0.95).

中文: Adadelta (Zeiler, 2012) 用于自动调整每个参数的学习率((cid:15)=10-6和 ^ 0.95).

<a id="S0271"></a> Source: p.15 S0271

Original: Published as a conference paper at ICLR 2015 normalized the L -norm of the gradient of the cost function each time to be at most a predefined 2 threshold of 1, when the norm was larger than the threshold (Pascanu et al., 2013b).

中文: 作为会议文件发表于ICLR 2015年,将每次成本函数梯度的L-规范正常化,最多达到预先界定的2个阈值为1,当规范大于阈值(Pascanu等人,2013年b)。

<a id="S0272"></a> Source: p.15 S0272

Original: Each SGD update direction was computed with a minibatch of 80 sentences.

中文: 每个SGD更新方向都是用80个句子的小批量来计算.

<a id="S0273"></a> Source: p.15 S0273

Original: At each update our implementation requires time proportional to the length of the longest sentence in a minibatch.

中文: 每次更新我们的执行需要时间 与小批量中最长刑期的长度成正比

<a id="S0274"></a> Source: p.15 S0274

Original: Hence, to minimize the waste of computation, before every 20-th update, we retrieved 1600 sentence pairs, sorted them according to the lengths and split them into 20 minibatches.

中文: 因此,为了尽量减少计算上的浪费,在每20次更新之前,我们检索出1600个句子对,按照长度进行分类,并分成20个小管.

<a id="S0275"></a> Source: p.15 S0275

Original: The training data was shuffled once before training and was traversed sequentially in this manner.

中文: 培训数据在培训前被洗刷过一次,并依次以这种方式进行。

<a id="S0276"></a> Source: p.15 S0276

Original: In Tables 2 we present the statistics related to training all the models used in the experiments. C TRANSLATIONS OF LONG SENTENCES Source An admitting privilege is the right of a doctor to admit a patient to a hospital or a medical centre to carry out a diagnosis or a procedure, based on his status as a health care worker at a hospital.

中文: 在表2中,我们提出了与培训实验中使用的所有模型有关的统计。 远距离传感器的转换 承认的特权是,医生有权根据病人在医院的保健工作者的身份,将病人送入医院或医疗中心进行诊断或程序。

<a id="S0277"></a> Source: p.15 S0277

Original: Reference Le privile`ge d’admission est le droit d’un me´decin, en vertu de son statut de membre soignant d’un hoˆpital, d’admettre un patient dans un hoˆpital ou un centre me´dical afin d’y de´livrer un diagnostic ou un traitement.

中文: 参考文献 以我为中心,

<a id="S0278"></a> Source: p.15 S0278

Original: RNNenc-50 Un privilege d’admission est le droit d’un me´decin de reconnaˆıtre un patient a l’hoˆpital ou un centre me´dical d’un diagnostic ou de prendre un diagnostic en fonction de son e´tat de sante´.

中文: RNNenc-50 (韩语) 我决定要耐心地接受医院的诊断,

<a id="S0279"></a> Source: p.15 S0279

Original: RNNsearch-50 Un privilege d’admission est le droit d’un me´decin d’admettre un patient a un hoˆpital ou un centre me´dical pour effectuer un diagnostic ou une proce´dure, selon son statut de travailleur des soins de sante´ a` l’hoˆpital.

中文: RNNSearch - 50 (韩语) 以我为中心,

<a id="S0280"></a> Source: p.15 S0280

Original: Google Un privile`ge admettre est le droit d’un me´decin d’admettre un patient dans un hoˆpital ou un Translate centre me´dical pour effectuer un diagnostic ou une proce´dure, fonde´e sur sa situation en tant que travailleur de soins de sante´ dans un hoˆpital.

中文: 谷歌 以我为中心,

<a id="S0281"></a> Source: p.15 S0281

Original: Source This kind of experience is part of Disney’s efforts to ”extend the lifetime of its series and build new relationships with audiences via digital platforms that are becoming ever more important,” he added.

中文: 来源 他补充说,这种经验是迪士尼努力“延长其系列的寿命并通过日益重要的数字平台与观众建立新的关系”。

<a id="S0282"></a> Source: p.15 S0282

Original: Reference Ce type d’expe´rience entre dans le cadre des efforts de Disney pour ”e´tendre la dure´e de vie de ses se´ries et construire de nouvelles relations avec son public graˆce a` des plateformes nume´riques qui sont de plus en plus importantes”, a-t-il ajoute´.

中文: 参考文献 迪士尼的干部们的努力 ” 。 。 。

<a id="S0283"></a> Source: p.15 S0283

Original: RNNenc-50 Ce type d’expe´rience fait partie des initiatives du Disney pour ”prolonger la dure´e de vie de ses nouvelles et de de´velopper des liens avec les lecteurs nume´riques qui deviennent plus complexes.

中文: RNNenc-50 (韩语) 迪士尼倡议的参与者,

<a id="S0284"></a> Source: p.15 S0284

Original: RNNsearch-50 Ce genre d’expe´rience fait partie des efforts de Disney pour ”prolonger la dure´e de vie de ses se´ries et cre´er de nouvelles relations avec des publics via des plateformes nume´riques de plus en plus importantes”, a-t-il ajoute´.

中文: RNNSearch - 50 (韩语) 迪士尼的努力“通过板块形式促进公众关系”“一提,

<a id="S0285"></a> Source: p.15 S0285

Original: Google Ce genre d’expe´rience fait partie des efforts de Disney a` “e´tendre la dure´e de vie de sa se´rie et Translate construire de nouvelles relations avec le public par le biais des plates-formes nume´riques qui deviennent de plus en plus important”, at-il ajoute´.

中文: 谷歌 迪士尼努力的“新时代和新时代关系解释”,载于“E'endre la dure'e vie de sa se'rie and translate construction de neual relations avec le public le biais nume'riques qui devienennt de plus en plus important'。

<a id="S0286"></a> Source: p.15 S0286

Original: Source In a press conference on Thursday, Mr Blair stated that there was nothing in this video that might constitute a ”reasonable motive” that could lead to criminal charges being brought against the mayor.

中文: 来源 Blair先生在星期四的新闻发布会上说,这段录像中没有任何内容构成可能导致对市长提出刑事指控的“合理动机”。

<a id="S0287"></a> Source: p.15 S0287

Original: Reference En confe´rence de presse, jeudi, M.

中文: 参考文献 (原始内容存档于2018-05-25). Enfe'rence de presse, jeudi, M.

<a id="S0288"></a> Source: p.15 S0288

Original: Blair a affirme´ qu’il n’y avait rien dans cette vide´o qui puisse constituer des ”motifs raisonnables” pouvant mener au de´poˆt d’une accusation criminelle contre le maire.

中文: Blair表示,

<a id="S0289"></a> Source: p.15 S0289

Original: RNNenc-50 Lors de la confe´rence de presse de jeudi, M.

中文: RNNenc-50 (韩语) (原始内容存档于2018-10-21) (英语). Lors de la confe'rence de presse de jeudi, M.

<a id="S0290"></a> Source: p.15 S0290

Original: Blair a dit qu’il n’y avait rien dans cette vide´o qui pourrait constituer une ”motivation raisonnable” pouvant entraˆıner des accusations criminelles porte´es contre le maire.

中文: Blair是一位“动机合理”的领袖,

<a id="S0291"></a> Source: p.15 S0291

Original: RNNsearch-50 Lors d’une confe´rence de presse jeudi, M.

中文: (原始内容存档于2019-03-25) (英语). RNNsearch-50 Lors d'une confe'rence de presse jeudi, M.

<a id="S0292"></a> Source: p.15 S0292

Original: Blair a de´clare´ qu’il n’y avait rien dans cette vide´o qui pourrait constituer un ”motif raisonnable” qui pourrait conduire a` des accusations criminelles contre le maire.

中文: Blair a de'claire' qu 'il n'y avait rien dans vide'o qui qui pourrait constituer un " motif 无理 " qui pourrait condure a " des criminelles against le maire " 。 黑手党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党党

<a id="S0293"></a> Source: p.15 S0293

Original: Google Lors d’une confe´rence de presse jeudi, M.

中文: Google Lors d'une confe'rence de presse jeudi, M. 互联网档案馆的存檔,存档日期2013-12-22., p.

<a id="S0294"></a> Source: p.15 S0294

Original: Blair a de´clare´ qu’il n’y avait rien dans cette vido Translate qui pourrait constituer un ”motif raisonnable” qui pourrait mener a` des accusations criminelles portes contre le maire.

中文: Blair a'clare' qu ' il n'y avait rien dans cette vido Translate qui pourrait constituer un " motif 无理" qui pourrait mener a " des criminelles portes against le maire. " ("犯罪港站"),"犯罪港站"。

<a id="S0295"></a> Source: p.15 S0295

Original: Table 3: The translations generated by RNNenc-50 and RNNsearch-50 from long source sentences (30 words or more) selected from the test set.

中文: 表3:RNNenc-50和RNNsearch-50从从测试集中选取的长源句(30个字或以上)所生成的翻译.

<a id="S0296"></a> Source: p.15 S0296

Original: For each source sentence, we also show the goldstandard translation.

中文: 对于每个源句,我们也展示了"金标准"的翻译.

<a id="S0297"></a> Source: p.15 S0297

Original: The translations by Google Translate were made on 27 August 2014. 15

中文: Google Translate的翻译于2014年8月27日完成. 15个