This post reviews the latest innovations of TCN based solutions. We first present a case study of motion detection and briefly review the TCN architecture and its advantages over conventional approaches such as Convolutional Neural Networks (CNN) and Recurrent Neural Network (RNN). Then, we introduce several novels using TCN, including improving traffic prediction, sound event localization & detection, and probabilistic forecasting.

这篇文章回顾了基于TCN的解决方案的最新创新。 我们首先介绍运动检测的案例研究，并简要回顾TCN架构及其相对于传统方法(如卷积神经网络(CNN)和递归神经网络(RNN))的优势。 然后，我们介绍了使用TCN的几本小说，包括改善交通预测，声音事件定位和检测以及概率预测。

A brief review of TCN

TCN简要回顾

The seminal work of Lea et al. (2016) first proposed a Temporal Convolutional Networks (TCNs) for video-based action segmentation. The two steps of this conventional process include: firstly, computing of low-level features using (usually) CNN that encode spatial-temporal information and secondly, input these low-level features into a classifier that captures high-level temporal information using (usually) RNN. The main disadvantage of such an approach is that it requires two separate models. TCN provides a unified approach to capture all two levels of information hierarchically.

Lea等人的开创性工作。 (2016)首先提出了基于视频的动作分割的时间卷积网络(TCN)。 此常规过程的两个步骤包括：首先，使用(通常)对时空信息进行编码的CNN计算低级特征，其次，将这些低级特征输入到使用(通常是)捕获高级时空信息的分类器中)RNN。 这种方法的主要缺点是需要两个单独的模型。 TCN提供了一种统一的方法来分层捕获所有两个级别的信息。

The encoder-decoder framework is presented in Fig.1, where further information regarding the architecture can be found in the first two references (at the end of the post). The most critical issues are provided as follows: TCN can take a series of any length and output it as the same length. A causal convolutional is used where a 1D fully convolutional network architecture is used. A key characteristic is that the output at time t is only convolved with the elements that occurred before t.

编码器-解码器框架如图1所示，其中有关体系结构的更多信息可以在前两个参考文献中找到(在文章末尾)。 提供了最关键的问题，如下所示：TCN可以采用一系列任意长度并将其输出为相同长度。 在使用一维完全卷积网络体系结构的情况下，使用因果卷积。 一个关键特征是，时间t的输出仅与t之前发生的元素卷积。

The buzz around TCN arrives even to Nature journal, with the recent publication of the work by Yan et al. (2020) on TCN for weather prediction tasks. In their work, a comparative experiment was conducted with TCN and LSTM. One of their results was that, among other approaches, the TCN performs well in prediction tasks with time-series data.

随着Yan等人最近发表的研究成果，围绕TCN的话题甚至传到了《自然》杂志上。 (2020)在TCN上进行天气预报任务。 在他们的工作中，使用TCN和LSTM进行了对比实验。 他们的结果之一是，除其他方法外，TCN在使用时序数据的预测任务中表现出色。

The next sections provide the implementation and extension of this classical TCN.

下一节提供了此经典TCN的实现和扩展。

Improving traffic prediction

改善流量预测

Ridesharing and online navigation services can improve traffic prediction and change the way of life on the road. Fewer traffic jams, less pollution, safe and fast driving are just a few examples of essential issues that can be achieved by better traffic predictions. As this is a real-time data-driven problem, it is necessary to utilize the accumulated data of upcoming traffic. For this reason, Dai et al. (2020) recently presented a Hybrid Spatio-Temporal Graph Convolutional Network (H-STGCN). The general idea is to take the advantages of the piecewise-liner-flow-density relationship and convert the upcoming traffic volume in its equivalent in travel time. One of the most interesting approaches they used in this work is the graph convolution to capture the spatial dependency. The compound adjacency matrix captures the innate characteristics of traffic approximation (for more information, please see Li, 2017). In the following architecture, four modules are presented to describe the entire prediction process.

拼车和在线导航服务可以改善交通预测并改变道路上的生活方式。 更少的交通拥堵，更少的污染，安全和快速的驾驶只是通过更好的交通预测可以实现的基本问题的几个例子。 由于这是实时数据驱动的问题，因此有必要利用即将到来的流量的累积数据。 由于这个原因，Dai等。 (2020)最近提出了一种混合时空图卷积网络(H-STGCN)。 总体思路是利用分段衬里流量密度关系的优势，并将即将来临的交通量转换为等效的行进时间。 他们在这项工作中使用的最有趣的方法之一是图卷积以捕获空间依赖性。 复合邻接矩阵捕获流量近似的固有特征(更多信息，请参见Li，2017)。 在以下架构中，提出了四个模块来描述整个预测过程。

Sound event localization & detection

声音事件定位和检测

The field of sound event localization and detection (SELD) continues to grow. Understanding the environment plays a critical role in autonomous navigation. Guirguis et al. (2020) recently proposed a novel architecture for sound events SELD-TCN. They claim that their framework outperforms the state-of-the-art in the field, with faster training time. In their SELDnet (architecture below), a multichannel audio recording, sampled at 44.1 kHz, extracts, by applying a short-time Fourier transformation, the phase and magnitude of the spectrum, and stacks it as separate input features. Then, convolutional blocks and recurrent blocks (bi-directional GRUs) are connected, followed by a fully-connected block. The output of the SELDnet is the SOUND Event Detection (SED) and Direction Of Arrival (DOA).

声音事件定位和检测(SELD)的领域继续增长。 了解环境在自主导航中起着至关重要的作用。 Guirguis等。 (2020)最近提出了一种声音事件SELD-TCN的新颖架构。 他们声称，他们的框架在现场培训方面比当前最先进的技术领先。 在他们的SELDnet(以下结构)中，以44.1 kHz采样的多通道音频记录通过应用短时傅立叶变换提取频谱的相位和幅度，并将其堆叠为单独的输入特征。 然后，连接卷积块和循环块(双向GRU)，然后连接完全连接的块。 SELDnet的输出是声音事件检测(SED)和到达方向(DOA)。

In order to outperform it, they present the SELD-TCN:

为了超越它，他们提出了SELD-TCN：

As the dilated convolutions enable the net to process a variety of inputs, a more in-depth network may be required (which will be affected by unstable gradients during backpropagation). They overcome this challenge by adapting the WaveNet (Dario et al., 2017) architecture. They showed that the recurrent layers are not required for SELD tasks, and successfully detected the start and the end times of active sound events.

由于扩张的卷积使网络能够处理各种输入，因此可能需要更深入的网络(在反向传播期间，网络会受到不稳定梯度的影响)。 他们通过适应WaveNet(Dario et al。，2017)架构克服了这一挑战。 他们表明SELD任务不需要循环层，并成功检测到活动声音事件的开始和结束时间。

Probabilistic forecasting

概率预测

A novel framework designed by Chen et al. (2020) can be applied to estimate probability density. Time series prediction improves many business decision-making scenarios (for example, resources management). Probabilistic forecasting can extract information from historical data and minimize the uncertainty of future events. When the prediction task is to predict millions of related data series (as in the retail business), it requires prohibitive labor and computing resources for parameter estimation. In order to solve these difficulties, they proposed a CNN-based density estimation and prediction framework. Their framework can learn the latent correlation among series. The novelty in their work is the deep TCN they proposed, as presented in their architecture:

由Chen等人设计的新颖框架。 (2020)可用于估计概率密度。 时间序列预测改善了许多业务决策方案(例如，资源管理)。 概率预测可以从历史数据中提取信息，并最大限度地减少未来事件的不确定性。 当预测任务要预测数百万个相关数据序列时(如在零售业务中)，它需要大量的劳动力和计算资源来进行参数估计。 为了解决这些困难，他们提出了一种基于CNN的密度估计和预测框架。 他们的框架可以学习系列之间的潜在关联。 他们的工作中的新颖之处在于他们提出的深层TCN，如其体系结构所示：

The encoder-decoder modules solution might help in the design of practical large-scale applications.

编码器-解码器模块解决方案可能有助于实际的大规模应用设计。

摘要 (Summary)

In this post, we presented recent works that involve the temporal convolutional network and outperform classical CNN, and RNN approaches for time series tasks. For further information, please feel free to email me.

在这篇文章中，我们介绍了涉及时间卷积网络并优于经典CNN的最新作品，以及用于时间序列任务的RNN方法。 有关更多信息，请随时给我发送电子邮件。

— — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — —

Visit my personal website: www.Barakor.com

访问我的个人网站： www.Barakor.com

Linkedin https://www.linkedin.com/in/barakor/

Linkedin https://www.linkedin.com/in/barakor/

Twitter: BarakOr2

推特：BarakOr2

— — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — —