报告题目：深度神经网络卷积核优化探究
时 间：2017年5月4日上午10:00-12:00
地 点：计算所446会议室
报告摘要：
In this work, we focus on investigation of the network solution properties that can potentially lead to good performance. Our research is inspired by theoretical and empirical results that use orthogonal matrices to initialize networks, but we are interested in investigating how orthogonal weight matrices perform when network training converges. To this end, we propose to constrain the solutions of weight matrices in the orthogonal feasible set during the whole process of network training, and achieve this by a simple yet effective method called Singular Value Bounding (SVB). In SVB, all singular values of each weight matrix are simply bounded in a narrow band around the value of 1. Based on the same motivation, we also propose Bounded Batch Normalization (BBN), which improves Batch Normalization by removing its potential risk of ill-conditioned layer transform. We present both theoretical and empirical results to justify our proposed methods. Experiments on benchmark image classification datasets show the efficacy of our proposed SVB and BBN. In particular, we achieve the state-of-the-art results of 3.06% error rate on CIFAR10 and 16.90% on CIFAR100, using off-the-shelf network architectures (WideResNets). Our preliminary results on ImageNet also show the promise in large-scale learning.
报告人简介：
贾奎，博士，华南理工大学电子与信息学院教授，博士生导师，主动感知与结构智能团队负责人，入选国家重要人才计划。2001年于西北工业大学获得学士学位，2004年于新加坡国立大学获得硕士学位，2007年于伦敦大学玛丽女王学院获得计算机科学博士学位。博士毕业后，曾先后于中科院深圳先进技术研究院、香港中文大学、伊利诺伊大学香槟分校新加坡高等研究院、及澳门大学从事教学和科研工作。贾奎博士的主要研究方向是计算机视觉、机器学习、图像处理、和模式识别等。在包括TPAMI，IJCV,TSP,TIP,ICCV,CVPR等在内的计算机视觉和模式识别顶级期刊和会议发表论文50余篇.
附件：