Deep neural evolution deep learning with evolutionary computation /
This book delivers the state of the art in deep learning (DL) methods hybridized with evolutionary computation (EC). Over the last decade, DL has dramatically reformed many domains: computer vision, speech recognition, healthcare, and automatic game playing, to mention only a few. All DL models, usi...
| Other Authors: | Iba, Hitoshi,, Noman, Nasimul,, SpringerLink (Online service) |
|---|---|
| Format: | eBook |
| Language: | English |
| Published: |
Singapore :
Springer,
[2020]
|
| Physical Description: |
1 online resource (xii, 438 pages) : illustrations. |
| Series: |
Natural computing series.
|
| Subjects: |
| Summary: |
This book delivers the state of the art in deep learning (DL) methods hybridized with evolutionary computation (EC). Over the last decade, DL has dramatically reformed many domains: computer vision, speech recognition, healthcare, and automatic game playing, to mention only a few. All DL models, using different architectures and algorithms, utilize multiple processing layers for extracting a hierarchy of abstractions of data. Their remarkable successes notwithstanding, these powerful models are facing many challenges, and this book presents the collaborative efforts by researchers in EC to solve some of the problems in DL. EC comprises optimization techniques that are useful when problems are complex or poorly understood, or insufficient information about the problem domain is available. This family of algorithms has proven effective in solving problems with challenging characteristics such as non-convexity, non-linearity, noise, and irregularity, which dampen the performance of most classic optimization schemes. Furthermore, EC has been extensively and successfully applied in artificial neural network (ANN) research -from parameter estimation to structure optimization. Consequently, EC researchers are enthusiastic about applying their arsenal for the design and optimization of deep neural networks (DNN). This book brings together the recent progress in DL research where the focus is particularly on three sub-domains that integrate EC with DL: (1) EC for hyper-parameter optimization in DNN; (2) EC for DNN architecture design; and (3) Deep neuroevolution. The book also presents interesting applications of DL with EC in real-world problems, e.g., malware classification and object detection. Additionally, it covers recent applications of EC in DL, e.g. generative adversarial networks (GAN) training and adversarial attacks. The book aims to prompt and facilitate the research in DL with EC both in theory and in practice. |
|---|---|
| Item Description: |
Includes bibliographical references and index. This book delivers the state of the art in deep learning (DL) methods hybridized with evolutionary computation (EC). Over the last decade, DL has dramatically reformed many domains: computer vision, speech recognition, healthcare, and automatic game playing, to mention only a few. All DL models, using different architectures and algorithms, utilize multiple processing layers for extracting a hierarchy of abstractions of data. Their remarkable successes notwithstanding, these powerful models are facing many challenges, and this book presents the collaborative efforts by researchers in EC to solve some of the problems in DL. EC comprises optimization techniques that are useful when problems are complex or poorly understood, or insufficient information about the problem domain is available. This family of algorithms has proven effective in solving problems with challenging characteristics such as non-convexity, non-linearity, noise, and irregularity, which dampen the performance of most classic optimization schemes. Furthermore, EC has been extensively and successfully applied in artificial neural network (ANN) research -from parameter estimation to structure optimization. Consequently, EC researchers are enthusiastic about applying their arsenal for the design and optimization of deep neural networks (DNN). This book brings together the recent progress in DL research where the focus is particularly on three sub-domains that integrate EC with DL: (1) EC for hyper-parameter optimization in DNN; (2) EC for DNN architecture design; and (3) Deep neuroevolution. The book also presents interesting applications of DL with EC in real-world problems, e.g., malware classification and object detection. Additionally, it covers recent applications of EC in DL, e.g. generative adversarial networks (GAN) training and adversarial attacks. The book aims to prompt and facilitate the research in DL with EC both in theory and in practice. Intro -- Preface -- Contents -- Abbreviations -- Part I Preliminaries -- 1 Evolutionary Computation and Meta-heuristics -- 1.1 Introduction -- 1.2 Evolutionary Algorithms: From Bullet Trains to Finance and Robots -- 1.3 Multi-Objective Optimization -- 1.4 Genetic Programming and Its Genome Representation -- 1.4.1 Tree-based Representation of Genetic Programming -- 1.4.2 Cartesian Genetic Programming (CGP) -- 1.5 Ant Colony Optimization (ACO) -- 1.6 Particle Swarm Optimization (PSO) -- 1.7 Artificial Bee Colony Optimization (ABC) -- 1.8 Firefly Algorithms -- 1.9 Cuckoo Search. 1.10 Harmony Search (HS) -- 1.11 Conclusion -- References -- 2 A Shallow Introduction to Deep Neural Networks -- 2.1 Introduction -- 2.2 (Shallow) Neural Networks -- 2.2.1 Backpropagation Algorithm for Training NNs -- 2.3 Deep Neural Networks: What, Why and How? -- 2.4 Architectures of Deep Networks -- 2.4.1 Convolutional Neural Network -- 2.4.1.1 Convolutional Layers -- 2.4.1.2 Pooling Layers -- 2.4.1.3 Fully Connected Layers -- 2.4.1.4 Training Strategies -- 2.4.1.5 Popular CNN Models -- 2.4.2 Recurrent Neural Network -- 2.4.2.1 RNN Architecture -- 2.4.2.2 RNN Training -- 2.4.2.3 Memory Cells. 2.4.3 Deep Autoencoder -- 2.4.4 Deep Belief Network (DBN) -- 2.4.5 Generative Adversarial Network (GAN) -- 2.4.5.1 GAN Architecture -- 2.4.5.2 GAN Training -- 2.4.5.3 Progresses in GAN Research -- 2.4.6 Recursive Neural Networks -- 2.5 Applications of Deep Learning -- 2.6 Conclusion -- References -- Part II Hyper-Parameter Optimization -- 3 On the Assessment of Nature-Inspired Meta-Heuristic Optimization Techniques to Fine-Tune Deep Belief Networks -- 3.1 Introduction -- 3.2 Theoretical Background -- 3.2.1 Restricted Boltzmann Machines -- 3.2.2 Contrastive Divergence. 3.2.3 Persistent Contrastive Divergence -- 3.2.4 Deep Belief Networks -- 3.3 Meta-heuristic Optimization Algorithms -- 3.4 Methodology -- 3.4.1 Modeling DBN Hyper-parameter Fine-tuning -- 3.4.2 Datasets -- 3.4.3 Experimental Setup -- 3.5 Experimental Results -- 3.5.1 Training Evaluation -- 3.5.2 Time Analysis -- 3.5.3 Hyper-Parameters Analysis -- 3.6 Conclusions and Future Works -- References -- 4 Automated Development of DNN Based Spoken Language Systems Using Evolutionary Algorithms -- 4.1 Spoken Language Processing Systems -- 4.1.1 Principle of Speech Recognition. 4.1.2 Hidden Markov Model Based Acoustic Modeling -- 4.1.3 End-to-End Speech Recognition System -- 4.1.4 Evaluation Measures -- 4.2 Evolutionary Algorithms -- 4.2.1 Genetic Algorithm -- 4.2.2 Evolution Strategy -- 4.2.3 Bayesian Optimization -- 4.3 Multi-Objective Optimization with Pareto Optimality -- 4.3.1 Pareto Optimality -- 4.3.2 CMA-ES with Pareto Optimality -- 4.3.3 Alternative Multi-Objective Methods -- 4.4 Experimental Setups -- 4.4.1 General Setups -- 4.4.2 Automatic Optimizations -- 4.5 Results -- 4.6 Conclusion -- References. 5 Search Heuristics for the Optimization of DBN for Time Series Forecasting. |
| Physical Description: |
1 online resource (xii, 438 pages) : illustrations. |
| Bibliography: |
Includes bibliographical references and index. |
| ISBN: |
9789811536854 9811536856 |