Hands-On Neural Network Programming With C# by Cole, Matt R. -- Read -- Imperial Library of Trantor

Index

Title Page Copyright and Credits

Hands-On Neural Network Programming with C#

Dedication Packt Upsell

Why subscribe? Packt.com

Contributors

About the author About the reviewers Packt is searching for authors like you

Preface

Who this book is for What this book covers To get the most out of this book

Download the example code files Download the color images

Code in Action

Conventions used

Get in touch

Reviews

A Quick Refresher

Technical requirements Neural network overview

Neural network training A visual guide to neural networks

The role of neural networks in today's enterprises Types of learning

Supervised learning Unsupervised learning Reinforcement learning

Understanding perceptrons

Is this useful?

Understanding activation functions

Visual activation function plotting Function plotting

Understanding back propagation

Forward and back propagation differences

Summary References

Building Our First Neural Network Together

Technical requirements Our neural network Neural network training

Synapses Neurons Forward propagation Sigmoid function Backward propagation Calculating errors Calculating a gradient Updating weights Calculating values

Neural network functions

Creating a new network Importing an existing network Importing datasets Testing the network Exporting the network Training the network Testing the network Computing forward propagation Exporting the network Exporting a dataset

The neural network

Neuron connection

Examples

Training to a minimum Training to a maximum

Summary

Decision Trees and Random Forests

Technical requirements Decision trees

Decision tree advantages Decision tree disadvantages When should we use a decision tree?

Random forests

Random forest advantages Random forest disadvantages When should we use a random forest?

SharpLearning

Terminology Loading and saving models

Example code and applications

Saving a model Mean squared error regression metric F1 score Optimizations Sample application 1

The code

Sample application 2 – wine quality

The code

Summary References

Face and Motion Detection

Technical requirements Facial detection Motion detection

Code

Summary

Training CNNs Using ConvNetSharp

Technical requirements Getting acquainted Filters Creating a network

Example 1 – a simple example Example 2 – another simple example Example 3 – our final simple example Using the Fluent API

GPU Fluent training with the MNIST database Training the network

Testing the data Predicting data Computational graphs

Summary References

Training Autoencoders Using RNNSharp

Technical requirements What is an autoencoder? Different types of autoencoder

Standard autoencoder Variational autoencoders De-noising autoencoders Sparse autoencoders

Creating your own autoencoder Summary References

Replacing Back Propagation with PSO

Technical requirements Basic theory

Swarm intelligence Particle Swarm Optimization

Types of Particle Swarm Optimizations Original Particle Swarm Optimization strategy Particle Swarm Optimization search strategy

Particle Swarm Optimization search strategy pseudo-code

Parameter effects on optimization

Replacing back propagation with Particle Swarm Optimization Summary

Function Optimizations: How and Why

Technical requirements Getting started Function minimization and maximization

What is a particle? Swarm initialization Chart initialization State initialization Controlling randomness Updating the swarm position Updating the swarm speed Main program initialization Running Particle Swarm Optimization Our user interface

Run button Rewind button Back button Play button Pause button Forward button

Hyperparameters and tuning

Function Strategy Dim size Upper bound Lower bound Upper bound speed Lower bound speed Decimal places Swarm size Max iterations Inertia Social weight Cognitive weight Inertia weight

Understanding visualizations

Understanding two-dimensional visualizations Understanding three-dimensional visualizations

Plotting results

Playing back results Updating the information tree

Adding new optimization functions

The purpose of functions Adding new functions Let's add a new function

Summary

Finding Optimal Parameters

Technical requirements Optimization

What is a fitness function?

Maximization Gradient-based optimization Heuristic optimization

Constraints

Boundaries Penalty functions General constraints Constrained optimization phases Constrained optimization difficulties Implementation

Meta-optimization

Fitness normalization Fitness weights for multiple problems Advice Constraints and meta-optimization Meta-meta-optimization

Optimization methods

Choosing an optimizer Gradient descent (GD)

How it works Drawbacks

Pattern Search (PS)

How it works

Local Unimodal Sampling (LUS)

How it works

Differential Evolution (DE)

How it works

Particle Swarm Optimization (PSO)

How it works

Many Optimizing Liaisons (MOL) Mesh (MESH)

Parallelism

Parallelizing the optimization problem Parallel optimization methods

Necessary parameter tuning

And finally, the code Performing meta-optimization Computing fitness Testing custom problems Base problem Creating a custom problem Our Custom Problem

Summary References

Object Detection with TensorFlowSharp

Technical requirements Working with Tensors

TensorFlowSharp

Developing your own TensorFlow application Detecting images

Minimum score for object highlighting

Summary References

Time Series Prediction and LSTM Using CNTK

Technical requirements Long short-term memory

LSTM variants Applications of LSTM

CNTK terminology Our example

Coding our application

Loading data and graphs Loading training data Populating the graphs Splitting data

Running the application Training the network Creating a model Getting the next data batch Creating a batch of data

How well do LSTMs perform? Summary References

GRUs Compared to LSTMs, RNNs, and Feedforward networks

Technical requirements QuickNN Understanding GRUs Differences between LSTM and GRU

Using a GRU versus a LSTM

Coding different networks

Coding an LSTM Coding a GRU

Comparing LSTM, GRU, Feedforward, and RNN operations Network differences Summary

Activation Function Timings Function Optimization Reference

The Currin Exponential function

Description Input domain Modifications and alternative forms

The Webster function

Description Input distributions

The Oakley & O'Hagan function

Description Input domain

The Grammacy function

Description Input fomain

Franke's function

Description Input domain

The Lim function

Description Input domain

The Ackley function