Genetic Programming for Image Classification

Appendix

Implementation Instructions

The appendix provides instructions on the implementations of several Genetic Programming (GP)-based algorithms presented in this book.

A.1 Basic Implementation of GP

The implementation of the GP-based algorithms are based on the DEAP (Distributed Evolutionary Algorithm in Python) [1] package in Python.

A.1.1 Genetic Operators

Under the DEAP framework, the selection method and the elitism, crossover and mutation operators for the GP-based algorithms can be set as follows.

../images/502867_1_En_BookBackmatter_Figa_HTML.png

A.1.2 Parameter Settings

Basic parameters for GP include the followings.

../images/502867_1_En_BookBackmatter_Figb_HTML.png

A.1.3 Fitness Function

The classification accuracy is the most commonly used fitness function of GP-based feature learning algorithms in image classification. In the proposed GP-based algorithms and many existing GP-based algorithms, there are two common ways to calculate the classification accuracy, i.e., using GP trees as classifiers and using external classifiers. The implementations of these two fitness evaluation methods are presented as follows.

A.1.3.1 Use GP Trees as Classifiers for Fitness Evaluation

In this fitness evaluation method, the output (feature) of a GP tree is often compared with a threshold, e.g., 0, to determine which class the input instance/image belongs to. The overall process can be found from the following codes.

../images/502867_1_En_BookBackmatter_Figc_HTML.png

A.1.3.2 Use External Classifiers for Fitness Evaluation

In this fitness evaluation method, a traditional classification algorithm is employed to train a classifier. In this process, the original training set (an image classification dataset has a training set and a test set, but the test set cannot be used during the evolutionary process) is split into an evaluation training set and an evaluation test set using the k-fold cross-validation or the hold-out method. The evaluation training set is used to train a classifier and the evaluation test set is used to test the classifier. For example, the fitness evaluation process based on stratified k-fold cross-validation is listed as follows.

../images/502867_1_En_BookBackmatter_Figd_HTML.png

A.2 MLGP: Multi-layer Genetic Programming

The codes of the main MLGP algorithm (presented in Chap. 4) are listed as follows. The runnable codes can be downloaded from https://github.com/YingBi92/BookCode/MLGP.

../images/502867_1_En_BookBackmatter_Fige_HTML.png

../images/502867_1_En_BookBackmatter_Figf_HTML.png

../images/502867_1_En_BookBackmatter_Figg_HTML.png

A.3 COGP: Genetic Programming with Convolution Operator

The codes of the main COGP algorithm (presented in Chap. 5) are listed as follows. The runnable codes can be downloaded from https://github.com/YingBi92/BookCode/COGP.

../images/502867_1_En_BookBackmatter_Figh_HTML.png

../images/502867_1_En_BookBackmatter_Figi_HTML.png

../images/502867_1_En_BookBackmatter_Figj_HTML.png

../images/502867_1_En_BookBackmatter_Figk_HTML.png

A.4 IDGP: Genetic Programming with Image Descriptors

The codes of the main IDGP algorithm (presented in Chap. 6) are listed as follows. The runnable codes can be downloaded from https://github.com/YingBi92/BookCode/IDGP.

../images/502867_1_En_BookBackmatter_Figl_HTML.png

A.5 FlexGP: Genetic Programming with Image-Related Operators

The codes of the main FlexGP algorithm (presented in Chap. 7) are listed as follows. The runnable codes can be downloaded from https://github.com/YingBi92/BookCode/FlexGP.

../images/502867_1_En_BookBackmatter_Figm_HTML.png

../images/502867_1_En_BookBackmatter_Fign_HTML.png

../images/502867_1_En_BookBackmatter_Figo_HTML.png

A.6 FELGP: Genetic Programming for Feature Learning and Ensemble Learning

The codes of the main FELGP algorithm (presented in Chap. 8) are listed as follows. The runnable codes can be downloaded from https://github.com/YingBi92/BookCode/FELGP.

../images/502867_1_En_BookBackmatter_Figp_HTML.png

../images/502867_1_En_BookBackmatter_Figq_HTML.png

Index

Symbols

k-nearest neighbours

2TGP

3TGP

CNN-GA

Shape and appearance feature

Adaptability

AlexNet

AmoebaNet-A

Ant colony optimisation

Approximation performance

Artificial bee colony

Artificial intelligence

Artificial neural networks

applications

layers

Automated design/learning

Averaging filter

Bayesian classifiers

Benchmark datasets

Benchmark problems

Binary classification

Binary image classification

British Machine Vision Association and Society for Pattern Recognition

Building blocks

CGP-CNN

Classification

classification algorithms

classifier

generalisation error

testing

test set

training

training error

training set

validation

validation set

Classification accuracy

Classification and regression trees

Classification function

Classification layer

Classification strategy

COGP

Combination function

Combination layer

Computational intelligence

Computer vision

applications

definition

image classification

object classification

object detection

object recognition

Concatenation function

Concatenation layer

ConvGP

Convolution

Convolutional neural networks

architecture

convolutional layer

example

fully connected layer

pooling layer

types

Convolution function

Convolution layer

Cross-validation

-fold cross-validation

leave-

-out cross-validation

leave-one-out cross-validation

Dataset splitting

Data type

Decision tree

Deep CNNs

Deep learning

Deep neural networks

DenseNet

DENSER

Dense SIFT

Difference of Gaussian filter

Differential evolution

Digit recognition

Domain knowledge

Ensembel methods

Ensemble learning

AdaBoost

architecture

bagging

boosting

combination

diversity

generalisation

heterogeneous ensemble

homogeneous ensemble

random forest (RF)

EvoCNN

Evolutionary algorithms

Evolutionary computation

evolutionary algorithms

others

swarm intelligence

Evolutionary computer vision

Evolutionary deep learning

Evolutionary multi-objective

Evolutionary process

Evolutionary programming

Evolutionary strategies

Evolving neural networks

Example GP program

Experiments

Face recognition

Facial expression classification

Feature concatenation function

Feature construction

Feature construction function

Feature construction layer

Feature extraction

Feature extraction function

Feature extraction layer

Feature learning

process

Feature normalisation

Feature number

Feature selection

Feature visualisation

FELGP

Filtering

Filtering & pooling function

Filtering & pooling layer

Filtering function

Fine-grained image classification

First-order edge detector

Fitness evaluation

Fitness function

Fitness prediction

FlexGP

Flexibility

Flexible representation

Foreword

Function frequency

Function set

Gamma correction

Gaussian filter

Generality

Genetic algorithms

Genetic CNN

Genetic programming (GP)

algorithm framework

crossover

fitness evaluation

functions

mutation

population initialisation

process

representation

reproduction

selection

terminals

GLCM

Global features

GoogleNet

GP-HoG

GP parameters

GP representation

Gradient magnitude

Haralick Features

High dimensionality

Histogram

Histogram of oriented gradients (HOG)

Hold-out

Human vision

IDGP

Image classification

classification

feature extraction

image preprocessing

Image descriptors

Image features

Image gradients

Image normalisation

Image preprocessing

edge detector

histogram equalisation

image smoothing

Image variation

Input layer

Interpretability

Iterative dichotomiser 3 (ID3)

Kernel functions

Keypoint description

Keypoint localisation

Keypoints

Knowledge transfer

Laplacian filter

Laplacian of Gaussian filter

LBP variants

Learning ability

Local binary patterns (LBP)

Local features

Logistic regression

Machine learning

definition

terminology

types

Mean filter

Median filter

MNIST

Model evaluation

Model selection

Multi-layer GP (MLGP)

Multi-objective evolutionary algorithms

Multi-objective optimisation

Multiple tasks

Naïve Bayes

Object classification

Object recognition

One-vs-one (OvO)

One-vs-rest (OvR)

Orientation assignment

Output layer

Overfitting

Painting classification

Particle swarm optimisation

Per-pixel transformation

Pooling function

Pooling layer

Population initialisation

Prewitt edge detector

Program structure

Region detection function

Region detection layer

Regions of interest

Reinforcement learning

q-learning

temporal difference learning

Representation

Representation learning

ResNet

RF-FlexGP

Roberts edge detector

Root mean square error

Scale invariant feature transform (SIFT)

Scale-space extrema detection

Scene classification

SCOOP

Second-order edge detector

Sobel edge detector

Softmax regression

Spearman's rank correlation coefficient

Speeded-up robust features (SURF)

Standard GP

Statistics features of GLCM

Strongly typed genetic programming

example

input type

output type

type constraints

Supervised learning

classification

regression

Support vector machines

Surrogate

Surrogate-assisted evolutionary algorithms

Surrogate-assisted FlexGP

Surrogate modelling

Surrogate performance measure

Survival of the fittest

Swarm intelligence

Syntax tree

Terminal set

Texture classification

Tournament selection

Transfer learning