Chapter 11

Pattern Classification and Its Applications to Control of Biomechatronic Systems

Victor H. Benitez, PhD    

Abstract

This chapter presents a new approach to identify, classify, and control biomechatronic systems, which are controlled via superficial electromyographic signals generated by the upper limbs. Electromyographic data are recorded while the hand of subjects is constricted to grasps a set of spheres with a small variation in diameter. Five muscles are monitored with noninvasive electrodes placed on the skin of volunteers while a set of grasp–hold–relax tasks are carried out randomly. A preprocess stage is performed to extract time domain features from the data, with the purpose of addressing both the course of dimensionality and the issues related to the nonstationary behavior of electromyographic signals. A pattern recognition module is used to classify the data and to assign the extracted features to categories corresponding to each sphere grasped. A tracking generator is proposed using artificial neural networks, which are trained to learn the dynamics of finger motions. The performance of the methodology is evaluated in simulations and via real-time implementation with an embedded system.

Keywords

Biomechatronics; Electromyography; Myoelectric control systems

11.1 Introduction

Biomedical (BM) signals are generated by the activity of living beings and they are manifested as electrical or magnetic phenomena that can be recorded and analyzed. They constitute a rich source of data that encode important information about the living organ that generates such information.

Particularly in human beings, electrocardiography (ECG), electroencephalography (EEG), and electromyography (EMG) are examples of techniques developed to record the electrical activity of heart, brain, and muscles, respectively. The BM signals generated by ECG, EEG, and EMG techniques are mostly used to diagnose various diseases. However, in the last decade, an enormous amount of research has been done regarding the use of BM signals to develop human machine interfaces (HMIs) [18,11]. It is impossible to cover the vast number of applications related to BM signals. However, of the three techniques described above, EMG stands out for its applicability in the design of biomechatronic (BioM) systems.

BioM is concerned with the integration of biology disciplines into traditional mechatronics [24], i.e., BioM is related to the research and design of devices which integrate biology disciplines with mechanics, electronics, and control technology with assistive, diagnostic, or therapeutic purposes, or even to augment or to enhance human capabilities.

This chapter presents a methodology to process EMG signals in order to design a pattern classification module and to develop a BioM system capable of studying the finger movements of the human hand. The proposed approach is constituted by the following components:

• •  a neural network control module;
• •  a pattern classification module;
• •  a preprocessing module;
• •  an embedded system platform used as a testbed and a brushless DC motor as a servo mechanism.

The BioM system is designed to control brushless DC motors from myoelectric signals obtained from the forearm of healthy subjects. The EMG technique is employed to acquire and record the signals generated by five superficial muscles of the forearm; a preprocessing stage is carried out via Matlab and a control system is developed to generate trajectories applied to control brushless DC motors. The approach is tested both in simulation and in real-time via an embedded servo system developed in our laboratory. A key subject in the development of advanced BioM human machine interfaces based on EMG signals is the processing technique employed. EMG is an experimental technique concerned with the development, recording, and analysis of myoelectric signals. Myoelectric signals are formed by physiological variations in the state of muscle fiber membranes.

11.2 Biomechatronic System Components

The monitoring of muscle activity opens a vast panorama of lines of research to develop applications in various fields of engineering. Especially the development of control systems based on EMG signals has grown enormously in recent years [23,25,26]. A particular case of BioM systems is the so-called myoelectric control system, which uses exclusively EMG signals as inputs to design a control system. A myoelectric control system [3] is constituted by the following modules: a biological system, surface electrodes, signal conditioning, and signal processing.

11.2.1 Biological System

The electrical manifestation of the neuromuscular activation is directly related with muscle contraction. The myoelectric signals generated by muscle contraction can be detected by two types of electrodes: one type is located on the skin surface, the other is inserted as a needle electrode. The signals recorded by means of the first technique are called superficial EMG (sEMG) signals. We use sEMG signals because this technique is less invasive for the user and the experiments can be done in a regular facility or laboratory without medical requirements.

11.2.2 Surface Electrodes

Electrodes consists of a metal detection surface that detects the electric current on the skin. Some electrodes include amplifier circuits with high input impedance; others are simple passive electrodes that adhere to the skin with specialized adhesive. The location of the electrodes plays a crucial role in EMG studies; considerations such as signal/noise ratio, signal reliability, and cross-talk must be taken into consideration to locate electrodes in the muscle of interest [14].

11.2.3 Signal Conditioning

This block is constituted by at least the following components:

• •  amplifier;
• •  filter;
• •  sampler.

The electrical manifestation of muscles goes from microvolts to millivolts; therefore, an amplifier stage is required to raise the magnitude of the signal to an adequate value.

11.2.4 Signal Processing

This block comprises three functional subprocesses:

• •  data segmentation;
• •  feature extraction;
• •  classification.

11.2.4.1 Data Segmentation

Studies performed by [10] highlighted the importance of segmenting the raw sEMG signal into time slots in order to extract features of myoelectric data. There are two popular techniques to segment raw sEMG data: adjacent windowing and overlapped windowing techniques [10]. In the first case, adjacent disjoint data segments with a predetermined size are used for feature extraction; with this technique, a processing delay is introduced after which each classified pattern emerges. In Fig. 11.1, the adjacent windowing technique is illustrated. The processing time delay is also depicted.

Fig. 11.1 Adjacent segmenting technique.

11.2.4.2 Feature Extraction

It is impractical to feed the classifier with the raw signal due to the large amount of data and the stochastic and nonstationarity nature of any biomedical signal. Instead, segmented data are mapped into a lower dimension feature vector whose entries are the features of the signal. According to the literature, features can be categorized in three groups: time frequency (TF), frequency, and time domain (TD) [27].

TD features are the ones most used in myoelectric classification due to their comparative computational simplicity, which in turn allows them to be implemented in real-time more easily. For the sake of completeness TD features are described as follows.

Mean of the Absolute Value

A calculation for mean absolute value (MAV) of the signal in segment i with N samples in length is given by

X i ‾ = 1 N ∑ k = 1 N | x k | ,

(11.1)

where N is the total number of data points in a window segment, whose length is determined by the sampling frequency, and xk is an EMG data value.

Root Mean Square

Root mean square (RMS) measures the signal energy, providing a strong measure of the information content of the signal. It is defined as

X R M S = 1 N ∑ k = 1 N | x k | 2 .

(11.2)

Variance

Variance (VAR) is a representation of EMG signal power, it is related to the force developed by the muscle, and it is defined as

V A R = 1 N − 1 ∑ k = 1 N ( x k − μ ) 2 .

(11.3)

Zero Crossing

Zero crossing is the number of times that the waveform crosses zero; it is calculated for two consecutive samples given. Zero crossing increases if the following condition is fulfilled:

sign ( − x k × x k + 1 ) and ( | x k − x k + 1 | ⩾ threshold ) .

(11.4)

Waveform Length

The waveform length gives information relative to the cumulative length of the waveform over the time segment and is defined as

W L = ∑ k = 1 N | x k − x k − 1 | .

(11.5)

Willison Amplitude

This feature represents, for every two consecutive data points, the number of times that the magnitude of the difference exceeds a threshold value. The Willison amplitude correlates to muscle contraction levels. It is defined as

W = ∑ k = 1 N f ( x k − x k − 1 ) ,

(11.6)

where

f ( x ) = { 1 , x > threshold , 0 otherwise .

Energy

The area under the curve for two consecutive samples is determined as

E = 1 F s ∑ k = 1 N | x k | 2 ,

(11.7)

where Fs is the sampling frequency.

Average Power

The average power (P) of a sequence with N samples is defined as

P = 1 N ∑ k = 1 N | x k | 2 .

(11.8)

On the other hand, the TF representation of features allows to obtain a more accurate description of the physical phenomenon because TF analysis can localize the energy of the signal in both time and frequency. However, TF features are generally computationally expensive, which is a serious restriction in real-time applications. Therefore, linear, discrete TFs, such as short-time Fourier transform (STFT), wavelet transform (WT), and wavelet packet transform (WPT), are selected to extract features from EMG signals.

Short-Time Fourier Transform

Given a finite-length sequence xk, k∈{0,...,L−1}, with L as the length of the segment. Then its discrete Fourier transform (DFT) is defined as

X [ m F ] = ∑ k = 0 L − 1 x [ k ] e − j 2 π ( m F ) ( k T s ) ,

(11.9)

where F=1/LTs is the frequency sampling step size. The STFT is obtained as a series of DTFs, indexed with respect to Ts, i.e.,

S T F T [ k T s , m F ] = ∑ k = 0 L − 1 x [ k ] g [ k − j ] e − j 2 π m k / L ,

(11.10)

where g[k] is the window function. The advantage of STFT is that it has a well-developed theory and can be computed very efficiently; the main drawback is that it requires each cell in the TF plane to have an identical shape, as depicted in Fig. 11.2.

Fig. 11.2 Tiling of short-time Fourier transform.

Wavelet Transform

In the WT, by means of a variation in the TF ratio, it is possible to overcome the main drawback of the STFT. Additionally, with a good frequency resolution Δf in low frequencies and a good time localization Δt at high frequencies, it is possible to produce a tiling of the time–frequency plane that is appropriate for most BM signals (Fig. 11.3).

Fig. 11.3 Tiling of WT.

The continuous WT is defined as

C W T x ( τ , a ) = 1 a ∫ x ( t ) Ψ ( t − τ a ) d t ,

(11.11)

where Ψ(t) represents the mother wavelet.

Wavelet Packet Transform

The WPT is a generalized version of the continuous WT and the discrete WT. The tiling of the time–frequency plane is configurable, as shown in Fig. 11.4.

Fig. 11.4 Tiling of WT.

A detailed review of the applicability of TF features is reported in [9].

11.2.4.3 Classification

It is well known in the literature that EMG signals are stochastic and nonlinear, with a nonstationary behavior; therefore, the properties of myoelectric signals change over time. Due to the BM nature of myoelectric signals, it is reasonable to expect large variations in the values of a particular feature. Various external factors, such as variations in electrode location, muscular fatigue, muscle tone variation between individuals, gender of subjects, and sweat, cause changes in a signal pattern over time. The pattern classifier must be able to deal with such varying features optimally and overcome overfitting. If real-time requirements must be fulfilled, the classifier processing time should be fast enough to meet real-time constraints.

11.2.5 Controller Deployment and User Application

Real-time constrictions and increasing demands for applications with high performance require a broad integration of technologies in the field of BioM such as powerful actuators, tiny gears, novel sensors, and miniature electronics. Advances in various fields of technology make it possible to integrate, for example, superflat BLDC motors with harmonic drivers and powerful 32-bit processors with smaller MOSFET power drivers. Prostheses, therapeutic and assistive/enhancing devices, requires BioM systems to fulfill requirements such as having a low weight and a small volume, being more human-like, having more flexible communication interfaces, having more manipulation capability, having more flexible stiffness in the joints, instead of rigid gears and linkage transmissions, allowing for high integration with the human body, containing a minimal amount of wires, and being capable of inertial sensing. Hardware architecture should meet high performance. Examples of powerful embedded systems are FPGAs, DSPs, and microcontrollers [16]. The building blocks of a myocontrol system are shown in Fig. 11.5.

Fig. 11.5 Block diagram for myocontrol system.

11.2.6 Neural Network Control

Recurrent high-order neural networks (RHONNs) can be represented by

x i ˙ = − a i x i + ∑ k = 1 L i ′ w i k z I i k + ∑ k = L i ′ + 1 L i w i j z I i j ,

(11.12)

where ai>0, xi is the state of the neuron, wik are the synaptic weights, and zIik are multiplicative terms of sigmoidal functions of x or u. On the other hand, wij are fixed weights, which allow us to incorporate a priori information about the structure of the mapping that we wish to model, zIij are functions of x or of u, Li′ is the number of synaptic weights, and Li is the number of neural connections of the ith neuron. The properties of (11.12) as identification and control scheme as well as its digital implementation have been widely described in works such as [5,4,2].

11.3 Biomechatronic System Proposed

One of the main challenges in myoelectric control is to classify the signals issued by muscles, as they exhibit nonstationary behavior. Current approaches use myoelectric signals to characterize the position of the hand, where fingers are located in very well distinguishable positions. In [22], a study to decode individual flexion and extension movements of fingers is presented with 90% accuracy. They employed TD features to feed a multilayer perceptron used to classify 12 movements. Reference [15] presents a strategy to classify finger motion applied to movement recognition for Indonesian sign language systems, where a multimodal controller is developed to deal with uncertainties with individual variations in performing sign language. A naive Bayes classifier is implemented which reaches classification rates up to 98%. Another application for finger motion classifications can be found in [17], where a method for predicting individual finger movements from surface EMG is presented. They use just one TD feature, the mean average value, to develop a linear model of EMG feature dynamics, and Bayesian inference is used as classifier.

However, those approaches fail in characterizing small changes in finger movements. To the best of our knowledge, there are no reports in the literature on any methodology to study finger movements, whose angular motion varies with a small desired value. In the next sections, a method to distinguish several finger positions with very low mechanical fluctuations is presented. The methodology is verified with simulations and with a BioM system designed in our laboratory, which is controlled via superficial EMG signals generated by the forearm.

11.3.1 Problem Formulation

It is well known that in motion control, there are primitive variables such as position (θ), velocity (ω), acceleration (α), jerk (φ), and torque (τ). It is possible to establish such variables as dynamics of electromechanical servo systems [8]. These dynamics can be used to obtain complex motion patterns M, which can be represented as a set of nonlinear functions, i.e.,

M = f ( θ , ω , α , φ , τ ) .

(11.13)

Proposition

We propose that it is possible to study finger motion by means of simple mechanical variables M in a controlled environment, i.e., a controlled sequence of continuous actions called scenes, as shown in Fig. 11.6. □

Fig. 11.6 The hand grasping an object in a scene.

In order to study the finger movement in a structured environment, let us introduce the term scene, which is defined here in below. The so-called scenes are constituted by movement sequences as follows.

Let n be a sequence of hand movements. A scene S is formally described as:

• •  Sequence 1: Hand performs a movement G0, into an scene S with a period of time t0<t⩽ti.
• •  Sequence k: Hand performs a movement Gk=Gk−1+ΔΓ, into a scene S with a duration ti<t⩽tk, where ΔΓ is a structured variation in the Euclidian space (Fig. 11.6).
• •  Sequence n: Hand performs a movement Gn=Gn−1+ΔΓ, into a scene S with a duration tn−1<t⩽tn.

Definition

A scene S is constituted by the set of movements

S = { G 0 , G 1 , . . . , G k , . . . , G n } .

(11.14)

 □

Considering that Gk terms are described by Gk−1 plus a differential ΔΓ, the movements are represented as follows:

G 0 , G 1 = G 0 + k 1 , G 2 = G 1 + k 2 , G 3 = G 2 + k 3 , ⋮ G n = G n − 1 + k n − 1 .

(11.15)

Substituting backward and reordering, (11.15) is represented as

G 0 = i n i t i a l _ c o n d , G 1 = G 0 + k 1 , G 2 = G 0 + k 1 + k 2 , G 3 = G 0 + k 1 + k 2 + k 3 , ⋮ G n = G 0 + K ,

(11.16)

where K=k1+k2+...+kn. Then, if (11.16) holds, the set S can be defined in terms of a triplet, i.e.,

S = { G 0 , G n , K } .

(11.17)

In order to fulfill (11.17), the following definitions are made.

Definition

The metric of Gi is

‖ G i ‖ = G i − 1 + k i .

 □

Definition

The differentials ki are selected in such way that their values represent the anatomical variation of the subject being studied. □

Definition

All ki terms are not necessary equals, but their sum is always equal to the final pose of fingers. □

As a consequence of the definition of K in (11.16), one can select the finger mechanical variation ki as small as possible and then build up a model M as (11.13), according to the ki selected values.

In the following sections, the definition of S, which constitutes a scene, is used to develop a structured environment such as grasping spheres with diameters Di, which are selected with parameter variation ki according to (11.16).

The range of movements (ROM) that the human hand can naturally perform is limited by several constraints, which can be grouped into three categories: static, intrafinger, and interfinger [7]. In this study only intrafinger constraints are considered because they are related to movement of flexion/extension of the fingers.

11.3.2 Experimental Preparation

Six healthy able-bodied subjects (four men, two women) aged 20–24 were trained to perform the experiment described in the next subsection. The experiment was conducted in accordance with the normative of the Ethics Committee of our institution; an ethical approval letter from the committee and consent of the volunteers was obtained prior to the experiment. The guidelines established in the surface EMG for the noninvasive assessment of muscles (SENIAM) project were followed (available at http://www.seniam.org).

Data were measured with a Trigno Wireless System (Delsys, Boston, Ma, USA); five channels were used in the experiment. Sampling frequency was set to 2000 Hz with a low-pass filter at 500 Hz; channel amplifiers were included in the experiment. Wireless RF electrodes were located on the arm, which was shaved, cleaned, and scrubbed with alcohol. Fig. 11.7 depicts the muscle electrode placement in the arm; the EMG system employed is illustrated in Fig. 11.8.

Fig. 11.7 Superficial muscles used to capture sEMG data.

Fig. 11.8 Hardware used to capture sEMG data.

The designed experiment considered aspects such as determination of the type and amount of movements of the fingers, the number of participants, and the muscles of the forearm to be included, which will also influence the number of channels and dimensions of the characteristic vector as well as the quantity of objects to grasp by the hand.

11.3.3 Experimental Set-Up

Subjects were instructed to grasp five spheres randomly. The spheres were made of expanded polystyrene with diameters D1=5.715, D2=6.985, D3=7.7978, D4=9.0678, and D5=10 cm. Volunteers were seated comfortably and performed the requested grabbing of a sphere Di (i=1,...,5). Electrodes were located on the following five superficial muscles: flexor carpi radialis, flexor carpi ulnaris, flexor pollicis longus, flexor digitorum profundus, and extensor digitorum [14] (Fig. 11.7). The experiment was organized in batches; each subject performed a batch, which is established as the grasps of five spheres with a span of 5 seconds between each grasp. A relaxing period of 20 seconds between batches was introduced to avoid muscular fatigue. Subjects were prompted to perform six batch repetitions. Five repetitions were used as training data, one was used for validation. In order to synchronize the task of grasping and the recording of signals, subjects were trained to introduce a short contraction at the beginning and at the ending of grasping each sphere. The short contractions are used to get markers that can be easily handled by a maximum peak detector in the capturing system. The whole batch can be summarized by the following tasks (Fig. 11.9):

• •  open hand,
• •  get sphere,
• •  close hand,
• •  make a short contraction,
• •  hold sphere i,
• •  make a short contraction,
• •  relax,
• •  repeat 1 to 6 three times,
• •  relax 20 seconds,
• •  repeat steps 1 to 8 thirty times.

Fig. 11.9 Flow diagram of the experimental set-up.

Before beginning the experiment, subjects were asked to grasp each sphere, and the angular positions of the fingers were measured with a goniometer. Four finger joints were measured for each sphere: the distal interphalangeal (DIP), proximal interphalangeal (PIP), metacarpophalangeal (MP), and trapezius-metacarpal (CMC) joints. The data for subject F1 are shown in Table 11.1. The grasping spheres (D1,..,D5) order was determined randomly. Table 11.2 shows the grasping sequence order for F1.

Table 11.1

Joint angles for F1 (degrees).

Subject: F1 Weight: 56 kg Age: 22 years Height: 1.64 m
Sphere	Thumb		Index			Middle			Ring			Little
	MP	PIP	MP	PIP	DIP	MP	PIP	DIP	MP	PIP	DIP	MP	PIP	DIP
D1	25	50	50	40	30	65	50	35	55	50	25	53	33	18
D2	24	45	40	40	25	55	50	30	50	40	25	43	20	20
D3	18	40	40	40	25	55	40	25	40	40	20	50	18	16
D4	15	40	35	30	15	40	40	25	40	28	20	33	15	15
D5	10	35	30	35	25	40	38	18	30	30	18	35	13	15

Table 11.2

Grasping sequence order for F1.

Experiment	Sequence of grasping
1	D1, D3, D4, D2, D5
2	D2, D5, D1, D3, D4
3	D4, D1, D3, D2, D5
4	D3, D1, D2, D5, D4
5	D5, D1, D3, D2, D4
6	D4, D3, D5, D1, D2

11.3.4 Signal Processing

11.3.4.1 Data Segmentation

A segmentation process is performed in order to handle the data before feature extraction. Each sEMG signal is segmented considering a segment length of 256 ms [3]. The DC content of raw data is eliminated and normalized in order to improve the performance of the classifier because TD features are of different orders of magnitude. The min-max approach is selected to map data to [01], which is expressed as

x ‾ = x − min ⁡ x max ⁡ x − min ⁡ x .

In Fig. 11.10, the raw data are shown for subject F1.

Fig. 11.10 Raw data for F1.

11.3.4.2 Feature Extraction

Instead of using raw signals, which is impractical due to the randomness of the sEMG signals, data are mapped into a lower dimension vector called a feature vector. Features are extracted from segmented data and used to feed the classification module. TD features are considered due to their computational simplicity, as demonstrated in the literature for myoelectric classification systems [3]. The TD features used are mean of the absolute value (MAV), root mean square (RMS), variance (V), zero crossing (ZC), waveform length (WL), Willison amplitude (WA), energy (E), and average power (AP), as described and detailed in [12,19,27]. Each feature is numbered with its corresponding muscle associated with a channel. Features were labeled as MAV1, MAV2,…, MAV5, RMS1, RMS2, and so on.

In Fig. 11.11, the RMS plot of normalized data for the flexor carpi ulnaris muscle is shown (subject F1). Notice the reduction of dimensionality obtained; the x-axis denotes time segments, where each segment represents a window of 255 ms.

Fig. 11.11 Normalized RMS1 data for F1.

Marker values are obtained as peaks in the feature vector and used to label the data according to Table 11.2. In Fig. 11.12 those markers are illustrated as peaks where Di spheres were grasped.

Fig. 11.12 RMS1 marker values for F1.

11.3.4.3 Classification

Four popular classifiers are considered: linear discriminant analysis (LDA) [1], decision trees (DTs) [6], support vector machine with Gaussian kernel (SVM-GK), and neural networks (NNs). Using raw sEMG signals and eight TD statistics, a feature vector was determined for muscles and linking all together to arrange a 40-dimensional feature vector, which feed the classifiers.

Decision Trees

An accuracy up to 91% was reached with DT. However, the classifier was unable to predict sphere 5 (B5). Predicted values are low for most of the spheres, except for sphere 2. The false discovery rate is high. Fig. 11.13 displays the confusion matrix.

Fig. 11.13 DT confusion matrix.

Linear Discriminant Analysis

Similar problems as with the DT occur with the LDA classifier; some spheres are not predicted at all and the false discovery rate is high, as can be seen in Fig. 11.14.

Fig. 11.14 LDA confusion matrix.

Support Vector Machine With Gaussian Kernel

This classifier's performance is the most convenient for this application, as can be seen in Fig. 11.15. The confusion matrix shows high predicted rates, reaching an accuracy of 97%.

Fig. 11.15 SVM-GK confusion matrix.

11.3.5 Control of Biomechatronic System

11.3.5.1 Human Hand Biomechanics

The ROM that the human hand can naturally perform is limited by several constraints, which can be grouped into three categories: static, intrafinger, and interfinger [7]. In this example only intrafinger constraints are considered because they are related to movement of flexion/extension of the hand. The advances presented in this study are restricted to the thumb tracking motion.

Thumb Biomechanics

The thumb is constituted by three bones: distal phalanx, proximal phalanx, and metacarpal, forming an open kinematic chain with three rotational joints: the interphalangeal (PIP), metacarpophalangeal (MP), and trapeziometacarpal (TMC) joints. This articulated system is shown in Fig. 11.16.

Fig. 11.16 Joints and phalanges of the thumb.

Without loss of generality, let us consider four positions to study the motion of the thumb (TP). Let us define the set

T P = { O H , D 1 , D 2 , D 3 } ,

(11.18)

where OH and Di for i=1,..,3 are the open hand, grasp sphere 1, grasp sphere 2, and grasp sphere 3 tasks, which restrict the hand at determined positions, as can be seen in Fig. 11.17.

Fig. 11.17 Four motion positions for fingers. In (A) the hand is in the OH position. In (B)–(D) the hand is in D₁ to D₃, respectively. The x-axis is considered in the direction of the thumb when the palm is completely extended.

Starting from OH, it is possible to design a motion path for the thumb angle positions while the hand grasps spheres of different diameters. Finger positions are defined in terms of their joint angles θ as defined in Table 11.1. Two joint angles θ for the thumb were chosen, so we have

θ t h u m b = { θ P I P , θ M P } .

(11.19)

The positions defined in (11.18) introduce a dynamic in the kinematics of (11.19). A mapping that relates (11.18) and (11.19) is proposed as

O H = { θ P I P 0 , θ M P 0 } , D 1 = { θ P I P 1 , θ M P 1 } , D 2 = { θ P I P 2 , θ M P 2 } , D 3 = { θ P I P 03 , θ M P 3 } .

(11.20)

The possible states of the thumb (TP*) are built with (11.20) and defined as

T P ⁎ = [ O H D 1 D 2 D 3 ] = [ θ P I P 0 θ M P 0 θ P I P 1 θ M P 1 θ P I P 2 θ M P 2 θ P I P 03 θ M P 3 ] .

(11.21)

The rows of (11.21) can be used to design a trajectory path for the thumb considering the mechanical variables position, speed, and acceleration, as described in the problem formulation section for angular joints. In this case, a polynomial path that is able to track angular motion of TP* is designed and applied to track thumb motions via EMG signals.

11.3.5.2 Path Planning

Angular positions trained by the classifier are used to synthesize a desired reference. There are two methodologies to generate paths in robotics: polynomial and nonpolynomial methods. The polynomial approach is selected for its simplicity [13]. Let us consider a state machine with two discrete states θPIP(k)(t0) and θPIP(k+1)(tf), which are selected from (11.21). Then a parametric cubic polynomial path is defined as

θ I P = a 0 + a 1 t + a 2 t 2 + a 3 t 3 .

(11.22)

Moreover, for any discrete point in (11.19), and considering (11.21), a polynomial function for two desired angular positions is represented as

θ t h u m b ( k ) ( t ) = [ 1 t k t k 2 t k 3 0 1 2 t k 3 t k 2 1 t k + 1 t k + 1 2 t k + 1 3 0 1 2 t k + 1 3 t k + 1 2 ] .

(11.23)

11.3.5.3 Neural Controller

A multilayer feedforward neural network with time delay is trained to track the desired trajectories. The neural network can approximate a function whose parameters are the initial and final points of (11.21). The neural network is trained to learn trajectories defined by a reference model which is constituted by the polynomials described by (11.23) and parametrized with four constraints

θ i ( t 0 ) = θ P I P 0 , θ ˙ i ( t 0 ) = d d t θ P I P 0 , θ i ( t f ) = θ M P f , θ ˙ i ( t f ) = d d t θ M P f ,

where θi is defined for each joint of the thumb according to (11.21); t0, tf are the initial and final time of trajectories. The output yn of the artificial neural network is applied to a model predictive control (MPC) architecture, with the purpose of minimizing a cost function J, defined as

J = ∑ i = N 1 N 2 [ y m ( n + i ) − y n ( n + i ) ] 2 + ∑ i = 1 N u α ( k ) [ Δ u ( n + k ) ] 2 ,

(11.24)

where N1, N2, and Nu are the costing horizons over which the error of the output and inputs are evaluated; ym is the output of the reference model, and α is the contribution that the sum of the squares of the control has over J. The MPC approach is well known in the literature; the foundations were established in [21,20], where artificial neural networks were implemented as the plant nonlinear model.

The control scheme proposed is depicted in Fig. 11.18. A mismatch error function is added to guarantee that only the required positions are passed to the path planning module.

Fig. 11.18 Control scheme proposed.

The mismatch function is expressed as

M = { 0 , i f T = R , 1 , i f T ≠ R ,

(11.25)

where T is vector generated by the classifier with correct prediction and R is the response vector generated with testing data. If M=1, a biofeedback loop [3] is activated to indicate to the subject that a mismatch is found and it must repeat steps 1 to 7, as described by the experimental set-up.

11.3.5.4 Hardware Implementation

The hardware set-up is shown in Fig. 11.19. It consists of a Trigno Wireless System, as described in the Experimental preparation section. Data are processed by Matlab, where the classifier, previously trained, is selected and the control algorithm is implemented. The output of the processing system is connected with a servo system, which integrates BLDC motors with a driver system with a microcontroller-based interface to receive control commands from Matlab.

Fig. 11.19 Hardware implementation.

11.3.5.5 Results

The SVM classifier is selected due to it demonstrated better performance. The BLDC motor is identified offline with a neural architecture constituted by 10 neurons in the hiding layer, two delayed plant inputs, and two delayed plant outputs with a sampling interval of 0.8 sec. Training data are constituted by 800 samples. Fig. 11.20 shows the input/output pairs. The MPC is configured with α=0.005, N2=8, and Nu=2; the minimization routine was based on the backtracking algorithm. Fig. 11.21 shows the velocity tracking profile applied to the MP joint (metacarpal link) of the thumb for subject F1 while performing a OH to Di task. The speed profile corresponds to a difference of one degree, which is shown in Fig. 11.21, i.e., the metacarpal bone motion considered corresponds to a motion angle variation from 1 to 2.1 degrees. As can be seen in Fig. 11.22, the servo is able to track the position with good performance.

Fig. 11.20 Input/output pairs used to train the artificial neural network.

Fig. 11.21 Tracking a velocity profile.

Fig. 11.22 Position of the MP joint.

11.4 Conclusion

The methodology proposed in this work can be applied to control electromechanical systems using superficial EMG signals. Moreover, the methodology proposed implements a new approach to study the motion of the fingers, which exhibits very small variations, which can be established by means of grasping spheres with diameters of predefined sizes. To the best of our knowledge, there is currently no methodology available that studies finger motions employing sEMG and considers small changes in the same movement.

References

1. A.H. Al-Timemy, G. Bugmann, J. Escudero, N. Outram, Classification of finger movements for the dexterous hand prosthesis control with surface electromyography, IEEE Journal of Biomedical and Health Informatics 2013;17(3):608–618.

2. A.Y. Alanis, J.D. Rios, J. Rivera, N. Arana-Daniel, C. Lopez-Franco, Real-time discrete neural control applied to a linear induction motor, Neurocomputing 2015;164:240–251.

3. O.M. Asghari, H. Hu, Myoelectric control systems—a survey, Biomedical Signal Processing and Control 2007;2(4):275–294.

4. V.H. Benitez, E.N. Sanchez, A.G. Loukianov, Neural identification and control for linear induction motors, Journal of Intelligent and Fuzzy Systems 2005;16(1).

5. V.H. Benitez, E.N. Sanchez, A.G. Loukianov, Decentralized adaptive recurrent neural control structure, Engineering Applications of Artificial Intelligence 2007;20(8):1125–1132.

6. L. Breiman, Classification and Regression Trees. Chapman & Hall; 1993.

7. F. Chen Chen, S. Appendino, A. Battezzato, A. Favetto, M. Mousavi, F. Pescarmona, Constraint study for a hand exoskeleton: human hand kinematics and dynamics, Journal of Robotics 2013:1–17.

8. J.N. Chiasson, Modeling and High Performance Control of Electric Machines. 1st ed. John Wiley & Sons; 2005.

9. K. Englehart, B. Hudgin, P.A. Parker, A wavelet-based continuous classification scheme for multifunction myoelectric control, IEEE Transactions on Biomedical Engineering 2001;48(3):302–311.

10. K. Englehart, B. Hudgins, A robust, real-time control scheme for multifunction myoelectric control, IEEE Transactions on Biomedical Engineering 2003;50(7):848–854.

11. S. Gupta, S.K. Chowdhary, Authentication through electrocardiogram signals based on emotions-a step towards atm security, 2017 6th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO). 2017:440–442.

12. B. Hudgins, P. Parker, R. Scott, A new strategy for multifunction myoelectric control, IEEE Transactions on Biomedical Engineering 1993;40(1):82–94.

13. R.N. Jazar, Theory of Applied Robotics: Kinematics, Dynamics, and Control. Springer; 2010.

14. Z. Ju, H. Liu, Human hand motion analysis with multisensory information, IEEE/ASME Transactions on Mechatronics 2014;19(2):456–466.

15. Khamid, A.D. Wibawa, S. Sumpeno, Gesture recognition for Indonesian sign language systems (ISLS) using multimodal sensor leap motion and Myo armband controllers based-on Naïve Bayes classifier, 2017 International Conference on Soft Computing, Intelligent System and Information Technology (ICSIIT). 2017:1–6.

16. H. Liu, K. Wu, P. Meusel, N. Seitz, G. Hirzinger, M.H. Jin, Y.W. Liu, S.W. Fan, T. Lan, Z.P. Chen, Multisensory five-finger dexterous hand: the DLR/HIT Hand II, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems. 2008:3692–3697.

17. N. Malesevic, D. Markovic, G. Kanitz, M. Controzzi, C. Cipriani, C. Antfolk, Decoding of individual finger movements from surface EMG signals using vector autoregressive hierarchical hidden Markov models (VARHHMM), 2017 International Conference on Rehabilitation Robotics (ICORR). 2017:1518–1523.

18. T. Mullen, C.A. Kothe, Y.M. Chi, A. Ojeda, T. Kerth, S. Makeig, T.-P. Jung, G. Cauwenberghs, Real-time neuroimaging and cognitive monitoring using wearable dry EEG, IEEE Transactions on Biomedical Engineering 2015;62(11):2553–2567.

19. C. Sapsanis, G. Georgoulas, A. Tzes, D. Lymberopoulos, Improving EMG based classification of basic hand movements using EMD, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 2013:5754–5757.

20. D. Soloway, P.J. Haley, Neural generalized predictive control, Proceedings of the 1996 IEEE International Symposium on Intelligent Control. 1996:277–282.

21. P.H. Sørensen, M. Nørgaard, O. Ravn, N.K. Poulsen, Implementation of neural network based non-linear predictive control, Neurocomputing 1999;28(1–3):37–51.

22. F.V. Tenore, A. Ramos, A. Fahmy, S. Acharya, R. Etienne-Cummings, N.V. Thakor, Decoding of individuated finger movements using surface electromyography, IEEE Transactions on Biomedical Engineering 2009;56(5):1427–1434.

23. L. Wang, S. Du, H. Liu, J. Yu, S. Cheng, P. Xie, A virtual rehabilitation system based on EEG-EMG feedback control, 2017 Chinese Automation Congress (CAC). 2017:4337–4340.

24. H. Witte, M. Fremerey, S. Weyrich, J. Mampel, L. Fischheiter, D. Voges, K. Zimmermann, C. Schilling, Biomechatronics is not just biomimetics, 9th International Workshop on Robot Motion and Control. 2013:74–79.

25. K. Xing, P. Yang, J. Huang, Y. Wang, Q. Zhu, A real-time EMG pattern recognition method for virtual myoelectric hand control, Neurocomputing 2014;136:345–355.

26. K. Xu, W. Guo, L. Hua, X. Sheng, X. Zhu, A prosthetic arm based on EMG pattern recognition, 2016 IEEE International Conference on Robotics and Biomimetics, ROBIO 2016. 2016.

27. M. Zecca, S. Micera, M. Carrozza, P. Dario, Control of multifunctional prosthetic hands by processing the electromyographic signal, Critical Reviews in Biomedical Engineering 2002;30(4–6):459–485.