Computer Vision Metrics · Survey, Taxonomy, and Analysis by Krig, Scott -- Read -- Imperial Library of Trantor

Index

Cover Title Copyright About ApressOpen Dedication Contents at a Glance Contents About the Author Acknowledgments Introduction Chapter 1: Image Capture and Representation

Image Sensor Technology

Sensor Materials Sensor Photo-Diode Cells Sensor Configurations: Mosaic, Foveon, BSI Dynamic Range and Noise Sensor Processing De-Mosaicking Dead Pixel Correction Color and Lighting Corrections Geometric Corrections

Cameras and Computational Imaging

Overview of Computational Imaging Single-Pixel Computational Cameras 2D Computational Cameras 3D Depth Camera Systems

Binocular Stereo Structured and Coded Light Optical Coding: Diffraction Gratings Time-of-Flight Sensors Array Cameras Radial Cameras Plenoptics: Light Field Cameras

3D Depth Processing

Overview of Methods Problems in Depth Sensing and Processing

The Geometric Field and Distortions

The Horopter Region, Panum’s Area, and Depth Fusion Cartesian vs. Polar Coordinates: Spherical Projective Geometry Depth Granularity

Correspondence

Holes and Occlusion Surface Reconstruction and Fusion

Noise

Monocular Depth Processing

Multi-View Stereo Sparse Methods: PTAM Dense Methods: DTAM Optical Flow, SLAM, and SFM

3D Representations: Voxels, Depth Maps, Meshes, and Point Clouds Summary

Chapter 2: Image Pre-Processing

Perspectives on Image Processing Problems to Solve During Image Pre-Processing

Vision Pipelines and Image Pre-Processing Corrections Enhancements Preparing Images for Feature Extraction

Local Binary Family Pre-Processing Spectra Family Pre-Processing Basis Space Family Pre-Processing Polygon Shape Family Pre-Processing

The Taxonomy of Image Processing Methods

Point Line Area Algorithmic Data Conversions

Colorimetry

Overview of Color Management Systems Illuminants, White Point, Black Point, and Neutral Axis Device Color Models Color Spaces and Color Perception Gamut Mapping and Rendering Intent Practical Considerations for Color Enhancements Color Accuracy and Precision

Spatial Filtering

Convolutional Filtering and Detection Kernel Filtering and Shape Selection

Shape Selection or Forming Kernels

Point Filtering Noise and Artifact Filtering Integral Images and Box Filters

Edge Detectors

Kernel Sets: Sobel, Scharr, Prewitt, Roberts, Kirsch, Robinson, and Frei-Chen Canny Detector

Transform Filtering, Fourier, and Others

Fourier Transform Family

Fundamentals Fourier Family of Transforms Other Transforms

Morphology and Segmentation

Binary Morphology Gray Scale and Color Morphology Morphology Optimizations and Refinements Euclidean Distance Maps Super-Pixel Segmentation

Graph-based Super-Pixel Methods Gradient-Ascent-Based Super-Pixel Methods

Depth Segmentation Color Segmentation

Thresholding

Global Thresholding

Histogram Peaks and Valleys, and Hysteresis Thresholds LUT Transforms, Contrast Remapping Histogram Equalization and Specification Global Auto Thresholding

Local Thresholding

Local Histogram Equalization Integral Image Contrast Filters Local Auto Threshold Methods

Summary

Chapter 3: Global and Regional Features

Historical Survey of Features

Key Ideas: Global, Regional, and Local

1960s, 1970s, 1980s—Whole-Object Approaches Early 1990s—Partial-Object Approaches Mid-1990s—Local Feature Approaches Late 1990s—Classified Invariant Local Feature Approaches Early 2000s—Scene and Object Modeling Approaches Mid-2000s—Finer-Grain Feature and Metric Composition Approaches Post-2010—Multi-Modal Feature Metrics Fusion

Textural Analysis

1950s thru 1970s—Global Uniform Texture Metrics 1980s—Structural and Model-Based Approaches for Texture Classification 1990s—Optimizations and Refinements to Texture Metrics 2000 toToday—More Robust Invariant Texture Metrics and 3D Texture

Statistical Methods

Texture Region Metrics

Edge Metrics

Edge Density Edge Contrast Edge Entropy Edge Directivity Edge Linearity Edge Periodicity Edge Size Edge Primitive Length Total

Cross-Correlation and Auto-Correlation Fourier Spectrum, Wavelets, and Basis Signatures Co-Occurrence Matrix, Haralick Features

Extended SDM Metrics Metric 1: Centroid Metric 2: Total Coverage Metric 3: Low-Frequency Coverage Metric 4: Corrected Coverage Metric 5: Total Power Metric 6: Relative Power Metric 7: Locus Mean Density Metric 8: Locus Length Metric 9: Bin Mean Density Metric 10: Containment Metric 11. Linearity Metric 12: Linearity Strength

Laws Texture Metrics LBP Local Binary Patterns Dynamic Textures

Statistical Region Metrics

Image Moment Features Point Metric Features Global Histograms Local Region Histograms Scatter Diagrams, 3D Histograms Multi-Resolution, Multi-Scale Histograms Radial Histograms Contour or Edge Histograms

Basis Space Metrics

Fourier Description Walsh–Hadamard Transform HAAR Transform Slant Transform Zernike Polynomials Steerable Filters Karhunen-Loeve Transform and Hotelling Transform Wavelet Transform and Gabor Filters

Gabor Functions

Hough Transform and Radon Transform

Summary

Chapter 4: Local Feature Design Concepts, Classification, and Learning

Local Features

Detectors, Interest Points, Keypoints, Anchor Points, Landmarks Descriptors, Feature Description, Feature Extraction Sparse Local Pattern Methods

Local Feature Attributes

Choosing Feature Descriptors and Interest Points Feature Descriptors and Feature Matching Criteria for Goodness Repeatability, Easy vs. Hard to Find Distinctive vs. Indistinctive Relative and Absolute Position Matching Cost and Correspondence

Distance Functions

Early Work on Distance Functions Euclidean or Cartesian Distance Metrics

Euclidean Distance Squared Euclidean Distance Cosine Distance or Similarity Sum of Absolute Differences (SAD) or L1 Norm Sum of Squared Differences (SSD) or L2 Norm Correlation Distance Hellinger Distance

Grid Distance Metrics

Manhattan Distance Chebyshev Distance

Statistical Difference Metrics

Earth Movers Distance (EMD) or Wasserstein Metric Mahalanobis Distance Bray Curtis Distance Canberra Distance

Binary or Boolean Distance Metrics

L0 Norm Hamming Distance Jaccard Similarity and Dissimilarity

Descriptor Representation

Coordinate Spaces, Complex Spaces Cartesian Coordinates Polar and Log Polar Coordinates Radial Coordinates Spherical Coordinates Gauge Coordinates Multivariate Spaces, Multimodal Data Feature Pyramids

Descriptor Density

Interest Point and Descriptor Culling Dense vs. Sparse Feature Description

Descriptor Shape Topologies

Correlation Templates Patches and Shape

Single Patches, Sub-Patches Deformable Patches Multi-Patch Sets

TPLBP, FPLBP

Strip and Radial Fan Shapes

D-NETS Strip Patterns

Object Polygon Shapes

Morphological Boundary Shapes Texture Structure Shapes Super-Pixel Similarity Shapes

Local Binary Descriptor Point-Pair Patterns

FREAK Retinal Patterns Brisk Patterns ORB and BRIEF Patterns

Descriptor Discrimination

Spectra Discrimination Region, Shapes, and Pattern Discrimination Geometric Discrimination Factors Feature Visualization to Evaluate Discrimination

Discrimination via Image Reconstruction from HOG Discrimination via Image Reconstruction from Local Binary Patterns Discrimination via Image Reconstruction from SIFT Features

Accuracy, Trackability Accuracy Optimizations, Sub-Region Overlap, Gaussian Weighting, and Pooling Sub-Pixel Accuracy

Search Strategies and Optimizations

Dense Search Grid Search Multi-Scale Pyramid Search Scale Space and Image Pyramids Feature Pyramids Sparse Predictive Search and Tracking Tracking Region-Limited Search Segmentation Limited Search Depth or Z Limited Search

Computer Vision, Models, Organization

Feature Space Object Models Constraints Selection of Detectors and Features

Manually Designed Feature Detectors Statistically Designed Feature Detectors Learned Features

Overview of Training Classification of Features and Objects

Group Distance: Clustering, Training, and Statistical Learning Group Distance: Clustering Methods Survey, KNN, RANSAC, K-Means, GMM, SVM, Others Classification Frameworks, REIN, MOPED Kernel Machines Boosting, Weighting Selected Examples of Classification

Feature Learning, Sparse Coding, Convolutional Networks

Terminology: Codebooks, Visual Vocabulary, Bag of Words, Bag of Features Sparse Coding Visual Vocabularies Learned Detectors via Convolutional Filter Masks Convolutional Neural Networks, Neural Networks Deep Learning, Pooling, Trainable Feature Hierarchies

Summary

Chapter 5: Taxonomy of Feature Description Attributes

Feature Descriptor Families

Prior Work on Computer Vision Taxonomies Robustness and Accuracy

General Robustness Taxonomy

Illumination Color Criteria Incompleteness Resolution and Accuracy Geometric Distortion Efficiency Variables, Costs and Benefits Discrimination and Uniqueness

General Vision Metrics Taxonomy

Feature Descriptor Family Spectra Dimensions Spectra Type Interest Point Storage Formats Data Types Descriptor Memory Feature Shapes Feature Pattern Feature Density Feature Search Methods Pattern Pair Sampling Pattern Region Size Distance Function

Euclidean or Cartesian Distance Family Grid Distance Family Statistical Distance Family Binary or Boolean Distance Family

Feature Metric Evaluation

Efficiency Variables, Costs and Benefits Image Reconstruction Efficiency Metric Example Feature Metric Evaluations

SIFT Example

VISION METRIC TAXONOMY FME GENERAL ROBUSTNESS ATTRIBUTES

LBP Example

VISION METRIC TAXONOMY FME GENERAL ROBUSTNESS ATTRIBUTES

Shape Factors Example

VISION METRIC TAXONOMY FME GENERAL ROBUSTNESS ATTRIBUTES

Summary

Chapter 6: Interest Point Detector and Feature Descriptor Survey

Interest Point Tuning Interest Point Concepts Interest Point Method Survey

Laplacian and Laplacian of Gaussian Moravac Corner Detector Harris Methods, Harris-Stephens, Shi-Tomasi, and Hessian-Type Detectors Hessian Matrix Detector and Hessian-Laplace Difference of Gaussians Salient Regions SUSAN, and Trajkovic and Hedly Fast, Faster, AGHAST Local Curvature Methods Morphological Interest Regions

Feature Descriptor Survey

Local Binary Descriptors

Local Binary Patterns

Neighborhood Comparison Histogram Composition Optionally Normalization Descriptor Concatenation

Rotation Invariant LBP (RILBP) Dynamic Texture Metric Using 3D LBPs

Volume LBP (VLBP) LPB-TOP

Other LBP Variants Census Modified Census Transform BRIEF ORB BRISK FREAK

Spectra Descriptors

SIFT

Create a Scale Space Pyramid Identify Scale-Invariant Interest Points Create Feature Descriptors

SIFT-PCA SIFT-GLOH SIFT-SIFER Retrofit SIFT CS-LBP Retrofit RootSIFT Retrofit CenSurE and STAR Correlation Templates HAAR Features Viola Jones with HAAR-Like Features SURF Variations on SURF Histogram of Gradients (HOG) and Variants PHOG and Related Methods Daisy and O-Daisy CARD Robust Fast Feature Matching RIFF, CHOG Chain Code Histograms

D-NETS Local Gradient Pattern Local Phase Quantization

Basis Space Descriptors

Fourier Descriptors Other Basis Functions for Descriptor Building Sparse Coding Methods

Examples of Sparse Coding Methods

Polygon Shape Descriptors

MSER Method Object Shape Metrics for Blobs and Polygons Shape Context

3D, 4D, Volumetric, and Multimodal Descriptors

3D HOG HON 4D 3D SIFT

Summary

Chapter 7: Ground Truth Data, Content, Metrics, and Analysis

What Is Ground Truth Data? Previous Work on Ground Truth Data: Art vs. Science

General Measures of Quality Performance Measures of Algorithm Performance Rosin’s Work on Corners

Key Questions For Constructing Ground Truth Data

Content: Adopt, Modify, or Create Survey Of Available Ground Truth Data Fitting Data to Algorithms Scene Composition and Labeling

Composition Labeling

Defining the Goals and Expectations

Mikolajczyk and Schmid Methodology Open Rating Systems Corner Cases and Limits Interest Points and Features

Robustness Criteria for Ground Truth Data

Illustrated Robustness Criteria Using Robustness Criteria for Real Applications

Pairing Metrics with Ground Truth

Pairing and Tuning Interest Points, Features, and Ground Truth Examples Using The General Vision Taxonomy

Synthetic Feature Alphabets

Goals for the Synthetic Dataset

Accuracy of Feature Detection via Location Grid Rotational Invariance via Rotated Image Set Scale Invariance via Thickness and Bounding Box Size Noise and Blur Invariance Repeatabilty Real Image Overlays of Synthetic Features

Synthetic Interest Point Alphabet

Synthetic Corner Alphabet

Hybrid Synthetic Overlays on Real Images

Method for Creating the Overlays

Summary

Chapter 8: Vision Pipelines and Optimizations

Stages, Operations, and Resources Compute Resource Budgets

Compute Units, ALUs, and Accelerators Power Use Memory Use I/O Performance

The Vision Pipeline Examples

Automobile Recognition

Segmenting the Automobiles Matching the Paint Color Measuring the Automobile Size and Shape Feature Descriptors Calibration, Set-up, and Ground Truth Data Pipeline Stages and Operations Operations and Compute Resources Criteria for Resource Assignments

Face, Emotion, and Age Recognition

Calibration and Ground Truth Data Interest Point Position Prediction Segmenting the Head and Face Using the Bounding Box Face Landmark Identification and Compute Features Pipeline Stages and Operations Operations and Compute Resources Criteria for Resource Assignments

Image Classification

Segmenting Images and Feature Descriptors Pipeline Stages and Operations Mapping Operations to Resources Criteria for Resource Assignments

Augmented Reality

Calibration and Ground Truth Data Feature and Object Description Overlays and Tracking Pipeline Stages and Operations Mapping Operations to Resources Criteria for Resource Assignments

Acceleration Alternatives

Memory Optimizations

Minimizing Memory Transfers Between Compute Units Memory Tiling DMA, Data Copy, and Conversions Register Files, Memory Caching, and Pinning Data Structures, Packing, and Vector vs. Scatter-Gather Data Organization

Coarse-Grain Parallelism

Compute-Centric vs. Data-Centric Threads and Multiple Cores

Fine-Grain Data Parallelism

SIMD, SIMT, and SPMD Fundamentals Shader Kernel Languages and GPGPU

Advanced Instruction Sets and Accelerators

Vision Algorithm Optimizations and Tuning

Compiler And Manual Optimizations Tuning Feature Descriptor Retrofit, Detectors, Distance Functions Boxlets and Convolution Acceleration Data-Type Optimizations, Integer vs. Float

Optimization Resources Summary

Appendix A: Synthetic Feature Analysis

Background Goals and Expectations Test Methodology and Results

Detector Parameters Are Not Tuned for the Synthetic Alphabets Expectations for Test Results

Summary of Synthetic Alphabet Ground Truth Images

Synthetic Interest Point Alphabet Synthetic Corner Point Alphabet Synthetic Alphabet Overlays

Test 1: Synthetic Interest Point Alphabet Detection

Annotated Synthetic Interest Point Detector Results Entire Images Available Online

Test 2: Synthetic Corner Point Alphabet Detection

Annotated Synthetic Corner Point Detector Results Entire Images Available Online

Test 3: Synthetic Alphabets Overlaid on Real Images

Annotated Detector Results on Overlay Images

Test 4: Rotational Invariance for Each Alphabet

Methodology for Determining Rotational Invariance

Analysis of Results and Non-Repeatability Anomalies

Caveats Non-Repeatability in Tests 1 and 2 Other Non-Repeatability in Test 3 Test Summary Future Work

Appendix B: Survey of Ground Truth Datasets Appendix C: Imaging and Computer Vision Resources

Commercial Products Open Source Organizations, Institutions, and Standards Journals and Their Abbreviations Conferences and Their Abbreviations Online Resources

Appendix D: Extended SDM Metrics Bibliography Index

← Prev
Back
Next →

← Prev
Back
Next →