Neural Networks Math; A Visual Introduction for Beginners by Michael Taylor -- Read -- Imperial Library of Trantor

Log In

Or create an account ->

Imperial Library

Home
About
News
Upload
Forum

Help

Login/SignUp

Index

Table of Contents Part 1: The Math of Neural Networks Part 2: Additional Resources Neural Networks Math: A Visual Introduction for Beginners by Michael Taylor What You'll Find Inside: Don't Waste Your Time The Math of Neural Networks: Introduction Terminology and Notation Pre-Stage: Creating the Network Structure Types of Hyperparameters Input Node Hidden Layer Hidden Node Output Node Weight Value Bias Value Learning Rate Momentum Stage 1: Forward Propagation Part 1: Mathematical Functions What is a Summation Operator? Why is a Summation Operator Used ? What is an Activation Function? Types of Activation Functions Why is an Activation Function Used ? Logistic and Hyperbolic Activation Functions Part 2: Forward Propagation Using Matrices What is a Matrix ? How Are The Entries in a Matrix Referenced ? What Type of Data Does a Matrix Contain? How Are Matrices Used? Part 3: Fitting it All Together: Review Big Picture Stage 2: Calculate The Total Error Part 1: Forward Part 2: Mathematical Functions What is a Cost Function? Types of Cost Functions Why is a Cost Function Used ? Part 3: Fitting it All Together: Review Big Picture Stage 3: Calculate The Gradients Section 1: Mathematical Functions What is a Partial Derivative ? Why is a partial derivative used ? What is the chain rule ? Why is the chain rule used ? Section 2: Why Gradients Are Important What is a gradient? Gradient importance: in-depth explanation Section 3: How to Calculate Gradients Part 1: Calculating the Partial Derivative of Output Layer Weights Step 1: Discovering The Formula Step 2: Unpacking the Formula Step 3: Calculating The Partial Derivatives Derivative 1: Derivative 2: Derivative 3: Step 4: Compacting The Formula Part 2: Calculating the Partial Derivative of Output Layer Bias Weights It is Already Calculated Beware: Different Deltaz's! Part 3: Calculating the Partial Derivative Hidden Layer Weights Step 1: Discovering The Formula Step 2: Unpacking The Formula Step 3: Solving The Derivatives Derivative #1 Derivative #2 Derivative #3 Derivative #4 Step 4: Compacting The Formula Part 4: Calculating The Partial Derivative of Hidden Layer Bias Weights It is Already Calculated Beware: Different Deltab 's! Section 4: Fitting it All Together: Review Stage 4: Checking the Gradients Section 1: Understanding Numerical Estimation Section 2: Discovering The Formula Section 3: Calculating The Numerical Estimate Part 1: Step 1. Add Epsilon Step 2. Recalculate The Total Error Of The Network Part 2 Step 3. Subtract Epsilon Step 4. Recalculate The Total Error Of The Network Step 5: Calculate The Numerical Approximation Step 6: Measure Against the Analytic Gradient Step 7: Compute The Relative Error Section 4: Fitting It All Together: Review Stage 5: Updating the Weights Section 1: What is Gradient Descent? Section 2: Gradient Descent Methods Introduction Batch Gradient Descent (also called Full-Batch) Stochastic Gradient Descent (also called SGD / Online) Mini-Batch Gradient Descent Section 3: Updating Weights General Weight Update Equation Batch Training Weight Update Equation SGD Training Weight Update Equation Mini-Batch Training Weight Update Equation Section 4: Fitting it All Together: Review Constructing a Network: Hands-On Example Defining the Scenario: Man vs Chicken Pre-Stage: Network Structure Section 1: Determining Structural Elements Section 2: Understanding the Input Layer Section 3: Understanding the Output Layer Section 4: Simplifying our Network Section 5: Stage Review Stage 1: Running Data Through the Network Section 1: Moving From the Input Layer to the Hidden Layer Section 2: Moving From the Hidden Layer to the Output Layer Introduction Moving From Layer B to C Stage Review: Stage 2: Calculating the Total Error Calculating the Local Error of Node c1 Calculating the Local Error of Node c2 Stage Review: Stage 3: Calculating the Gradients Section 1: Calculating Gradients For Output Layer Weights Calculating the gradient with respect to w5 Calculating the gradient with respect to w6 Calculating the gradient with respect to w7 Calculating the gradient with respect to w8 Section 2: Calculating Gradients For Output Layer Bias Weights Section 3: Calculating Gradients For Hidden Layer Weights Calculating for W1: Part 1 ��z * w5 for output node c1. Calculating for W1: Parts 2 and 3 Calculating for W2: Part 1 Calculating for W2: Part 1 ��z * w7 for output node c1. Calculating for W2: Parts 2 and 3 Calculating for w3 Calculating for w4 Section 4: Calculating Gradients For Hidden Layer Bias Weights Section 5: All Gradients Stage 4: Gradient Checking Part 1 Step 1. Add Epsilon Step 2. Recalculate the Total Error of the network Part 2 Step 3. Subtract Epsilon Step 4. Recalculate the Total Error of the network Step 5: Calculate the Numerical Approximation Step 6: Measure Against the Analytic Gradient Step 7: Compute The Relative Error Stage 5: Updating Weights Updating General Weights Updating Bias Weights Wrapping it All up: Final Review Expanded Definitions for Neural Networks Terms That Describe the Same Function, Action or Object Terms General Definitions Learning Rate Schedule Bibliography

← Prev
Back
Next →

← Prev
Back
Next →

Chief Librarian: Las Zenow <zenow@riseup.net>
Fork the source code from gitlab.

This is a mirror of the Tor onion service:
http://kx5thpx2olielkihfyo4jgjqfb7zx7wxr3sd4xzt26ochei4m6f7tayd.onion