Dive into Deep Learning

Table Of Contents

2. The Preliminaries: A Crashcourse

To get started with deep learning, we will need to develop a few basic skills. All machine learning is concerned with extracting information from data. So we will begin by learning the practical skills for storing and manipulating data with Apache MXNet. Moreover machine learning typically requires working with large datasets, which we can think of as tables, where the rows correspond to examples and the columns correspond to attributes. Linear algebra gives us a powerful set of techniques for working with tabular data. We won’t go too far into the weeds but rather focus on the basic of matrix operations and their implementation in Apache MXNet. Additionally, deep learning is all about optimization. We have a model with some parameters and we want to find those that fit our data the best. Determining which way to move each parameter at each step of an algorithm requires a little bit of calculus. Fortunately, Apache MXNet’s autograd package covers this for us, and we will cover it next. Next, machine learning is concerned with making predictions: what is the likely value of some unknown attribute, given the information that we observe? To reason rigorously under uncertainty we will need to invoke the language of probability and statistics. To conclude the chapter, we will present your first basic classifier, Naive Bayes.

• 2.1. Data Manipulation
  • 2.1.1. Getting Started
  • 2.1.2. Operations
  • 2.1.3. Broadcast Mechanism
  • 2.1.4. Indexing and Slicing
  • 2.1.5. Saving Memory
  • 2.1.6. Mutual Transformation of NDArray and NumPy
  • 2.1.7. Exercises
  • 2.1.8. Scan the QR Code to Discuss
• 2.2. Linear Algebra
  • 2.2.1. Scalars
  • 2.2.2. Vectors
  • 2.2.3. Length, dimensionality and shape
  • 2.2.4. Matrices
  • 2.2.5. Tensors
  • 2.2.6. Basic properties of tensor arithmetic
  • 2.2.7. Sums and means
  • 2.2.8. Dot products
  • 2.2.9. Matrix-vector products
  • 2.2.10. Matrix-matrix multiplication
  • 2.2.11. Norms
  • 2.2.12. Norms and objectives
  • 2.2.13. Intermediate linear algebra
  • 2.2.14. Summary
  • 2.2.15. Scan the QR Code to Discuss
• 2.3. Automatic Differentiation
  • 2.3.1. A Simple Example
  • 2.3.2. Backward for Non-scalar Variable
  • 2.3.3. Detach Computations
  • 2.3.4. Attach Gradients to Internal Variables
  • 2.3.5. Head gradients
  • 2.3.6. Computing the Gradient of Python Control Flow
  • 2.3.7. Training Mode and Prediction Mode
  • 2.3.8. Summary
  • 2.3.9. Exercises
  • 2.3.10. Scan the QR Code to Discuss
• 2.4. Probability and Statistics
  • 2.4.1. Basic probability theory
  • 2.4.2. Dealing with multiple random variables
  • 2.4.3. Conditional independence
  • 2.4.4. Sampling
  • 2.4.5. Summary
  • 2.4.6. Exercises
  • 2.4.7. Scan the QR Code to Discuss
• 2.5. Naive Bayes Classification
  • 2.5.1. Optical Character Recognition
  • 2.5.2. The Probabilistic Model for Classification
  • 2.5.3. The Naive Bayes Classifier
  • 2.5.4. Training
  • 2.5.5. Summary
  • 2.5.6. Exercises
  • 2.5.7. Scan the QR Code to Discuss
• 2.6. Documentation
  • 2.6.1. Finding all the functions and classes in the module
  • 2.6.2. Finding the usage of specific functions and classes
  • 2.6.3. API Documentation
  • 2.6.4. Exercise
  • 2.6.5. Scan the QR Code to Discuss

Previous

1. Introduction

Next

2.1. Data Manipulation