20.
Generative Adversarial Networks
search
Quick search
code
Show Source
Preview Version
PyTorch
MXNet
Notebooks
Courses
GitHub
中文版
Table Of Contents
Preface
Installation
Notation
1. Introduction
2. Preliminaries
2.1. Data Manipulation
2.2. Data Preprocessing
2.3. Linear Algebra
2.4. Calculus
2.5. Automatic Differentiation
2.6. Probability and Statistics
2.7. Documentation
3. Linear Neural Networks for Regression
3.1. Linear Regression
3.2. Object-Oriented Design for Implementation
3.3. Synthetic Regression Data
3.4. Linear Regression Implementation from Scratch
3.5. Concise Implementation of Linear Regression
3.6. Generalization
3.7. Weight Decay
4. Linear Neural Networks for Classification
4.1. Softmax Regression
4.2. The Image Classification Dataset
4.3. The Base Classification Model
4.4. Softmax Regression Implementation from Scratch
4.5. Concise Implementation of Softmax Regression
4.6. Generalization in Classification
4.7. Environment and Distribution Shift
5. Multilayer Perceptrons
5.1. Multilayer Perceptrons
5.2. Implementation of Multilayer Perceptrons
5.3. Forward Propagation, Backward Propagation, and Computational Graphs
5.4. Numerical Stability and Initialization
5.5. Generalization in Deep Learning
5.6. Dropout
5.7. Predicting House Prices on Kaggle
6. Builders’ Guide
6.1. Layers and Modules
6.2. Parameter Management
6.3. Parameter Initialization
6.4. Lazy Initialization
6.5. Custom Layers
6.6. File I/O
6.7. GPUs
7. Convolutional Neural Networks
7.1. From Fully Connected Layers to Convolutions
7.2. Convolutions for Images
7.3. Padding and Stride
7.4. Multiple Input and Multiple Output Channels
7.5. Pooling
7.6. Convolutional Neural Networks (LeNet)
8. Modern Convolutional Neural Networks
8.1. Deep Convolutional Neural Networks (AlexNet)
8.2. Networks Using Blocks (VGG)
8.3. Network in Network (NiN)
8.4. Multi-Branch Networks (GoogLeNet)
8.5. Batch Normalization
8.6. Residual Networks (ResNet) and ResNeXt
8.7. Densely Connected Networks (DenseNet)
8.8. Designing Convolution Network Architectures
9. Recurrent Neural Networks
9.1. Working with Sequences
9.2. Converting Raw Text into Sequence Data
9.3. Language Models
9.4. Recurrent Neural Networks
9.5. Recurrent Neural Network Implementation from Scratch
9.6. Concise Implementation of Recurrent Neural Networks
9.7. Backpropagation Through Time
10. Modern Recurrent Neural Networks
10.1. Long Short-Term Memory (LSTM)
10.2. Gated Recurrent Units (GRU)
10.3. Deep Recurrent Neural Networks
10.4. Bidirectional Recurrent Neural Networks
10.5. Machine Translation and the Dataset
10.6. The Encoder-Decoder Architecture
10.7. Encoder-Decoder Seq2Seq for Machine Translation
10.8. Beam Search
11. Attention Mechanisms and Transformers
11.1. Queries, Keys, and Values
11.2. Attention Pooling by Similarity
11.3. Attention Scoring Functions
11.4. The Bahdanau Attention Mechanism
11.5. Multi-Head Attention
11.6. Self-Attention and Positional Encoding
11.7. The Transformer Architecture
11.8. Transformers for Vision
11.9. Large-Scale Pretraining with Transformers
12. Optimization Algorithms
12.1. Optimization and Deep Learning
12.2. Convexity
12.3. Gradient Descent
12.4. Stochastic Gradient Descent
12.5. Minibatch Stochastic Gradient Descent
12.6. Momentum
12.7. Adagrad
12.8. RMSProp
12.9. Adadelta
12.10. Adam
12.11. Learning Rate Scheduling
13. Computational Performance
13.1. Compilers and Interpreters
13.2. Asynchronous Computation
13.3. Automatic Parallelism
13.4. Hardware
13.5. Training on Multiple GPUs
13.6. Concise Implementation for Multiple GPUs
13.7. Parameter Servers
14. Computer Vision
14.1. Image Augmentation
14.2. Fine-Tuning
14.3. Object Detection and Bounding Boxes
14.4. Anchor Boxes
14.5. Multiscale Object Detection
14.6. The Object Detection Dataset
14.7. Single Shot Multibox Detection
14.8. Region-based CNNs (R-CNNs)
14.9. Semantic Segmentation and the Dataset
14.10. Transposed Convolution
14.11. Fully Convolutional Networks
14.12. Neural Style Transfer
14.13. Image Classification (CIFAR-10) on Kaggle
14.14. Dog Breed Identification (ImageNet Dogs) on Kaggle
15. Natural Language Processing: Pretraining
15.1. Word Embedding (word2vec)
15.2. Approximate Training
15.3. The Dataset for Pretraining Word Embeddings
15.4. Pretraining word2vec
15.5. Word Embedding with Global Vectors (GloVe)
15.6. Subword Embedding
15.7. Word Similarity and Analogy
15.8. Bidirectional Encoder Representations from Transformers (BERT)
15.9. The Dataset for Pretraining BERT
15.10. Pretraining BERT
16. Natural Language Processing: Applications
16.1. Sentiment Analysis and the Dataset
16.2. Sentiment Analysis: Using Recurrent Neural Networks
16.3. Sentiment Analysis: Using Convolutional Neural Networks
16.4. Natural Language Inference and the Dataset
16.5. Natural Language Inference: Using Attention
16.6. Fine-Tuning BERT for Sequence-Level and Token-Level Applications
16.7. Natural Language Inference: Fine-Tuning BERT
17. Reinforcement Learning
17.1. Markov Decision Process (MDP)
17.2. Value Iteration
17.3. Q-Learning
18. Gaussian Processes
18.1. Introduction to Gaussian Processes
18.2. Gaussian Process Priors
18.3. Gaussian Process Inference
19. Hyperparameter Optimization
19.1. What Is Hyperparameter Optimization?
19.2. Hyperparameter Optimization API
19.3. Asynchronous Random Search
19.4. Multi-Fidelity Hyperparameter Optimization
19.5. Asynchronous Successive Halving
20. Generative Adversarial Networks
20.1. Generative Adversarial Networks
20.2. Deep Convolutional Generative Adversarial Networks
21. Recommender Systems
21.1. Overview of Recommender Systems
21.2. The MovieLens Dataset
21.3. Matrix Factorization
21.4. AutoRec: Rating Prediction with Autoencoders
21.5. Personalized Ranking for Recommender Systems
21.6. Neural Collaborative Filtering for Personalized Ranking
21.7. Sequence-Aware Recommender Systems
21.8. Feature-Rich Recommender Systems
21.9. Factorization Machines
21.10. Deep Factorization Machines
22. Appendix: Mathematics for Deep Learning
22.1. Geometry and Linear Algebraic Operations
22.2. Eigendecompositions
22.3. Single Variable Calculus
22.4. Multivariable Calculus
22.5. Integral Calculus
22.6. Random Variables
22.7. Maximum Likelihood
22.8. Distributions
22.9. Naive Bayes
22.10. Statistics
22.11. Information Theory
23. Appendix: Tools for Deep Learning
23.1. Using Jupyter Notebooks
23.2. Using Amazon SageMaker
23.3. Using AWS EC2 Instances
23.4. Using Google Colab
23.5. Selecting Servers and GPUs
23.6. Contributing to This Book
23.7. Utility Functions and Classes
23.8. The
d2l
API Document
References
Table Of Contents
Preface
Installation
Notation
1. Introduction
2. Preliminaries
2.1. Data Manipulation
2.2. Data Preprocessing
2.3. Linear Algebra
2.4. Calculus
2.5. Automatic Differentiation
2.6. Probability and Statistics
2.7. Documentation
3. Linear Neural Networks for Regression
3.1. Linear Regression
3.2. Object-Oriented Design for Implementation
3.3. Synthetic Regression Data
3.4. Linear Regression Implementation from Scratch
3.5. Concise Implementation of Linear Regression
3.6. Generalization
3.7. Weight Decay
4. Linear Neural Networks for Classification
4.1. Softmax Regression
4.2. The Image Classification Dataset
4.3. The Base Classification Model
4.4. Softmax Regression Implementation from Scratch
4.5. Concise Implementation of Softmax Regression
4.6. Generalization in Classification
4.7. Environment and Distribution Shift
5. Multilayer Perceptrons
5.1. Multilayer Perceptrons
5.2. Implementation of Multilayer Perceptrons
5.3. Forward Propagation, Backward Propagation, and Computational Graphs
5.4. Numerical Stability and Initialization
5.5. Generalization in Deep Learning
5.6. Dropout
5.7. Predicting House Prices on Kaggle
6. Builders’ Guide
6.1. Layers and Modules
6.2. Parameter Management
6.3. Parameter Initialization
6.4. Lazy Initialization
6.5. Custom Layers
6.6. File I/O
6.7. GPUs
7. Convolutional Neural Networks
7.1. From Fully Connected Layers to Convolutions
7.2. Convolutions for Images
7.3. Padding and Stride
7.4. Multiple Input and Multiple Output Channels
7.5. Pooling
7.6. Convolutional Neural Networks (LeNet)
8. Modern Convolutional Neural Networks
8.1. Deep Convolutional Neural Networks (AlexNet)
8.2. Networks Using Blocks (VGG)
8.3. Network in Network (NiN)
8.4. Multi-Branch Networks (GoogLeNet)
8.5. Batch Normalization
8.6. Residual Networks (ResNet) and ResNeXt
8.7. Densely Connected Networks (DenseNet)
8.8. Designing Convolution Network Architectures
9. Recurrent Neural Networks
9.1. Working with Sequences
9.2. Converting Raw Text into Sequence Data
9.3. Language Models
9.4. Recurrent Neural Networks
9.5. Recurrent Neural Network Implementation from Scratch
9.6. Concise Implementation of Recurrent Neural Networks
9.7. Backpropagation Through Time
10. Modern Recurrent Neural Networks
10.1. Long Short-Term Memory (LSTM)
10.2. Gated Recurrent Units (GRU)
10.3. Deep Recurrent Neural Networks
10.4. Bidirectional Recurrent Neural Networks
10.5. Machine Translation and the Dataset
10.6. The Encoder-Decoder Architecture
10.7. Encoder-Decoder Seq2Seq for Machine Translation
10.8. Beam Search
11. Attention Mechanisms and Transformers
11.1. Queries, Keys, and Values
11.2. Attention Pooling by Similarity
11.3. Attention Scoring Functions
11.4. The Bahdanau Attention Mechanism
11.5. Multi-Head Attention
11.6. Self-Attention and Positional Encoding
11.7. The Transformer Architecture
11.8. Transformers for Vision
11.9. Large-Scale Pretraining with Transformers
12. Optimization Algorithms
12.1. Optimization and Deep Learning
12.2. Convexity
12.3. Gradient Descent
12.4. Stochastic Gradient Descent
12.5. Minibatch Stochastic Gradient Descent
12.6. Momentum
12.7. Adagrad
12.8. RMSProp
12.9. Adadelta
12.10. Adam
12.11. Learning Rate Scheduling
13. Computational Performance
13.1. Compilers and Interpreters
13.2. Asynchronous Computation
13.3. Automatic Parallelism
13.4. Hardware
13.5. Training on Multiple GPUs
13.6. Concise Implementation for Multiple GPUs
13.7. Parameter Servers
14. Computer Vision
14.1. Image Augmentation
14.2. Fine-Tuning
14.3. Object Detection and Bounding Boxes
14.4. Anchor Boxes
14.5. Multiscale Object Detection
14.6. The Object Detection Dataset
14.7. Single Shot Multibox Detection
14.8. Region-based CNNs (R-CNNs)
14.9. Semantic Segmentation and the Dataset
14.10. Transposed Convolution
14.11. Fully Convolutional Networks
14.12. Neural Style Transfer
14.13. Image Classification (CIFAR-10) on Kaggle
14.14. Dog Breed Identification (ImageNet Dogs) on Kaggle
15. Natural Language Processing: Pretraining
15.1. Word Embedding (word2vec)
15.2. Approximate Training
15.3. The Dataset for Pretraining Word Embeddings
15.4. Pretraining word2vec
15.5. Word Embedding with Global Vectors (GloVe)
15.6. Subword Embedding
15.7. Word Similarity and Analogy
15.8. Bidirectional Encoder Representations from Transformers (BERT)
15.9. The Dataset for Pretraining BERT
15.10. Pretraining BERT
16. Natural Language Processing: Applications
16.1. Sentiment Analysis and the Dataset
16.2. Sentiment Analysis: Using Recurrent Neural Networks
16.3. Sentiment Analysis: Using Convolutional Neural Networks
16.4. Natural Language Inference and the Dataset
16.5. Natural Language Inference: Using Attention
16.6. Fine-Tuning BERT for Sequence-Level and Token-Level Applications
16.7. Natural Language Inference: Fine-Tuning BERT
17. Reinforcement Learning
17.1. Markov Decision Process (MDP)
17.2. Value Iteration
17.3. Q-Learning
18. Gaussian Processes
18.1. Introduction to Gaussian Processes
18.2. Gaussian Process Priors
18.3. Gaussian Process Inference
19. Hyperparameter Optimization
19.1. What Is Hyperparameter Optimization?
19.2. Hyperparameter Optimization API
19.3. Asynchronous Random Search
19.4. Multi-Fidelity Hyperparameter Optimization
19.5. Asynchronous Successive Halving
20. Generative Adversarial Networks
20.1. Generative Adversarial Networks
20.2. Deep Convolutional Generative Adversarial Networks
21. Recommender Systems
21.1. Overview of Recommender Systems
21.2. The MovieLens Dataset
21.3. Matrix Factorization
21.4. AutoRec: Rating Prediction with Autoencoders
21.5. Personalized Ranking for Recommender Systems
21.6. Neural Collaborative Filtering for Personalized Ranking
21.7. Sequence-Aware Recommender Systems
21.8. Feature-Rich Recommender Systems
21.9. Factorization Machines
21.10. Deep Factorization Machines
22. Appendix: Mathematics for Deep Learning
22.1. Geometry and Linear Algebraic Operations
22.2. Eigendecompositions
22.3. Single Variable Calculus
22.4. Multivariable Calculus
22.5. Integral Calculus
22.6. Random Variables
22.7. Maximum Likelihood
22.8. Distributions
22.9. Naive Bayes
22.10. Statistics
22.11. Information Theory
23. Appendix: Tools for Deep Learning
23.1. Using Jupyter Notebooks
23.2. Using Amazon SageMaker
23.3. Using AWS EC2 Instances
23.4. Using Google Colab
23.5. Selecting Servers and GPUs
23.6. Contributing to This Book
23.7. Utility Functions and Classes
23.8. The
d2l
API Document
References
20.
Generative Adversarial Networks
¶
20.1. Generative Adversarial Networks
20.1.1. Generate Some “Real” Data
20.1.2. Generator
20.1.3. Discriminator
20.1.4. Training
20.1.5. Summary
20.1.6. Exercises
20.2. Deep Convolutional Generative Adversarial Networks
20.2.1. The Pokemon Dataset
20.2.2. The Generator
20.2.3. Discriminator
20.2.4. Training
20.2.5. Summary
20.2.6. Exercises
Previous
19.5. Asynchronous Successive Halving
Next
20.1. Generative Adversarial Networks
