AI Features
Log In
0% completed
All Lessons
Free Lessons (5)
Before We Start
About the Course
Starting Off with BERT
Introduction: BERT
A Primer on Transformers
Introduction to the Transformer
Understanding the Encoder of the Transformer
Self-Attention Mechanism
Understanding the Self-Attention Mechanism
Multi-Head Attention Mechanism
Learning Position with Positional Encoding
Encoder: Feedforward and Add and Norm Component
Putting All the Encoder Components Together
Understanding the Decoder of the Transformer
Masked Multi-Head Attention
Multi-Head Attention
Decoder: Feedforward and Add and Norm Component
Putting All the Decoder Components Together
Putting the Encoder and Decoder Together
Summary: A Primer on Transformers
Exercise: A Primer on Transformers
Exercise: Decoder Architecture
Quiz: A Primer on Transformers
Project
Semantic Search with Transformers
Understanding the BERT Model
Introduction to the BERT Model
Working of BERT
Configurations of BERT
Pre-Training the BERT Model
The WordPiece Tokenizer
Pre-Training Strategies for the BERT Model
Masked Language Modeling
Next Sentence Prediction
Pre-Training Procedure
Subword Tokenization Algorithms
Byte Pair Encoding
Byte-Level Byte Pair Encoding and WordPiece Algorithms
Summary: Understanding the BERT Model
Quiz: Understanding the BERT Model
Getting Hands-On with BERT
Exploring the Pre-Trained BERT Model
Transformers
Extracting Embeddings From All Encoder Layers of BERT
Fine-Tuning BERT for Downstream Tasks
Sentiment Analysis
Natural Language Inference
Question-Answering
Performing Question-Answering with the Fine-Tuned BERT
Named Entity Recognition
Summary: Getting Hands-On with BERT
Quiz: Getting Hands-On with BERT
Exploring BERT Variants
Introducion: BERT Variants
Different BERT Variants
ALBERT
ALBERT : Training the Model
ALBERT : Embeddings Extraction
RoBERTa
RoBERTa Tokenizer
ELECTRA
Generator and Discriminator of the ELECTRA Model
Training the ELECTRA Model
SpanBERT
SpanBERT: Exploring Architecture
Summary: Different BERT Variants
Quiz: Different BERT Variants
BERT Variants—Based on Knowledge Distillation
Knowledge Distillation
Training the Student Network
DistilBERT
Training the Student BERT (DistilBERT)
TinyBERT
Teacher-Student Architecture
Distillation of Transformer Layer
Distillation of Embedding and Prediction Layer
Distillation Techniques for Pre-training and Fine-tuning
Data Augmentation
Transferring Knowledge from BERT to Neural Networks
The Data Augmentation Methods
Summary: BERT Variants—Based on Knowledge Distillation
Quiz: BERT Variants—Based on Knowledge Distillation
Applications of BERT
Introduction: Applications of BERT
Exploring BERTSUM for Text Summarization
Text Summarization
Fine-Tuning BERT for Extractive Summarization
BERTSUM for Extractive Summary
BERTSUM for Abstractive Summarization
ROUGE Evaluation Metrics
Training the BERTSUM Model
Summary: Exploring BERTSUM for Text Summarization
Quiz: Exploring BERTSUM for Text Summarization
Applying BERT to Other Languages
Understanding Multilingual BERT
Evaluating M-BERT on the NLI task
How Multilingual is Multilingual BERT?
M-BERT Generalization
Effect of Code-Switching and Transliteration
The Cross-Lingual Language Model (XLM)
The XLM-R Model
Language-Specific BERT
BETO for Spanish
BERTje for Dutch
German BERT
Chinese BERT
Japanese BERT
FinBERT for Finnish
BERT Models for Italian and Portuguese
RuBERT for Russian
Summary: Applying BERT to Other Languages
Quiz: Applying BERT to Other Languages
Exploring Sentence and Domain-Specific BERT
Sentence-BERT
Sentence-BERT with a Siamese Network
Sentence-BERT with a Triplet Network
Use Cases of Sentence-BERT Model
Learning Multilingual Embeddings Through Knowledge Distillation
Multilingual Sentence-BERT Model
Domain-Specific BERT
BioBERT
Summary: Exploring Sentence and Domain-Specific BERT
Quiz: Exploring Sentence and Domain-Specific BERT
Working with VideoBERT, BART, and More
VideoBERT Model
Pre-training Dataset and Applications of VideoBERT
BART Model
Noising Techniques
Performing Text Summarization with BART
Exploring BERT Libraries
Building a Document Answering Model
Document Summarization
Summary: Working with VideoBERT, BART, and More
Quiz: Working with VideoBERT, BART, and More
Conclusion
What We've Learned
Project
Similarity Detection in English Language Using RoBERTa
Home
Courses
Getting Started with Google BERT
What We've Learned
Let's wrap up this course!
We'll cover the following...
Feedback
You’ve reached the end
...
Ask