BERT (language model)

Use the mouse wheel or two fingers (on touchscreens) to zoom in and out of the map.

Export map as:

Research this topic

Explore the main themes, entities and connections around BERT (language model). Start with the topic map, then use the sections below for research and deeper semantic analysis.

Explore this topic

Start with a few of the strongest sections from the source topic. These are research directions, not a list of keywords you must use.

History

Semi-supervised sequence learning Unsupervised Text corpus Word2vec GloVe

7 related topics

Architecture

Embedding Euclidean space Self-attention Question answering Sentiment classification

9 related topics

Training

Part-of-speech tagging Fine-tuned Natural language inference Text classification State-of-the-art

9 related topics

Overview

Language model Google Self-supervised learning Encoder-only transformer Large language models

15 related topics

Key facts & relationships

High-confidence facts extracted from structured source data. Use them as anchors for further research.

License: Apache 2.0
Original author: Google AI
Release: October 31, 2018
Repository: github.com/google-research/bert
Type: Large language model · Transformer · Foundation model

Topics to explore

Browse the full topic structure. Each item opens a new analysis centered on that subject.

Interpretation

Vector representations Vector space model
Encoder-only Transformer (deep learning architecture)
Be prompted Prompt engineering
Generate text Natural language generation

History

Semi-supervised sequence learning Semi-supervised learning
Unsupervised Unsupervised learning
Text corpus
Word2vec
GloVe GloVe (machine learning)
English-language
Google Search

Variants

Mini-batch Stochastic gradient descent
Distills Knowledge distillation
Generative adversarial networks Generative adversarial network

Advanced semantic analysis

Deeper signals for content research, entity SEO and topical coverage. The plain-language headings explain what each technical view is useful for.

Map overview Semantic statistics

Nodes54

Edges53

Triples14

Avg. degree1.96

Density0.037037

Components1

How this topic connects Entity context

See the strongest relationship patterns around the current topic before diving into the raw triples.

BERT (language model)

Top relations

Type · 3

BERT (language model) → Foundation model, Large language model, Transformer

License · 1

BERT (language model) → Apache 2.0

Original author · 1

BERT (language model) → Google AI

Release · 1

BERT (language model) → October 31, 2018

Repository · 1

BERT (language model) → github.com/google-research/bert

Website · 1

BERT (language model) → arxiv.org/abs/1810.04805

Important terminology Word statistics

Use these terms to understand the vocabulary surrounding the topic, not as a checklist for keyword stuffing.

Important terminology

bert language token model tokens embedding sentence training layer text two vector models task representation one transformer sentences sequence parameters

Entity relationships Subject–Predicate–Object triples

Subject	Predicate	Object	Confidence	Src
BERT (language model)	License	Apache 2.0	1.00	infobox
BERT (language model)	Original author	Google AI	1.00	infobox
BERT (language model)	Release	October 31, 2018	1.00	infobox
BERT (language model)	Repository	github.com/google-research/bert	1.00	infobox
BERT (language model)	Type	Large language model	1.00	infobox
BERT (language model)	Type	Transformer	1.00	infobox
BERT (language model)	Type	Foundation model	1.00	infobox
BERT (language model)	Website	arxiv.org/abs/1810.04805	1.00	infobox
natural language inference	instance of	BERT can be fine-tuned with fewer resources on smaller datasets to optimize its performance on specific tasks	0.80	text
text classification	instance of	BERT can be fine-tuned with fewer resources on smaller datasets to optimize its performance on specific tasks	0.80	text
and sequence-to-sequence-based language generation tasks such as question answering	instance of	BERT can be fine-tuned with fewer resources on smaller datasets to optimize its performance on specific tasks	0.80	text
conversational response generation.The original BERT paper published results demonstrating that a small amount of finetuning	instance of	BERT can be fine-tuned with fewer resources on smaller datasets to optimize its performance on specific tasks	0.80	text

Related concept clusters Concept neighborhoods

These clusters group vocabulary that occurs around closely connected concepts in the source material.

Connections between topic areas Semantic bridges

Bridges can reveal useful research angles that are easy to miss in a flat list of related terms.

Min side: 3