Topic modeling is about finding themes (topics) within clusters of textual documents. The topics are labels (keywords) that capture the meaning of the cluster.

BERTopic is a modular topic modeling technique that extract topic representations. BERTopic uses two steps to extract topics: text clustering and representation topics. The text clustering step provides BERTopic with the clusters of the documents that are semantically similar.

Example (to simplify the example, we choose one-word sentences):
Cluster 1: ['cats', 'dogs', 'elephants', 'birds'] ==> topic: animals Cluster 2: ['cars', 'trains', 'planes'] ==> topic: car
See this page for more details about BERTopic:
https://maartengr.github.io/BERTopic/index.html

We will create topics from a set of words.

Install the required modules:
$ pip install bertopic
Python code:
$ vi topic_modeling.py from sentence_transformers import SentenceTransformer from umap import UMAP from hdbscan import HDBSCAN from bertopic import BERTopic sentences = ['cats', 'dogs', 'elephants', 'birds', 'cars', 'trains', 'planes'] model = SentenceTransformer("sentence-transformers/all-MiniLM-L12-v2") embeddings = model.encode(sentences) topic_model = BERTopic( embedding_model=model, umap_model=UMAP(n_components=5, random_state=42), hdbscan_model=HDBSCAN(min_cluster_size=2), verbose=True ).fit(sentences, embeddings) print("Topics info:") print(topic_model.get_topic_info()) print("Topic 0 info:") print(topic_model.get_topic(0)) print("Topic 1 info:") print(topic_model.get_topic(1)) # save the barchart figure fig = topic_model.visualize_barchart() fig.write_html("bertopic-barchart-figure.html")
Run the Python script:
$ python topic_modeling.py
Output:
Topics info: Topic Count Name Representation Representative_Docs 0 0 4 0_cats_birds_elephants_dogs [cats, birds, elephants, dogs, , , , , , ] [birds, cats, dogs] 1 1 3 1_cars_trains_planes_ [cars, trains, planes, , , , , , , ] [planes, cars, trains] Topic 0 info: [ ('cats', np.float64(0.34657359027997264)), ('birds', np.float64(0.34657359027997264)), ('elephants', np.float64(0.34657359027997264)), ('dogs', np.float64(0.34657359027997264)), ('', 1e-05), ('', 1e-05), ('', 1e-05), ('', 1e-05), ('', 1e-05), ('', 1e-05) ] Topic 1 info: [ ('cars', np.float64(0.46209812037329684)), ('trains', np.float64(0.46209812037329684)), ('planes', np.float64(0.46209812037329684)), ('', 1e-05), ('', 1e-05), ('', 1e-05), ('', 1e-05), ('', 1e-05), ('', 1e-05), ('', 1e-05) ]
Topics are represented by the main keywords in the text. Their names are composed of these keywords concatenated with the underscore character ("_"). A specific topic with the tag "-1" can be listed and should include all keywords that do not match a specific topic. This topic may also include outliers which are candidates that do not match any of the found topics.

Chart of the topics (Topic Word Scores): bertopic-barchart-figure.html

BERTopic can use a model to generate proper labels for the topics.

We will create labeled topics from a set of words.

For that we need to craft a prompt that should have two parts:

• A subset of documents that best represent the topics that will inserted using the [DOCUMENTS] tag.
  
• The keywords that make up the topics of the cluster that will be inserted using the [KEYWORDS] tag.
  

INPUT + subset of documents + data list of documents: [DOCUMENTS] list of keywords: [KEYWORDS] predict the label of the topic. OUTPUT: <labeled topics>
Python code:
$ vi topic_modeling.py from sentence_transformers import SentenceTransformer from umap import UMAP from hdbscan import HDBSCAN from bertopic import BERTopic from transformers import pipeline from bertopic.representation import TextGeneration sentences = ['cats', 'dogs', 'elephants', 'birds', 'cars', 'trains', 'planes'] # Generate topics model = SentenceTransformer("sentence-transformers/all-MiniLM-L12-v2") embeddings = model.encode(sentences) topic_model = BERTopic( embedding_model=model, umap_model=UMAP(n_components=5, random_state=42), hdbscan_model=HDBSCAN(min_cluster_size=2), verbose=True ).fit(sentences, embeddings) # use a model ("google/flan-t5-small") to label the topics prompt = """These documents belong to the same topic: [DOCUMENTS] These keywords give details about the topic: '[KEYWORDS]'. Given these documents and keywords, what is this topic about?""" generator = pipeline("text2text-generation", model="google/flan-t5-small") representation_model = TextGeneration( generator, prompt=prompt, doc_length=50, tokenizer="whitespace" ) topic_model.update_topics(sentences, representation_model=representation_model) print(topic_model.get_topic_info())
Run the Python script:
$ python topic_modeling.py
Output:
Topic Count Name Representation Representative_Docs 0 0 4 0_animals___ [animals, , , , , , , , , ] [birds, cats, dogs] 1 1 3 1_car___ [car, , , , , , , , , ] [planes, cars, trains]