Tag: lda

How to handle numbers embedded in text during NLP pre-processing?

I am trying to run the LDA algorithm on a data set of news articles. I understand that numbers must be removed during the pre-processing step, and I have written a simple regex code to replace numbers with blanks. However, I would like to retain some numbers since removing them can potentially change the context/topic. For example, [Desired] ‘The fourth

How to fix LDA model coherence score runtime Error?

lda nlp python runtime-error topic-modeling

text=’Alice is a student.She likes studying.Teachers are giving a lot of homewok.’ I am trying to get topics from a simple text(like above) with coherance score.This is my LDA model: When i try to run this coherance model: I am supposed to get this king of output-> Coherence Score: 0.532947587081 I get this error: raise RuntimeError(”’ RuntimeError: An attempt has

Gensim LDA Coherence Score Nan

gensim lda machine-learning python topic-modeling

I created a Gensim LDA Model as shown in this tutorial: https://www.machinelearningplus.com/nlp/topic-modeling-gensim-python/ And it generates 10 topics with a log_perplexity of: lda_model.log_perplexity(data_df[‘bow_corpus’]) = -5.325966117835991 But when I run the coherence model on it to calculate coherence score, like so: My LDA-Score is nan. What am I doing wrong here? Answer Solved! Coherence Model requires the original text, instead of the

CalledProcessError: Returned non-zero exit status 1

gensim lda mallet python

When I try to run: I get the following error: What can I do in my code specifically to make it work? Furthermore, the question on this error has been asked a few times before. However, each answer seems so specific to a particular case, that I don’t see what I can change on my code now so that it