I am trying to perform in python the cosine similarity between two words which are in a dataset of texts (each text represents a tweet). I want to evaluate the similarity based on the context where they are placed. I have set a code like the following: The result is the similarity between the texts but I want…
Tag: cosine-similarity
How to count word similarity between two pandas dataframe
Here’s my first dataframe df1 Here’s my second dataframe df2 Similarity Matrix, columns is Id from df1, rows is Id from df2 Note: 0 value in (1,1) and (3,2) because no text similar 1 value in (3,1) is because of Bersatu and Kita’ (Id 1ondf2is avalilable in Id3ondf1` 0.33 is counted because o…
Sort dictionary python by value (word2vec)
I want to sort my dict by value, but if I apply this code it doesn’t work (it print only my key-value pairs without any kind of sorting). If I change key=lambda x: x[1] to x[0] it correctly sort by key, so I don’t understand what I’m doing wrong. My code: Answer You’re trying to sort s…
how to compare two text document with tfidf vectorizer?
I have two different text which I want to compare using tfidf vectorization. What I am doing is: tokenizing each document vectorizing using TFIDFVectorizer.fit_transform(tokens_list) Now the vectors that I get after step 2 are of different shape. But as per the concept, we should have the same shape for both …
What’s the fastest way in Python to calculate cosine similarity given sparse matrix data?
Given a sparse matrix listing, what’s the best way to calculate the cosine similarity between each of the columns (or rows) in the matrix? I would rather not iterate n-choose-two times. Say the input matrix is: The sparse representation is: In Python, it’s straightforward to work with the matrix-i…