Liav Ariel (212830871)

Nehorai Josef (322774852)

Exercise 1 was mainly a preprocess task. There is not much content in it except the way the matrices are created, so I will briefly summarize what is being done and I will not go into too much detail. I will just note that his output was used by us to continue the tasks that I will detail later. You can see Task1/README.md for more details.

In Exercise 1 we received:

2 sets of TF-IDF matrices one for words and the other for die, each set with 3 matrices, A, B and C

2 sets of matrices of W2V one for words and the other for die, each set with 3 matrices, A, B and C

A set of one doc2vec matrix of the source files, this group has 3 matrices, A, B and C

A set of one BERT or UV matrix of the source files, this group has 3 matrices, A, B, and C

A total of 18 matrices.

In this exercise for all three matrices A, B and C in each of the groups you will form three pairs: (1) A and B, (2) A and C, (3) B and C, I will use different classifiers to form clusters.

At first, I used unsupervised learning (K-means, DBSCAN, Mixture of Gaussian) and then supervised learning (ANN, Naive Bayes, Logistic Regression).

This report will discuss how we work and the results.

I retrieved all groups of matrices to data structure of list of lists:

vectors_A = [v1, v2, ..., v500]

vectors_B = [v1, v2, ..., v500]

vectors_C = [v1, v2, ..., v500]

and then I create list of tuples, every tuple is a pair:

pairs = [(vectors_A, vectors_B), (vectors_A, vectors_C), (vectors_B, vectors_C)]

For each pair in pairs, I label group of pair[0] as 0, and group of pair[1] as 1.

I did a consolidation operation into one numpy list for runtime efficiency.

And I create another numpy array of labels, first 5000 values are 0 and the other are 1.

I use this for all the next classifiers.

Liav Ariel was Assigned to this task.

K-means get only the numpy array of vectors with n_clusters parameter of 2 and do his operation.

The result will list of labels, every index will represent the label that K-means choose for the vector in the same index.

Then I use the truth labels and the predicted labels for evaluate the results.

I can't know which label the classifier chose for each group, so I'm trying to evaluate the 2 options:

1. pair[0] as label 0 and pair[1] as label 1.
2. pair[0] as label 1 and pair[1] as label 0.

K-means on Bert On Source matrices

Results

For A & B groups:

Visualization using t-SNE:

For A & C groups:

Visualization using t-SNE:

For B & C groups:

Visualization using t-SNE:

K-means on D2V On Source matrices

Results

For A & B groups:

Visualization using t-SNE:

For A & C groups:

Visualization using t-SNE:

For B & C groups:

Visualization using t-SNE:

K-means on TFIDF On Lemmas matrices

Results

For A & B groups:

Visualization using t-SNE:

For A & C groups:

Visualization using t-SNE:

For B & C groups:

Visualization using t-SNE:

K-means on TFIDF On Words matrices

Results

For A & B groups:

Visualization using t-SNE:

For A & C groups:

Visualization using t-SNE:

For B & C groups:

Visualization using t-SNE:

K-means on W2V On Lemmas matrices

Results

For A & B groups:

Visualization using t-SNE:

For A & C groups:

Visualization using t-SNE:

For B & C groups:

Visualization using t-SNE:

K-means on W2V On Words matrices

Results

For A & B groups:

Visualization using t-SNE:

For A & C groups:

Visualization using t-SNE:

For B & C groups:

Visualization using t-SNE:

Liav Ariel was Assigned to this task.

DBSCAN also get only the numpy array of vectors with epsilon & min_samples parameters.

The result will list of labels, every index will represent the label that DBSCAN chose for the vector in the same index.

Then I use the truth labels and the predicted labels for evaluate the results.

I can't know which label the classifier chose for each group, so I'm trying to evaluate the 2 options:

1. pair[0] as label 0 and pair[1] as label 1.
2. pair[0] as label 1 and pair[1] as label 0.

Before going on the results, I will explain how I set the epsilon parameter:

For any pair in any type of matrices (Bert, D2V, TFIDF On Lemmas, TFIDF On Words, W2V On Lemmas, W2V On Words):

I plot a graph f(x) of K-Distance as a function of number of points.

Let see the results for any groups:

Epsilon parameter for on Bert On Source matrices

We can see that there is a knee on K-distance of 8.

So I set:

Bert epsilon = 8.

Epsilon parameter for on D2V On Source matrices

Let's add to the plot the graph for D2V On Source matrices:

We can see that the graphs are under Bert matrices and there is a knee on K-distance of 5.

So I set:

D2V epsilon = 5.

Epsilon parameter for on TFIDF

TFIDF On Lemmas matrices

Let's add to the plot the graph for TFIDF On Lemmas matrices:

TFIDF On Words matrices

Let's add to the plot the graph for TFIDF On Words matrices:

We can see that the graphs of the both groups are above the previous ones and there is a knee on K-distance of 30.

So I set:

TFIDF epsilon = 30.

Epsilon parameter for on W2V

W2V On Lemmas matrices

Let's add to the plot the graph for W2V On Lemmas matrices:

W2V On Words matrices

Let's add to the plot the graph for W2V On Words matrices:

We can see that the graphs of the both groups are extremely above the previous ones and there is a knee on K-distance of 450.

So I set:

W2V epsilon = 450.

I saved the parameters in: "./input/dbscan-eps.json":

{ "Bert_eps": 8, "D2V_eps": 5, "TFIDF_eps": 30, "W2V_eps": 450 }

Let see the results of DBSCAN Clustering:

DBSCAN on Bert On Source matrices

Results

For A & B groups:

Visualization using UMAP:

For A & C groups:

Visualization using UMAP:

For B & C groups:

Visualization using UMAP:

DBSCAN on D2V On Source matrices

Results

For A & B groups:

Visualization using UMAP:

For A & C groups:

Visualization using UMAP:

For B & C groups:

Visualization using UMAP:

DBSCAN on TFIDF On Lemmas matrices

Results

For A & B groups:

Visualization using UMAP:

For A & C groups:

Visualization using UMAP:

For B & C groups:

Visualization using UMAP:

DBSCAN on TFIDF On Words matrices

Results

For A & B groups:

Visualization using UMAP:

For A & C groups:

Visualization using UMAP:

For B & C groups:

Visualization using UMAP:

DBSCAN on W2V On Lemmas matrices

Results

For A & B groups:

Visualization using UMAP:

For A & C groups:

Visualization using UMAP:

For B & C groups:

Visualization using UMAP:

DBSCAN on W2V On Words matrices

Results

For A & B groups:

Visualization using UMAP:

For A & C groups:

Visualization using UMAP:

For B & C groups:

Visualization using UMAP:

Liav Ariel was Assigned to this task.

Mixture of Gaussian get only the numpy array of vectors with n_components=2 parameter of 2 and do his operation.

The result will list of labels, every index will represent the label that Mixture of Gaussian chose for the vector in the same index.

Then I use the truth labels and the predicted labels for evaluate the results.

I can't know which label the classifier chose for each group, so I'm trying to evaluate the 2 options:

1. pair[0] as label 0 and pair[1] as label 1.
2. pair[0] as label 1 and pair[1] as label 0.

Mixture of Gaussian on Bert On Source matrices

Results

For A & B groups:

Visualization using t-SNE:

For A & C groups:

Visualization using t-SNE:

For B & C groups:

Visualization using t-SNE:

Mixture of Gaussian on D2V On Source matrices

Results

For A & B groups:

Visualization using t-SNE:

For A & C groups:

Visualization using t-SNE:

For B & C groups:

Visualization using t-SNE:

Mixture of Gaussian on TFIDF On Lemmas matrices

Results

For A & B groups:

Visualization using t-SNE:

For A & C groups:

Visualization using t-SNE:

For B & C groups:

Visualization using t-SNE:

Mixture of Gaussian on TFIDF On Words matrices

Results

For A & B groups:

Visualization using t-SNE:

For A & C groups:

Visualization using t-SNE:

For B & C groups:

Visualization using t-SNE:

Mixture of Gaussian on W2V On Lemmas matrices

Results

For A & B groups:

Visualization using t-SNE:

For A & C groups:

Visualization using t-SNE:

For B & C groups:

Visualization using t-SNE:

Mixture of Gaussian on W2V On Words matrices

Results

For A & B groups:

Visualization using t-SNE:

For A & C groups:

Visualization using t-SNE:

For B & C groups:

Visualization using t-SNE:

Liav Ariel was Assigned to this task.

ANN get the numpy array of vectors and the list of labels suitable to any vector index.

I shuffled the both of arrays, for make the learning process randomly.

Then, I split the data into: train_vectors and test_vectors (20% I used for testing & 80% for learning).

I'm also split the 80% of learning data into: train_vectors, val_vectors (10% I used for validation & 90% for learning).

Parameters I used for the Neural Network:

1. EPOCHS = 15.
2. BATCH_SIZE = 128.
3. TOPOLOGY, there is two options:
   1. [(10, relu), (10, relu), (7, relu)]
   2. [(10, gelu), (10, gelu), (7, gelu)]
   • every pair in the list represent: (layer_size, activation_func) as a Dense Layer.

Callbacks I used for the Neural Network Fit:

1. early_stopping = EarlyStopping(monitor='val_accuracy', patience=5, restore_best_weights=True)
2. model_checkpoint = ModelCheckpoint(checkpoint_path, monitor='val_accuracy', save_best_only=True)

The result will list of labels, every index will represent the label that ANN chose for the vector in the same index.

Then I use the truth labels and the predicted labels for evaluate the results.

Let's see the results:

ANN on Bert On Source matrices

For A & B groups:

Training History:

Best model saved in:

• Task2/output/ANN/Bert_On_Source_Groups/best_model_A&B.h5

Results

Visualization using UMAP:

For A & C groups:

Training History:

Best model saved in:

• Task2/output/ANN/Bert_On_Source_Groups/best_model_A&C.h5

Results

Visualization using UMAP:

For B & C groups:

Training History:

Best model saved in:

• Task2/output/ANN/Bert_On_Source_Groups/best_model_B&C.h5

Results

Visualization using UMAP:

ANN on D2V On Source matrices

For A & B groups:

Training History:

Best model saved in:

• Task2/output/ANN/D2V_On_Source_Groups/best_model_A&B.h5

Results

Visualization using UMAP:

For A & C groups:

Training History:

Best model saved in:

• Task2/output/ANN/D2V_On_Source_Groups/best_model_A&C.h5

Results

Visualization using UMAP:

For B & C groups:

Training History:

Best model saved in:

• Task2/output/ANN/D2V_On_Source_Groups/best_model_B&C.h5

Results

Visualization using UMAP:

ANN on TFIDF On Lemmas matrices

For A & B groups:

Training History:

Best model saved in:

• Task2/output/ANN/TFIDF_On_Lemots_Groups/best_model_A&B.h5

Results

Visualization using UMAP:

For A & C groups:

Training History:

Best model saved in:

• Task2/output/ANN/TFIDF_On_Lemots_Groups/best_model_A&C.h5

Results

Visualization using UMAP:

For B & C groups:

Training History:

Best model saved in:

• Task2/output/ANN/TFIDF_On_Lemots_Groups/best_model_B&C.h5

Results

Visualization using UMAP:

ANN on TFIDF On Words matrices

For A & B groups:

Training History:

Best model saved in:

• Task2/output/ANN/TFIDF_On_Words_Groups/best_model_A&B.h5

Results

Visualization using UMAP:

For A & C groups:

Training History:

Best model saved in:

• Task2/output/ANN/TFIDF_On_Words_Groups/best_model_A&C.h5

Results

Visualization using UMAP:

For B & C groups:

Training History:

Best model saved in:

• Task2/output/ANN/TFIDF_On_Words_Groups/best_model_B&C.h5

Results

Visualization using UMAP:

ANN on W2V On Lemmas matrices

For A & B groups:

Training History:

Best model saved in:

• Task2/output/ANN/W2V_On_Lemots_Groups/best_model_A&B.h5

Results

Visualization using UMAP:

For A & C groups:

Training History:

Best model saved in:

• Task2/output/ANN/W2V_On_Lemots_Groups/best_model_A&C.h5

Results

Visualization using UMAP:

For B & C groups:

Training History:

Best model saved in:

• Task2/output/ANN/W2V_On_Lemots_Groups/best_model_B&C.h5

Results

Visualization using UMAP:

ANN on W2V On Words matrices

For A & B groups:

Training History:

Best model saved in:

• Task2/output/ANN/W2V_On_Words_Groups/best_model_A&B.h5

Results

Visualization using UMAP:

For A & C groups:

Training History:

Best model saved in:

• Task2/output/ANN/W2V_On_Words_Groups/best_model_A&C.h5

Results

Visualization using UMAP:

For B & C groups:

Training History:

Best model saved in:

• Task2/output/ANN/W2V_On_Words_Groups/best_model_B&C.h5

Results

Visualization using UMAP:

Liav Ariel was Assigned to this task.

Naive Bayes get the numpy array of vectors and the list of labels suitable to any vector index.

I shuffled the both of arrays, for make the learning process randomly.

Then, I split the data into: train_vectors and test_vectors (20% I used for testing & 80% for learning).

I'm also split the 80% of learning data into: train_vectors, val_vectors (10% I used for validation & 90% for learning).

The result will list of labels, every index will represent the label that Naive Bayes chose for the vector in the same index.

Then I use the truth labels and the predicted labels for evaluate the results.

Let's see the results:

Naive Bayes on Bert On Source matrices

For A & B groups:

Results

Visualization using UMAP:

For A & C groups:

Results

Visualization using UMAP:

For B & C groups:

Results

Visualization using UMAP:

Naive Bayes on D2V On Source matrices

For A & B groups:

Results

Visualization using UMAP:

For A & C groups:

Results

Visualization using UMAP:

For B & C groups:

Results

Visualization using UMAP:

Naive Bayes on TFIDF On Lemmas matrices

For A & B groups:

Results

Visualization using UMAP:

For A & C groups:

Results

Visualization using UMAP:

For B & C groups:

Results

Visualization using UMAP:

Naive Bayes on TFIDF On Words matrices

For A & B groups:

Results

Visualization using UMAP:

For A & C groups:

Results

Visualization using UMAP:

For B & C groups:

Results

Visualization using UMAP:

Naive Bayes on W2V On Lemmas matrices

For A & B groups:

Results

Visualization using UMAP:

For A & C groups:

Results

Visualization using UMAP:

For B & C groups:

Results

Visualization using UMAP:

Naive Bayes on W2V On Words matrices

For A & B groups:

Results

Visualization using UMAP:

For A & C groups:

Results

Visualization using UMAP:

For B & C groups:

Results

Visualization using UMAP:

![Alt Text](Task2/output/NB/W2V_On_Words_Groups/umap_B&C.png

Liav Ariel was Assigned to this task.

Logistic Regression get the numpy array of vectors and the list of labels suitable to any vector index.

I shuffled the both of arrays, for make the learning process randomly.

Then, I split the data into: train_vectors and test_vectors (20% I used for testing & 80% for learning).

I'm also split the 80% of learning data into: train_vectors, val_vectors (10% I used for validation & 90% for learning).

The result will list of labels, every index will represent the label that Logistic Regression chose for the vector in the same index.

Then I use the truth labels and the predicted labels for evaluate the results.

Let's see the results:

Logistic Regression on Bert On Source matrices

For A & B groups:

Results

Visualization using UMAP:

For A & C groups:

Results

Visualization using UMAP:

For B & C groups:

Results

Visualization using UMAP:

Logistic Regression on D2V On Source matrices

For A & B groups:

Results

Visualization using UMAP:

For A & C groups:

Results

Visualization using UMAP:

For B & C groups:

Results

Visualization using UMAP:

Logistic Regression on TFIDF On Lemmas matrices

For A & B groups:

Results

Visualization using UMAP:

For A & C groups:

Results

Visualization using UMAP:

For B & C groups:

Results

Visualization using UMAP:

Logistic Regression on TFIDF On Words matrices

For A & B groups:

Results

Visualization using UMAP:

For A & C groups:

Results

Visualization using UMAP:

For B & C groups:

Results

Visualization using UMAP:

Logistic Regression on W2V On Lemmas matrices

For A & B groups:

Results

Visualization using UMAP:

For A & C groups:

Results

Visualization using UMAP:

For B & C groups:

Results

Visualization using UMAP:

Logistic Regression on W2V On Words matrices

For A & B groups:

Results

Visualization using UMAP:

For A & C groups:

Results

Visualization using UMAP:

For B & C groups:

Results

Visualization using UMAP:

In this exercise, we return to work on the original files of 15000 documents (A, B and C).

We also get data of names, we will use it next for cleaning the documents from names.

We start the exercise with this data:

1. source data of 3 groups of 15000 documents split into 3 groups A, B and C:

   1. group A: './data/15000_docs/original_A.xlsx'
   2. group B: './data/15000_docs/original_B.xlsx'
   3. group C: './data/15000_docs/original_C.xlsx'

2. data of names:

   1. first names collection: './data/names/first-names.xlsx'
   2. last names collection: './data/names/last-names.xlsx'

We need first to preprocess the source data.

There is two things we should do.

First, translated the data to english for use 'vaderSentiment' model to tag every document as Positive / Negative / Neutral.

Second, we want to search if the names entity effect on the sentiment tagging, so we want to create a copy of the source data, but after we clean the names.

After prepare all the data we need.

We execute 'Sentiment' task on the translated data using 'vaderSentiment' model.

Then, we execute 'Sentiment' task using AlephBertGimmel on the source files and on the files without real names.

Finally, we execute 'Sentiment' task using heBert on the source files and on the files without real names.

Now, you can read about how we have done every thing step by step:

Translating Data

Nehorai Josef was Assigned to this task.

We worked on Google Colab in this task.

But we have grouped the main parts of the code into one Python file.

Script path:

1. ./translate_data.py

Output path:

1. './data/translated_15000_docs/translated_A.xlsx'
2. './data/translated_15000_docs/translated_B.xlsx'
3. './data/translated_15000_docs/translated_C.xlsx'

The vaderSentiment library does not work in Hebrew, so there is no choice but to translate it into English.

The translation will be done for us by the Google Translate Python API.

example of use:

from googletrans import Translator
translator = Translator()

he_sentence = translator.translate('איזה יום שמח לי היום')
print(he_sentence.text)

We have a problem with some documents, because their length was too big, the Google Translate Python API crashed.

For solving that, we saved all the problematics documents in different Excel file.

We, splitting the documents into small chunks, translating them, and finally we concat the part of documents to one document.

Clean Data From Names

Liav Ariel was Assigned to this task.

Script path:

1. './replace_names_using_ner.py'

Output path:

1. '/data/cleaned_from_names_15000_docs/without_names_A.xlsx'
2. '/data/cleaned_from_names_15000_docs/without_names_B.xlsx'
3. '/data/cleaned_from_names_15000_docs/without_names_C.xlsx'

First step was to find out which strings on the document represent names of persons.

I used dictaBert model that was pre-trained on NER (Name Entity Recognition) Task.

Here is a link for the model: https://huggingface.co/dicta-il/dictabert-ner

Sample usage:

from transformers import pipeline

oracle = pipeline('ner', model='dicta-il/dictabert-ner', aggregation_strategy='simple')

# if we set aggregation_strategy to simple, we need to define a decoder for the tokenizer. Note that the last wordpiece of a group will still be emitted
from tokenizers.decoders import WordPiece
oracle.tokenizer.backend_tokenizer.decoder = WordPiece()

sentence = '''דוד בן-גוריון (16 באוקטובר 1886 - ו' בכסלו תשל"ד) היה מדינאי ישראלי וראש הממשלה הראשון של מדינת ישראל.'''
oracle(sentence)

Output:

[
  {
    "entity_group": "PER",
    "score": 0.9999443,
    "word": "דוד בן - גוריון",
    "start": 0,
    "end": 13
  },
  {
    "entity_group": "TIMEX",
    "score": 0.99987966,
    "word": "16 באוקטובר 1886",
    "start": 15,
    "end": 31
  },
  {
    "entity_group": "TIMEX",
    "score": 0.9998579,
    "word": "ו' בכסלו תשל\"ד",
    "start": 34,
    "end": 48
  },
  {
    "entity_group": "TTL",
    "score": 0.99963045,
    "word": "וראש הממשלה",
    "start": 68,
    "end": 79
  },
  {
    "entity_group": "GPE",
    "score": 0.9997943,
    "word": "ישראל",
    "start": 96,
    "end": 101
  }
]

I write a script that scan all the source files, and find out all the strings that belongs to 'entity_group': 'PER'.

For each string from this group, I get random names from 'first-name' and 'last-name' data.

I finally, replace the real names with random name.

Sentiment On Translated Data Using 'vaderSentiment':

Liav Ariel was Assigned to this task.

Script path:

1. './tag_translated_data.py'

Output path:

1. './output/tagged_translated_15000_docs/tagged_translated_A.xlsx'
2. './output/tagged_translated_15000_docs/tagged_translated_B.xlsx'
3. './output/tagged_translated_15000_docs/tagged_translated_C.xlsx'
4. './output/tagged_translated_15000_docs/result_translated_A.json'
5. './output/tagged_translated_15000_docs/result_translated_B.json'
6. './output/tagged_translated_15000_docs/result_translated_C.json'

I write script that scan all the translated data.

For every document I do Sentiment Tagging using 'vaderSentiment'.

Link I used for learning this model:

1. https://reshetech.co.il/machine-learning-tutorials/sentiment-analysis-in-hebrew-kind-of

I create new Excel file that contains the same information as the translated data but with some new columns:

1. Sentiment: column of the decided label of tagging - 'Positive' / 'Negative' / 'Neutral'.
2. Compound: column of the total sentiment score of 'vaderSentiment'.
3. Neg: column of the score for Negative result by 'vaderSentiment'.
4. Neu: column of the score for Neutral result by 'vaderSentiment'.
5. Pos: column of the score for Positive result by 'vaderSentiment'.

And I also creat json file that summarize how much from each label I have in each group.

Json file structure: {'Positive': <amount_of_positives>, 'Negative': <amount_of_negatives>, 'Neutral': <amount_of_neutrals>}

How to import 'vaderSentiment':

from nltk.sentiment.vader import SentimentIntensityAnalyzer
import nltk
nltk.download('vader_lexicon')
sid = SentimentIntensityAnalyzer()

How to use 'vaderSentiment':

Code:

sentence = 'I happily got out of bed at half past ten'
sid.polarity_scores(sentence)

Output:

{'compound': 0.5574, 'neg': 0.0, 'neu': 0.69, 'pos': 0.31}

How I decided to tag each document:

I decided according to 'compound_score':

1. 'compound_score' > 0.5 ==> 'Positive'.
2. -0.5 < 'compound_score' < 0.5 ==> 'Neutral'.
3. 'compound_score' < -0.5 ==> 'Negative'.

Let's discuss the results:

Conclusion:

The results look good, but I would expect different results because I have prior knowledge of the data that is supposed to be more negative.

I think, my classification decision according to the 'compound_score' is incorrect.

Therefore, I decide to run another experience, but now I decide to tag every document according to this rules:

1. 'compound_score' > 1/3 ==> 'Positive'.
2. -1/3 < 'compound_score' < 1/3 ==> 'Neutral'.
3. 'compound_score' < -1/3 ==> 'Negative'.

I save the result output here:

1. './output/tagged_translated_15000_docs/tagged_translated_A_NEW.xlsx'
2. './output/tagged_translated_15000_docs/tagged_translated_B_NEW.xlsx'
3. './output/tagged_translated_15000_docs/tagged_translated_C_NEW.xlsx'
4. './output/tagged_translated_15000_docs/result_translated_A_NEW.json'
5. './output/tagged_translated_15000_docs/result_translated_B_NEW.json'
6. './output/tagged_translated_15000_docs/result_translated_C_NEW.json'

Let's discuss the results:

Conclusion:

It didn't improve by much, but it did improve a bit.

Maybe if we take the optimal thresholds we will get even better results.

Or maybe the premise that translation preserves semantics is wrong.

But according to these results, it seems that the percentage of positive texts is greater.

Sentiment Task Using 'heBert-Sentiment':

Nehorai Josef was Assigned to this task.

Script path:

1. './tag_with_heBert.py'

Output path:

1. './output/tagged_heBert_15000_docs/tagged_with_heBert_A.xlsx'
2. './output/tagged_heBert_15000_docs/tagged_with_heBert_B.xlsx'
3. './output/tagged_heBert_15000_docs/tagged_with_heBert_C.xlsx'
4. './output/tagged_heBert_15000_docs/result_heBert_A.json'
5. './output/tagged_heBert_15000_docs/result_heBert_B.json'
6. './output/tagged_heBert_15000_docs/result_heBert_C.json'
7. './output/tagged_heBert_15000_without_names_docs/tagged_heBert_without_names_A.xlsx'
8. './output/tagged_heBert_15000_without_names_docs/tagged_heBert_without_names_B.xlsx'
9. './output/tagged_heBert_15000_without_names_docs/tagged_heBert_without_names_C.xlsx'
10. './output/tagged_heBert_15000_without_names_docs/result_heBert_without_names_A.json'
11. './output/tagged_heBert_15000_without_names_docs/result_heBert_without_names_B.json'
12. './output/tagged_heBert_15000_without_names_docs/result_heBert_without_names_C.json'

I do the same as before, just now I work on hebrew data with heBert-Sentiment pre-trained model. First, on the source data. Second, on the data without names.

Link for the model:

1. https://huggingface.co/avichr/heBERT_sentiment_analysis

How to Use:

For sentiment classification model (polarity ONLY):

from transformers import AutoTokenizer, AutoModel, pipeline
tokenizer = AutoTokenizer.from_pretrained("avichr/heBERT_sentiment_analysis") #same as 'avichr/heBERT' tokenizer
model = AutoModel.from_pretrained("avichr/heBERT_sentiment_analysis")

# how to use?
sentiment_analysis = pipeline(
    "sentiment-analysis",
    model="avichr/heBERT_sentiment_analysis",
    tokenizer="avichr/heBERT_sentiment_analysis",
    return_all_scores = True
)

>>>  sentiment_analysis('אני מתלבט מה לאכול לארוחת צהריים')	
[[{'label': 'natural', 'score': 0.9978172183036804},
{'label': 'positive', 'score': 0.0014792329166084528},
{'label': 'negative', 'score': 0.0007035882445052266}]]

>>>  sentiment_analysis('קפה זה טעים')
[[{'label': 'natural', 'score': 0.00047328314394690096},
{'label': 'possitive', 'score': 0.9994067549705505},
{'label': 'negetive', 'score': 0.00011996887042187154}]]

>>>  sentiment_analysis('אני לא אוהב את העולם')
[[{'label': 'natural', 'score': 9.214012970915064e-05}, 
{'label': 'possitive', 'score': 8.876807987689972e-05}, 
{'label': 'negetive', 'score': 0.9998190999031067}]]

Let's discuss the results:

Sentiment on Source Data:

Sentiment on Data Without Real Names:

Conclusion:

We can see that ignoring real names does not affect the classification much.

In fact there is an effect of exactly 1.5-2 percent.

Another conclusion is that the texts are very negative for the 3 groups.

The conclusion this time matches my expectations based on familiarity with the texts.

Sentiment Task Using 'AlephBertGimmel-Sentiment':

Liav Ariel was Assigned to this task.

Script path:

1. './tag_with_ABG.py'

Output path:

1. './output/tagged_ABG_15000_docs/tagged_with_ABG_A.xlsx'
2. './output/tagged_ABG_15000_docs/tagged_with_ABG_B.xlsx'
3. './output/tagged_ABG_15000_docs/tagged_with_ABG_C.xlsx'
4. './output/tagged_ABG_15000_docs/result_ABG_A.json'
5. './output/tagged_ABG_15000_docs/result_ABG_B.json'
6. './output/tagged_ABG_15000_docs/result_ABG_C.json'
7. './output/tagged_ABG_15000_without_names_docs/tagged_ABG_without_names_A.xlsx'
8. './output/tagged_ABG_15000_without_names_docs/tagged_ABG_without_names_B.xlsx'
9. './output/tagged_ABG_15000_without_names_docs/tagged_ABG_without_names_C.xlsx'
10. './output/tagged_ABG_15000_without_names_docs/result_ABG_without_names_A.json'
11. './output/tagged_ABG_15000_without_names_docs/result_ABG_without_names_B.json'
12. './output/tagged_ABG_15000_without_names_docs/result_ABG_without_names_C.json'

I do the same as before, just now I work on hebrew data with AlephBertGimmel-Sentiment pre-trained model. First, on the source data. Second, on the data without names.

Link for the model:

1. https://huggingface.co/Perlucidus/alephbertgimmel-base-sentiment

How to Use:

from transformers import AutoTokenizer, AutoModelForSequenceClassification

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("Perlucidus/alephbertgimmel-base-sentiment")
model = AutoModelForSequenceClassification.from_pretrained("Perlucidus/alephbertgimmel-base-sentiment")

# Tokenize the text
text = "This is a great movie!"
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)

# Perform forward pass to get the logits
outputs = model(**inputs)

# Convert logits to probabilities
probs = outputs['logits'].softmax(dim=-1)

# Get the predicted sentiment label
predicted_label = probs.argmax().item()

# Define the mapping of sentiment labels
sentiment_labels = {0: 'Negative', 1: 'Neutral', 2: 'Positive'}

# Print the sentiment label
print(sentiment_labels[predicted_label])

Let's discuss the results:

Sentiment on Source Data:

Sentiment on Data Without Real Names:

Conclusion:

Also, there is more negative than positive.

NER affect in favor of the positive classification and increases it by 1.5-2 percent.