Tag: gensim

BBC News classification algorithm comparison

Keras, Machine Learning, Neural networks, NLP, Scikit-Learn, TheanoNo Comments

Fork BBC News dataset (available for download in Insight Project Resources website) is made up of 2225 newslines classified into 5 categories (Politics, Sport, Entertainment, Tech, Business) and, similarly to Reuters-21578, it can be adopted in order to test both the efficacy and the efficiency of different classification strategies. In the repository: https://github.com/giuseppebonaccorso/bbc_news_classification_comparison, I’ve committed a Jupyter (IPython) notebook (based on Scikit-Learn, NLTK, Gensim, Keras (with Theano or Tensorflow)) where I’ve collected some experiments aimed at comparing four different algorithms: Multinomial Naive-Bayes with Count (TF) vectorizer Multinomial Naive-Bayes with TF-IDF vectorizer SVM (linear and kernelized) with Doc2Vec (Gensim-based) vectorization MLP (Keras-based) with Doc2Vec vectorization (Every experiment has been performed on tokens without stop-words and processed with WordNet lemmatization). As expected (thanks to several research projects – see references for further information), Naive-Bayes performs even better than the other strategies, in particular comparing its simplicity and naiveness with the complexity of SVM and neural […]

Reuters-21578 text classification with Gensim and Keras

Artificial Intelligence, Deep Learning, Generic, Keras, Machine Learning, Neural networks, NLP, Python2 Comments

Fork Reuters-21578 is a collection of about 20K news-lines (see reference for more information, downloads and copyright notice), structured using SGML and categorized with 672 labels. They are diveded into five main categories: Topics Places People Organizations Exchanges However, most of them are unused and, looking at the distribution, it’s possible to notice a complete lack of homogeneity. These are the 20 top categories (the prefix is made with the two initial letter of each main category) with the number of related news-lines: ID Name Category Newslines 161 pl_usa Places 12542 533 to_earn Topics 3987 498 to_acq Topics 2448 158 pl_uk Places 1489 84 pl_japan Places 1138 31 pl_canada Places 1104 571 to_money-fx Topics 801 526 to_crude Topics 634 543 to_grain Topics 628 167 pl_west-germany Places 567 624 to_trade Topics 552 553 to_interest Topics 513 56 pl_france Places 469 185 or_ec Organizations 349 23 pl_brazil Places 332 628 to_wheat Topics […]