This is a project aimed at training and evaluating various classifiers on news articles collected from NU.nl, to then predict their popularity as expressed in number of comments.

crawling contains a script to collect articles from the news site, save them to a database, and update them with the number of comments they have received.
preprocessing contains a script for preprocessing all text in the collected articles.
learning contains scripts to transform the collected data into input for the classifiers, and a script to train and evaluate classifiers on the data.

Currently, when trained on a thousand articles, the multinomial Naive Bayes classifier can classify 50% of the articles correctly while the linear Support Vector Machine scores around 48%.

Some of the ideas for trying to improve classification performance are:

• Collecting more data
• Applying feature selection
• Investigating the effects of training the classifiers with different parameters

See the wiki.