What does the text talk about?
code-challenge machine-learning string note that the machine-learning tag will be new
META: this is far from being done. I also understand that this challenge depends heavily on manual opinion about the "type" of a piece of text. Hence, if you take issue with that, I would appreciate your giving a comment that suggests ways to fix that issue, rather than an unjustified downvote related to that issue.
Additionally, this might be a duplicate. I would appreciate your pointing this out before I compile the list of texts, if possible. However, if you identify a duplicate after I start compiling the list of texts, that is also fine.
Some parts use the future tense to talk about what I will do. Obviously I will have done them by the time I post the challenge.
The sections in italic could be taken as being ambiguous.
The links to the training, validation and test sets are not available yet. And of course, I don't yet have labels on my side.
This is a project that I once attempted to do, having learned machine learning. I run a forum app, and I was thinking of incorporating my new machine learning knowledge into that app by creating a model that could detect topics related to a given topic. After having worked on it for a few days, I hadn't made much progress, so I abandoned it. I hereby challenge you to make a similar model, ideally with machine learning, but that is not required. Your model will classify the topic of a piece of text. Such a model could then be used to find text related to a given piece of text by finding texts with a similar topic. You can choose to write your model without machine learning.
This is thus essentially a machine learning challenge. (or ideally, it will be. You may choose to write your solution without machine learning, but I am mainly looking forward to seeing machine learning solutions.) I will provide a large set of "articles", divided into a training set, a validation set, and a test set, with a 60-30-10 split. I expect there to be about 500 articles in all.
The articles in the training and validation sets are labeled with their topics: for instance, history, geography, mathematics, programming, etc. The test set, importantly, does not have public-facing labels, but I have labels on my side.
The training set is available here.
The validation set is available here.
The test set is available here.
Challenge
Write a classifier that attempts to classify the topic of a piece of text. The possible topics are:
(coming soon)
You can choose any of (coming soon) distinct values to represent the topic.
It should be able to produce an output that is one, and only one, of the chosen distinct values given any input string.
You have access to the train set to teach your classifier to recognize the topics (if you are using machine learning). The validation set can be used to compare different approaches.
Your submission will be scored based on how well it does on the test set. I will write the test set articles in such a way that they are not ambiguous (500 years ago, a mathematician discovered a method to calculate integrals
is ambiguous as the sentence could be about mathematics or history, but Learn about the way people lived 1000 years ago
is only about history). Your score is the number of articles it can correctly classify out of the test set. The higher the number of articles your submission can correctly classify, the better the score is. Thus the winner of this challenge is the submission that classifies the most articles correctly.
Importantly, this is not code-golf. I expect this to be a challenge that demands significant time and effort to produce a solution that scores highly, so you may post a link to a GitHub repository hosting the solution if required.
You are encouraged to either provide a way to easily run your solution, or provide the list of outputs that your code produces when given the test set articles. Even better, you could post a Jupyter notebook (if you are answering with a supported language) containing your solution, complete with test set outputs.
Important: please do not post a solution that is optimized only for the test set. It should work reasonably well in general.
Just so that you can get an idea of the topics:
Article: The dinosaurs went extinct 66 million years ago due to an asteroid impact.
Topic: history
Article: Time complexity is a measure of the complexity of an algorithm. For instance, the operation of adding two integers is usually taken to have a complexity of O(1). The operation of summing a list given as input has a complexity of O(n) where n is the size of the input.
Topic: programming
Article: Partial derivatives are derivatives taken with respect to one variable.
Topic: mathematics
Article: Planes are for going on holiday, especially island getaways.
Topic: holiday
Article: Carrot Cake Potato Mushroom
Topic: food