Make me a k-NN classifier

code-golf machine-learning The "machine-learning" tag will be created

The k Nearest Neighbors (k-NN) classifier is a simple machine learning classifier. Although k-NN also works for regression tasks, we will be focusing on classification tasks for this challenge.

The classifier can be customized in a number of different ways. For this challenge, the following can be customized:

• k itself;
• the distance metric; and
• the aggregation function.

Main idea

As you can probably tell by its name, k-NN works using the concept of neighbors. First, we "train" classifiers by giving it (preferably a lot of) training data.

A training data point consists of two parts: its "features" and its target value. For instance, if predicting car brand, features might be maximum speed, size etc, and the target variable would be brand. We can also specify k, a distance metric, and/or an aggregation function, or let the classifier fine-tune these hyperparameters itself. For this challenge, the above hyperparameters will be given directly to the classifier.

When we give the classifier a list of features to be assigned a predicted target value, it takes the k train points closest to the test point, measured by distance metric via features, and runs their target values through its aggregation function to obtain its prediction.

The hyperparameters are explained below:

k

k is the neighbor count. If k is 5, for instance, the classifier will consider 5 neighbors when predicting.

Distance metric

The distance metric measures how far two data points are from each other, measured by their features. A standard distance metric is Euclidean distance, but many others can be used, such as the Manhattan distance.

Aggregation function

Once the classifier has found the k nearest neighbors, it calls its aggregation function using the target values of these neighbors. A standard aggregation function for classification is majority vote, but many others can be used.

Challenge

The following are to be taken as input:

• a list of train data points where each data point consists of:
  • a list of numbers, where the lengths of these lists are the same for all training data points; and
  • a target value;
• k;
• a distance metric which takes two data points' features and outputs a positive number; and
• an aggregation function which takes k data points' target values and outputs a value that is one of the target values contained within the training data (this means that you can't just output some arbitrary number; it must be a target value for at least one of the train data points);

You may also take in the following:

• a list of features of the same length as every list of features within the training data.

Output:

• if an additional list of features was given, the prediction, found using the procedure described above, for that list of features;
• if no such list was given, a function that takes in a list of features as described above and outputs the prediction for that list of features.

Input is flexible so long as it is within reason.

The distance metric and aggregation function are black-box functions.

This challenge is code-golf, so the shortest code, measured in bytes, wins.

Test case

Given in python.

k = 5
distance = lambda a, b: (b[0] - a[0]) ** 2 + (b[1] - a[1]) ** 2 # Euclidean distance
aggregation = lambda a, b, c, d, e: max([a, b, c, d, e], key=lambda i: [a, b, c, d, e].count(i)) # output the one with the most occurrences
train = [
 [[1, 2, 3, 4, 5], 0],
 [[2, 3, 4, 5, 6], 1],
 [[3, 4, 5, 6, 7], 1],
 [[1, 4, 5, 6, 7], 0],
 [[1, 8, 9, 9, 9], 0],
 [[9, 0, 0, 0, 0], 1],
 [[2, 3, 4, 1, 1], 1]
]

classifier_func = knn_classifier(train, k, distance, aggregation) # function output

print(classifier_func([1, 0, 0, 0, 0])) # outputs 1

print(knn_direct_classifier(train, k, distance, aggregation, [1, 0, 0, 0, 0])) # directly predicts given input, outputs 1

Do we have any feedback? Duplicate? Clarification needed? Wrong terminology? Wrong test case?