KAN Practical Implementation (Kolmogorov–Arnold Networks Algorithm)

Kolmogorov–Arnold Networks (KAN) are put to work on a heart-disease classification task using a practical Python pipeline: load a Kaggle dataset, preprocess it, train a KAN-based classifier, evaluate it, then improve performance through hyperparameter tuning. The core takeaway is that careful tuning of KAN-specific settings can substantially raise test-set accuracy compared with default parameters, turning a baseline model into a stronger predictor for whether heart disease is present.

The workflow starts in a Google Colab notebook and uses the IM model SX library (with scikit-learn utilities for data handling and evaluation). The dataset is read from a CSV file into a pandas DataFrame, with the target label encoded as 0/1 for “without heart disease” vs “heart disease.” Feature distributions are visualized to get a sense of the input space. Before modeling, the code separates features (X) from the target (Y) and applies standard scaling so each feature is normalized to mean 0 and standard deviation 1. The dataset is then split into training and test sets with a 70/30 split, using stratification to preserve the class balance in both partitions.

A baseline KAN classifier is trained with default parameter values and evaluated on the test set. Reported performance includes about 75% accuracy, with class-wise F1 scores for the two labels that are roughly similar (around 70% and 71% in the classification report). Additional diagnostics—Cohen’s Kappa and a confusion matrix—are used to quantify agreement beyond chance and to show where misclassifications occur (true positives/negatives and false positives/negatives).

The project then shifts to hyperparameter tuning, targeting KAN’s key knobs: hidden layer size (number of neurons), regularized activation, regularization entropy, regularization R, and spline order. Multiple runs are executed on the training data, each time predicting the unseen test set and printing accuracy, Cohen’s Kappa, classification reports, and confusion matrices. The tuning search identifies a best-performing configuration, with the highest test accuracy reaching roughly 76% during the search.

That best hyperparameter set is then used to train a final model with additional training hyperparameters such as batch size (64), learning rate (0.07), and weight decay (0.01). This tuned model delivers a major jump in test performance to about 85% accuracy. Cohen’s Kappa rises to about 60.9%, and the F1 scores per class land around the mid-60s. The confusion matrix indicates fewer errors, with 27 and 28 samples misclassified (across the two classes). Finally, a ROC curve is plotted, showing an AUC around 0.92, reflecting strong probability estimates for the positive class.

Overall, the implementation frames KAN as a viable alternative modeling approach for tabular medical data, while demonstrating that performance gains depend heavily on selecting KAN-specific hyperparameters rather than relying on defaults. The next planned step is a comparison against NLP methods on the same dataset to benchmark relative effectiveness.