Diabetes Prediction (Stacking)
A stacking ensemble for early diabetes prediction on the PIMA dataset: six base classifiers, five of them hyperparameter-tuned, feed a Random Forest meta-learner. Co-authored and published at IEEE AIMV 2021.
OVERVIEW
A stacking ensemble for early diabetes prediction on the PIMA Indians dataset (768 patients, 8 clinical features), co-authored and published at IEEE AIMV 2021. Six base classifiers (Gaussian Naive Bayes, Random Forest, Decision Tree, SVM, an ANN and Logistic Regression, all but Naive Bayes tuned with randomized search) are combined as the level-0 estimators of a scikit-learn StackingClassifier (cv=4) whose Random Forest meta-learner makes the final prediction. The whole pipeline, data through evaluation, lives in one notebook.
ARRIVED AS
“Single classifiers on the small PIMA Indians Diabetes dataset trade off against each other: one is better on some patients, another on others. The question was whether stacking (letting a meta-model learn how to combine several tuned base classifiers) would predict diabetes more reliably than any one model on its own.”
This was a research project that became a co-authored IEEE paper (AIMV 2021) with five collaborators. The dataset is the well-known PIMA Indians Diabetes set: 768 patients, 8 clinical features (glucose, blood pressure, BMI, age, and so on), and a binary diabetes outcome. It is small and noisy, which is exactly the setting where ensembling is supposed to help, so the project tunes several classifiers and then stacks them under a meta-learner to see whether the combination is steadier than any single model.
WHAT I BUILT
- 01Six base classifiers: Gaussian Naive Bayes, Random Forest, Decision Tree, SVM, an ANN (MLP) and Logistic Regression. Five are tuned with randomized search and cross-validation; Naive Bayes uses a fixed var_smoothing.
- 02A stacking ensemble where the six base models' predictions become inputs to a Random Forest meta-learner (scikit-learn StackingClassifier, cv=4).
- 03Standard preprocessing on the PIMA Indians dataset (768 patients, 8 clinical features) with a 70/30 train/test split.
WHAT CHANGED
- Co-authored and published at the 2021 International Conference on Artificial Intelligence and Machine Vision (AIMV 2021) on IEEE Xplore (DOI 10.1109/AIMV53313.2021.9670920).
- Re-run from the committed notebook with fixed seeds; the gap to the published number is stated below rather than smoothed over.
- A single reproducible notebook: data, preprocessing, six base models, the stacking ensemble, and per-model evaluation end to end.
ARCHITECTURE
The pipeline is a single notebook. After loading and preprocessing the PIMA data, it splits 70/30 (random_state fixed for reproducibility) and trains six base classifiers: five tuned with RandomizedSearchCV over their own hyperparameter grids, and Gaussian Naive Bayes with a fixed var_smoothing. Those six models become the level-0 estimators of a scikit-learn StackingClassifier; their out-of-fold predictions (cv=4) feed a Random Forest level-1 meta-learner that produces the final prediction. Every model, the six base classifiers and the stacked ensemble, is evaluated on the same held-out test set so they can be compared directly.
Data flow
click a stage
Read the PIMA dataset (768 rows, 8 clinical features) and prepare the feature matrix and binary outcome.
COMPONENT
Preprocessing + splitLoads the PIMA dataset, assembles the 8 feature columns and the outcome, and makes a reproducible 70/30 train/test split.
ALSO IN THE BUILD
Random Forest meta-learnerThe level-1 estimator that learns how to weight the base models' predictions into the final diabetes prediction.
Decisions, with the cost of each.
A decision without its trade-off is marketing. Each row says what was chosen, why, and what it gave up.
Stacking rather than a single tuned model or simple voting
On a small, noisy dataset different classifiers make different mistakes. Stacking lets a meta-learner learn how to combine them from data, which is more flexible than hard or soft voting and was the hypothesis the paper set out to test.
INSTEAD OF
One well-tuned model (simpler, but leaves the complementary strengths of others unused); majority voting (combines models but cannot learn how to weight them).
Tune the base models before stacking
A stack is only as good as its members. Running RandomizedSearchCV on five of the base classifiers first means the ensemble combines reasonable models rather than carrying weak defaults, and randomized search keeps tuning cheap across five models. Naive Bayes has little to tune and keeps a fixed var_smoothing.
INSTEAD OF
Stack untuned defaults (weaker base learners); exhaustive GridSearchCV (far more expensive across five models for little gain here).
Evaluate on a fixed held-out split and report it honestly
With 768 rows the result is sensitive to the split, so fixing the random_state keeps it reproducible. The stack's test accuracy is reported as it is.
INSTEAD OF
Report only the best-looking number (misleading on small data); skip a held-out set (no honest generalization estimate).
The part that mattered.
The facts behind the work, and the code that produced them.
- model ensemble
- 6 + 1
- 6 base models (5 tuned) + RF meta-learner
- test accuracy
- 73.59%
- 30% held-out (76.91% train); 82.68% in the paper
- PIMA dataset
- 768 x 8
- patients x clinical features
- co-authored paper
- IEEE 2021
- AIMV, 6 authors
from sklearn.ensemble import StackingClassifier, RandomForestClassifier
level0 = [
('nb', nb_model),
('rfc', rf_grid_search),
('dt', dt_grid_search),
('lr', lr_grid_search),
('svm', svm_grid_search),
('ann', ann_grid_search),
]
level1 = RandomForestClassifier(random_state=42) # meta-learner
model = StackingClassifier(estimators=level0, final_estimator=level1, cv=4)
model.fit(X_train, y_train.ravel())
The six base classifiers (five tuned, plus Naive Bayes) are the level-0 estimators; the `*_grid_search` names are the notebook's, and each holds a RandomizedSearchCV; the StackingClassifier runs them with 4-fold cross-validation and feeds their predictions to a Random Forest level-1 meta-learner that makes the final call.
feature_col_names = ['Pregnancies', 'Glucose', 'BloodPressure',
'SkinThickness', 'Insulin', 'BMI',
'DiabetesPedigreeFunction', 'Age']
X = data_frame[feature_col_names].values
y = data_frame[['Outcome']].values
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=42)
Eight clinical features predict the binary outcome. With only 768 patients the score depends on the split, so random_state is fixed: the 30% held-out set is 231 patients, and that is the set every model, base and stacked, is scored on.
LEARNED
Stacking helps most when the base models are individually decent and make different errors, so tuning each one before combining them matters more than the choice of meta-learner.
Stacking did not beat the base models here. With 231 test patients one point is about two patients, and most of the models sit within a couple of points of each other, so this setup cannot name a winner.
Fixing the split and reporting the held-out number as-is is the honest way to present a result that a different random seed would move.