Sonar Object Classification: Logistic Regression vs. SVM

This project implements and compares two different machine learning models to classify sonar signals reflected off metal cylinders (mines) and rocks. By analyzing 60 different frequency patterns, the system determines whether the detected object is a potential threat or a natural formation.

Project Overview

The core of this project is a comparative analysis between a linear baseline (Logistic Regression) and a high-complexity non-linear model (Support Vector Machine). We focus on balancing raw accuracy with model stability and generalization.

Features

• Data Preprocessing: Handles the Sonar dataset with 208 observations and 61 attributes
• Logistic Regression: Serves as the baseline linear classifier
• SVM with RBF Kernel: An optimized Support Vector Machine using non-linear transformation
• Comprehensive Evaluation: Includes Accuracy, F1-Score, Cross-Validation (5-fold), and Overfitting Analysis

Model Parameters

Based on experimental "tournaments" and testing, the following configurations were found to be optimal for this specific dataset:

Model	Parameters
Logistic Regression	Default Scikit-Learn settings
SVM	kernel='rbf', C=10, gamma=1

Performance Comparison

The SVM model consistently outperformed Logistic Regression in terms of raw predictive power, though it required careful monitoring for overfitting.

Key Results (Summary)

• SVM Test Accuracy: ~90.48%
• Logistic Regression Test Accuracy: ~76% (average baseline)
• Generalization Gap:
  • SVM: 14.44% (Higher complexity/Overfitting risk)
  • LogReg: ~7-9% (Higher stability/Lower accuracy)

Detailed Metrics Analysis

The project includes a custom "Reliability Score" to calculate the statistical "floor" of the model:

• Mean CV Accuracy: The average performance across 5 different data splits
• Stability (Std Dev): How much the model performance fluctuates
• Reliability Score: The lower bound of expected performance in real-world deployment

How to Use

1. Prepare Data: Ensure Copy of sonar data.csv is in the project directory
2. Run the Notebook: Execute the cells sequentially to:
   • Load and split data
   • Train the log_model and svm_model
   • Generate the Comparison Report
3. Make a Prediction: Use the prediction block at the end of the notebook to input new sonar frequency arrays for real-time classification

Conclusion

While the SVM model exhibits a higher degree of overfitting (indicated by a 1.0 training accuracy), its Reliability Score (0.7291) and Test Accuracy (0.9048) make it the superior choice for high-stakes detection where catching every "Mine" (High Recall) is critical.