AutoML Pipeline 🚀

A comprehensive, configurable automated machine learning pipeline that handles the entire ML workflow from data loading to model deployment. Built for developers, data scientists, and ML practitioners who want to quickly prototype and deploy machine learning solutions. Agile Creative Labs Inc. positioned the project as a more user-friendly, all-in-one solution compared to some of the more specialized or complex alternatives in the AutoML ecosystem.

What Makes automlpipeline Different

The automlpipeline project appears to be similar to existing tools but we focuses on:

• Comprehensive data inspection and quality analysis
• Multiple file format support
• Rich visualization capabilities
• Production-ready deployment features
• Simple configuration-based usage

✨ Features

• 📊 Comprehensive Data Processing

  • Support for multiple file formats (CSV, Excel, JSON, Parquet)
  • Automated data inspection and quality analysis
  • Missing value detection and handling
  • Outlier detection and removal
  • Data validation and integrity checks

• 🔧 Advanced Preprocessing

  • Intelligent feature type detection
  • Multiple imputation strategies (mean, median, mode, KNN)
  • Categorical encoding (one-hot, label encoding)
  • Feature scaling and normalization
  • Automated feature selection

• 🤖 Smart Model Selection

  • Automatic problem type detection (classification/regression)
  • Multiple algorithms support (Random Forest, SVM, Linear/Logistic Regression)
  • Cross-validation with configurable folds
  • Hyperparameter tuning (Grid Search, Random Search)
  • Model comparison and selection

• 📈 Rich Analytics & Visualization

  • Comprehensive model evaluation reports
  • Feature importance analysis
  • Data distribution visualizations
  • Correlation heatmaps
  • Model performance comparisons

• 💾 Production Ready

  • Model and pipeline persistence
  • Configurable output formats
  • Detailed logging and error handling
  • Easy deployment and integration

🚀 Quick Start

Installation

# Clone the repository
git clone https://github.com/Agile-Creative-Labs/automl-pipeline.git
cd automl-pipeline

# Install dependencies
pip install -r requirements.txt

Basic Usage

# Run with minimal configuration
python automl_pipeline.py --data your_data.csv --target target_column

# Run with custom output directory
python automl_pipeline.py --data data.csv --target price --output results/

# Specify problem type explicitly
python automl_pipeline.py --data data.csv --target category --problem-type classification

Using Configuration Files

Create a config.yaml file:

# Data settings
data_path: "data/housing.csv"
target_column: "price"
problem_type: "regression"

# Preprocessing options
handle_missing: "auto"
scaling_method: "standard"
outlier_detection: true

# Model settings
models_to_try: ["random_forest", "linear_regression", "svm"]
cross_validation_folds: 5
hyperparameter_tuning: true

# Output settings
output_dir: "ml_results"
create_visualizations: true
generate_report: true

Then run:

python automl_pipeline.py --config config.yaml

📖 Detailed Usage

Data Requirements

Your data should be in a structured format with:

• Target column: The variable you want to predict
• Feature columns: Input variables for prediction
• Supported formats: CSV, Excel (.xlsx, .xls), JSON, Parquet

Configuration Options

Parameter	Description	Default	Options
data_path	Path to your dataset	-	Any valid file path
target_column	Name of target variable	-	Column name in your data
problem_type	ML problem type	"auto"	"classification", "regression", "auto"
handle_missing	Missing value strategy	"auto"	"drop", "impute_mean", "impute_median", "knn"
scaling_method	Feature scaling method	"standard"	"standard", "minmax", "none"
models_to_try	Models to evaluate	["random_forest", "logistic_regression", "svm"]	List of model names
hyperparameter_tuning	Enable hyperparameter tuning	true	true, false
cross_validation_folds	Number of CV folds	5	Integer > 1
test_size	Test set proportion	0.2	Float between 0 and 1

Advanced Usage

Custom Model Configuration

from automl_pipeline import AutoMLPipeline, PipelineConfig

# Create custom configuration
config = PipelineConfig(
    data_path="data/customer_data.csv",
    target_column="churn",
    problem_type="classification",
    models_to_try=["random_forest", "svm"],
    hyperparameter_tuning=True,
    create_visualizations=True
)

# Initialize and run pipeline
pipeline = AutoMLPipeline(config)
pipeline.run_full_pipeline()

Loading Saved Models

import joblib

# Load the trained model
model = joblib.load("automl_output/best_model.joblib")

# Load preprocessing pipeline
preprocessor = joblib.load("automl_output/preprocessing_pipeline.joblib")

# Make predictions on new data
predictions = model.predict(new_data)

📁 Output Structure

After running the pipeline, you'll get:

automl_output/
├── best_model.joblib              # Trained model
├── preprocessing_pipeline.joblib   # Data preprocessing pipeline
├── data_inspection_report.json    # Data analysis results
├── model_evaluation_report.json   # Model performance metrics
├── feature_info.json             # Feature metadata
├── pipeline.log                   # Execution logs
└── visualizations/               # Generated plots
    ├── feature_distributions.png
    ├── correlation_heatmap.png
    └── model_comparison.png

🔧 Supported Models

Classification

• Random Forest Classifier: Ensemble method with excellent performance
• Logistic Regression: Linear classifier with probabilistic output
• Support Vector Machine: Powerful for complex decision boundaries

Regression

• Random Forest Regressor: Robust ensemble method
• Linear Regression: Simple and interpretable
• Support Vector Regression: Effective for non-linear relationships

📊 Example Results

Model Performance Report

{
  "model_results": {
    "random_forest": {
      "cross_validation": {"mean_score": 0.94, "std_score": 0.02},
      "test_metrics": {"accuracy": 0.93, "precision": 0.94, "recall": 0.93}
    }
  }
}

Automated Insights

• Feature importance rankings
• Data quality assessment
• Model recommendation based on performance
• Visualization of key patterns

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Setup

# Clone and setup development environment
git clone https://github.com/your-username/automl-pipeline.git
cd automl-pipeline

# Install development dependencies
pip install -r requirements-dev.txt

# Run tests
python -m pytest tests/

# Run linting
flake8 automl_pipeline.py
black automl_pipeline.py

Ways to Contribute

• 🐛 Bug Reports: Found an issue? Let us know!
• 💡 Feature Requests: Have ideas for new features?
• 📖 Documentation: Help improve our docs
• 🧪 Testing: Add test cases and improve coverage
• 🔧 Code: Submit pull requests with improvements

📋 Roadmap

• Advanced Feature Engineering

  • Polynomial features
  • Feature interactions
  • Time-series features

• More Algorithms

  • XGBoost and LightGBM
  • Neural networks
  • Ensemble methods

• Enhanced Deployment

  • REST API generation
  • Docker containerization
  • Cloud deployment templates

• Advanced Analytics

  • SHAP explanations
  • Fairness metrics
  • A/B testing framework

• GUI Interface

  • Web-based dashboard
  • Drag-and-drop pipeline builder
  • Real-time monitoring

📚 Examples

Example 1: House Price Prediction

# Download sample data
wget https://raw.githubusercontent.com/datasets/house-prices/master/data/train.csv

# Run regression pipeline
python automl_pipeline.py \
  --data train.csv \
  --target SalePrice \
  --problem-type regression \
  --output house_price_results

Example 2: Customer Churn Classification

# Run classification pipeline
python automl_pipeline.py \
  --data customer_data.csv \
  --target churn \
  --problem-type classification \
  --output churn_analysis

🔍 Troubleshooting

Common Issues

ImportError: Missing dependencies

pip install -r requirements.txt

ValueError: Target column not found

• Check that your target column name is spelled correctly
• Ensure the column exists in your dataset

MemoryError: Dataset too large

• Consider sampling your data first
• Use outlier_detection: false for very large datasets

Poor model performance

• Check data quality in the inspection report
• Try different preprocessing options
• Consider feature engineering

Getting Help

• 📖 Check our Documentation
• 💬 Join our Discord Community
• 🐛 Report issues on GitHub Issues
• 📧 Email us at automl-pipeline@agilecreativelabs.com

📄 License

This project is licensed under the MIT License - see the LICENSE file for details. This project is created by Agile Creative Labs Inc. [Contact Us] (https://agilecreativelabs.com)

🙏 Acknowledgments

• scikit-learn: For the excellent ML library
• pandas: For powerful data manipulation
• matplotlib/seaborn: For beautiful visualizations
• Open Source Community: For inspiration and contributions

⭐ Star History

---

Made with ❤️ for the Open Source Community by Agile Creative Labs Inc.

If you find this project helpful, please consider giving it a star ⭐ and sharing it with others!