Scikit-learn (often abbreviated as sklearn) is one of the most popular machine learning libraries in Python. It provides simple and efficient tools for data analysis and modeling, including classification, regression, clustering, dimensionality reduction, model evaluation, and preprocessing. Scikit-learn is built on top of NumPy, SciPy, and Matplotlib, and its user-friendly API allows you to build machine learning models quickly and efficiently.
In this guide, we’ll explore how to work with Scikit-learn for various machine learning tasks, including data preparation, model training, evaluation, and making predictions.
1. Installing Scikit-learn
You can install Scikit-learn via pip:
pip install scikit-learn
Once installed, you can import Scikit-learn modules as needed:
pythonCopyEditimport sklearn
2. Scikit-learn Workflow
In general, the workflow for using Scikit-learn involves the following steps:
- Loading the dataset
- Preprocessing the data (cleaning, scaling, encoding)
- Splitting the data into training and testing sets
- Choosing a model (classification, regression, clustering)
- Training the model on the training data
- Evaluating the model on the test data
- Making predictions
3. Loading and Preprocessing Data
3.1. Loading Data
Scikit-learn provides several built-in datasets like Iris, Digits, Boston, and others. You can load them using sklearn.datasets.load_*.
from sklearn.datasets import load_iris
# Load the Iris dataset
iris = load_iris()
X, y = iris.data, iris.target # Features and labels
In this example, X contains the features (input data), and y contains the target variable (labels).
3.2. Data Splitting
Before training a model, it’s important to split your dataset into training and testing sets. Scikit-learn provides train_test_split() for this purpose.
from sklearn.model_selection import train_test_split
# Split the data into training and testing sets (80% training, 20% testing)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
3.3. Feature Scaling
Many machine learning algorithms perform better when the features are scaled. Scikit-learn provides several preprocessing methods, such as StandardScaler, to standardize features.
from sklearn.preprocessing import StandardScaler
# Scale the features
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
4. Classification with Scikit-learn
4.1. Choosing a Classification Model
In Scikit-learn, classification models include algorithms like Logistic Regression, K-Nearest Neighbors, Decision Trees, Random Forests, and more. Here’s an example using Logistic Regression:
from sklearn.linear_model import LogisticRegression
# Initialize the classifier
clf = LogisticRegression(max_iter=200)
# Train the model
clf.fit(X_train_scaled, y_train)
# Evaluate the model
accuracy = clf.score(X_test_scaled, y_test)
print(f'Accuracy: {accuracy * 100:.2f}%')
4.2. Making Predictions
Once the model is trained, you can use it to make predictions on new data.
# Make predictions on the test set
y_pred = clf.predict(X_test_scaled)
# Print the predicted labels
print(y_pred)
4.3. Confusion Matrix
For classification tasks, it’s often helpful to evaluate model performance with a confusion matrix.
from sklearn.metrics import confusion_matrix
# Compute confusion matrix
cm = confusion_matrix(y_test, y_pred)
print('Confusion Matrix:')
print(cm)
5. Regression with Scikit-learn
Scikit-learn also provides regression models such as Linear Regression, Ridge Regression, and Support Vector Regression. Here’s an example using Linear Regression:
5.1. Linear Regression
from sklearn.linear_model import LinearRegression
# Load the Boston dataset for regression (example dataset)
from sklearn.datasets import load_boston
boston = load_boston()
X, y = boston.data, boston.target
# Split the dataset
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Initialize the model
regressor = LinearRegression()
# Train the model
regressor.fit(X_train, y_train)
# Evaluate the model
y_pred = regressor.predict(X_test)
# Calculate R^2 score (goodness of fit)
r2_score = regressor.score(X_test, y_test)
print(f'R^2 Score: {r2_score:.2f}')
6. Model Evaluation and Metrics
Scikit-learn provides several metrics to evaluate model performance. These include classification metrics (e.g., accuracy, precision, recall) and regression metrics (e.g., Mean Squared Error, R^2 score).
6.1. Classification Metrics
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
# Calculate classification metrics
accuracy = accuracy_score(y_test, y_pred)
precision = precision_score(y_test, y_pred, average='weighted')
recall = recall_score(y_test, y_pred, average='weighted')
f1 = f1_score(y_test, y_pred, average='weighted')
print(f'Accuracy: {accuracy * 100:.2f}%')
print(f'Precision: {precision:.2f}')
print(f'Recall: {recall:.2f}')
print(f'F1 Score: {f1:.2f}')
6.2. Regression Metrics
For regression tasks, common metrics include Mean Squared Error (MSE) and R² score.
from sklearn.metrics import mean_squared_error, r2_score
# Calculate MSE and R² score
mse = mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)
print(f'Mean Squared Error: {mse:.2f}')
print(f'R² Score: {r2:.2f}')
7. Model Selection and Tuning
7.1. Cross-Validation
To better evaluate model performance, you can use cross-validation, which splits the data into multiple subsets and trains and tests the model multiple times.
from sklearn.model_selection import cross_val_score
# Perform 5-fold cross-validation
cv_scores = cross_val_score(clf, X, y, cv=5)
print(f'Cross-validation scores: {cv_scores}')
print(f'Mean CV score: {cv_scores.mean():.2f}')
7.2. Hyperparameter Tuning (GridSearchCV)
You can use GridSearchCV to search for the best hyperparameters for your model.
from sklearn.model_selection import GridSearchCV
# Define the parameter grid for Logistic Regression
param_grid = {'C': [0.1, 1, 10], 'solver': ['liblinear', 'saga']}
# Initialize GridSearchCV
grid_search = GridSearchCV(LogisticRegression(), param_grid, cv=5)
# Fit the model to find the best parameters
grid_search.fit(X_train_scaled, y_train)
# Print the best parameters
print(f'Best Parameters: {grid_search.best_params_}')
8. Clustering with Scikit-learn
Scikit-learn also provides several clustering algorithms, such as K-Means and DBSCAN. Here’s how you can use K-Means clustering:
from sklearn.cluster import KMeans
# Load the Iris dataset
iris = load_iris()
X = iris.data
# Initialize the KMeans model
kmeans = KMeans(n_clusters=3)
# Train the model
kmeans.fit(X)
# Get the cluster centers and labels
centers = kmeans.cluster_centers_
labels = kmeans.labels_
print(f'Cluster Centers:\n{centers}')
print(f'Labels: {labels[:10]}') # Print first 10 labels