---

name: ds-feature-engineering compatibility: opencode completeness: 95 content-types:

• code
• guidance
• do-dont
• examples description: '"Creates and transforms features including polynomial features, interactions domain-specific features, and feature transformations"' license: MIT maturity: stable metadata: domain: coding output-format: code related-skills: ds-categorical-encoding, ds-dimensionality-reduction, ds-feature-scaling-normalization ds-feature-selection ds-missing-data role: implementation scope: implementation triggers: feature engineering, feature creation, feature transformation, how do I engineer features, feature design archetypes:
  • tactical
  • generation anti_triggers:
  • brainstorming
  • vague ideation
  • code golf
  • over-engineering response_profile: verbosity: low directive_strength: high abstraction_level: operational version: "1.0.0"

---

Feature Engineering

Comprehensive guide to feature engineering in machine learning and data science workflows.

When to Use This Skill

• Solving real-world feature engineering problems
• Building machine learning pipelines with feature engineering
• Implementing best practices for feature engineering
• Optimizing model performance using feature engineering techniques
• Learning industry-standard approaches to feature engineering

When NOT to Use This Skill

• When using pre-built libraries without understanding underlying concepts
• For toy problems that don't require feature engineering rigor
• When domain expertise in specific problem requires different approach
• If your problem doesn't require the complexity this skill provides

Purpose and Key Concepts

Feature Engineering is a critical component of the machine learning workflow. This skill covers:

1. Theoretical foundations — Mathematical principles and statistical concepts
2. Practical implementation — Working code examples and patterns
3. Common pitfalls — Mistakes to avoid and how to recover from them
4. Best practices — Industry-standard approaches and optimization techniques

Core Workflow

1. Understand the problem — Clearly define what you're solving for
2. Select approach — Choose the right technique for your data and constraints
3. Implement solution — Write clean, tested code following best practices
4. Validate results — Verify your implementation with tests and validation
5. Optimize performance — Improve efficiency and accuracy incrementally

Implementation Patterns

Pattern 1: Basic Feature Engineering

import pandas as pd
import numpy as np
from sklearn.preprocessing import PolynomialFeatures, StandardScaler
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline

def basic_feature_engineering(df: pd.DataFrame, target_col: str, numeric_cols: list) -> tuple:
    """
    Apply basic feature engineering transformations to a DataFrame.
    Handles polynomial features, log transforms, and standard scaling.
    """
    if df.empty:
        raise ValueError("Input DataFrame cannot be empty")
    
    # Separate features and target
    X = df.drop(columns=[target_col])
    y = df[target_col]
    
    # Identify numeric columns for transformation
    numeric_features = [col for col in numeric_cols if col in X.columns]
    
    # Create transformation pipeline
    transformer = ColumnTransformer(
        transformers=[
            ('poly', PolynomialFeatures(degree=2, include_bias=False), numeric_features)
            ('scaler', StandardScaler(), numeric_features)
        ]
        remainder='passthrough'
    )
    
    # Fit and transform data
    X_transformed = transformer.fit_transform(X)
    
    # Create DataFrame with new feature names
    poly_names = transformer.named_transformers_['poly'].get_feature_names_out(numeric_features)
    new_df = pd.DataFrame(X_transformed, columns=poly_names, index=X.index)
    
    return new_df, transformer, y

Pattern 2: Production-Ready Feature Engineering

import logging
import pandas as pd
import numpy as np
from typing import Any, Dict, List, Optional
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
import warnings

logger = logging.getLogger(__name__)
warnings.filterwarnings('ignore')

class FeatureEngineering:
    """Production-grade feature engineering pipeline with validation and logging."""
    
    def __init__(self, numeric_cols: List[str], categorical_cols: List[str], 
                 target_col: str, degree: int = 2, handle_missing: bool = True):
        self.numeric_cols = numeric_cols
        self.categorical_cols = categorical_cols
        self.target_col = target_col
        self.degree = degree
        self.handle_missing = handle_missing
        self.pipeline = None
        self.feature_names = None
        
    def _validate_input(self, data: pd.DataFrame) -> None:
        missing_cols = set(self.numeric_cols + self.categorical_cols + [self.target_col]) - set(data.columns)
        if missing_cols:
            raise ValueError(f"Missing required columns: {missing_cols}")
        if data.isnull().sum().sum() > 0 and not self.handle_missing:
            logger.warning("Data contains missing values but handle_missing is False")
            
    def execute(self, data: pd.DataFrame) -> Dict[str, Any]:
        """Execute feature engineering on data with full validation and logging."""
        self._validate_input(data)
        logger.info(f"Starting feature engineering on {len(data)} rows")
        
        X = data.drop(columns=[self.target_col])
        y = data[self.target_col]
        
        # Build preprocessing pipeline
        numeric_transformer = Pipeline(steps=[
            ('imputer', SimpleImputer(strategy='median')) if self.handle_missing else None
            ('scaler', StandardScaler())
        ])
        
        categorical_transformer = Pipeline(steps=[
            ('imputer', SimpleImputer(strategy='most_frequent')) if self.handle_missing else None
            ('encoder', OneHotEncoder(handle_unknown='ignore', sparse_output=False))
        ])
        
        preprocessor = ColumnTransformer(
            transformers=[
                ('num', numeric_transformer, self.numeric_cols)
                ('cat', categorical_transformer, self.categorical_cols)
            ]
            remainder='drop'
        )
        
        # Fit and transform
        X_transformed = preprocessor.fit_transform(X)
        self.pipeline = preprocessor
        
        # Get feature names
        num_names = preprocessor.named_transformers_['num'].named_steps['scaler'].get_feature_names_out(self.numeric_cols)
        cat_names = preprocessor.named_transformers_['cat'].named_steps['encoder'].get_feature_names_out(self.categorical_cols)
        self.feature_names = np.concatenate([num_names, cat_names])
        
        logger.info(f"Feature engineering complete. Output shape: {X_transformed.shape}")
        
        return {
            'X_transformed': X_transformed
            'y': y.values
            'feature_names': self.feature_names
            'pipeline': self.pipeline
            'metadata': {'original_shape': data.shape, 'transformed_shape': X_transformed.shape}
        }

Best Practices

• ✅ Always validate your implementation on test data
• ✅ Document your assumptions and methodology
• ✅ Use version control for reproducibility
• ✅ Monitor performance metrics in production
• ✅ Periodically review and update your approach
• ✅ Test with edge cases and outliers
• ✅ Log all significant operations for debugging

Common Pitfalls

| Pitfall | Problem | Solution | |

---

---

Constraints

MUST DO

• Document every engineered feature with its formula, data source, and expected range before adding to model pipeline
• Validate new features against the target variable using mutual information or point-biserial correlation
• Implement feature transformations in a composable pipeline: fit-only on training data, transform on all splits
• Track feature drift over time by monitoring distribution statistics (mean, std, skewness) per batch

MUST NOT DO

• Do not create features that leak the target variable (e.g., using future values or post-event metrics)
• Avoid creating too many features without dimensionality reduction — aim for 20-50 high-signal features
• Never engineer features using statistics computed over the full dataset including test set — causes data leakage
• Do not use custom feature engineering that cannot be reproduced in production — stick to standard transformations

Live References

Authoritative documentation links for this skill's domain. The model follows markdown links at load time to resolve external references and inline content.

• Feature Engineering — Scikit-learn docs
• Feature Engineering Guide (Towards Data Science)
• Featuretools — Automated Feature Engineering
• PyCaret Feature Engineering
• Kaggle Feature Engineering Course