Linear regression models the relationship between a dependent variable (Y) and one or more independent variables (X) using a linear equation: Y = β0 + β1X + ε. The model minimizes the residual sum of squares (RSS) using the least squares method. Regularization techniques like Lasso (L1) and Ridge (L2) regression prevent overfitting.