Decision trees recursively split the dataset based on feature values to minimize entropy (ID3) or Gini impurity (CART). The depth and pruning techniques control complexity to reduce overfitting. Decision trees work well for both classification and regression problems but are prone to variance.