Random Forests are an ensemble learning method that builds multiple decision trees during training. For classification, each tree votes on the most popular class, and for regression, it averages the predictions of individual trees. The "random" aspect comes from two key features:
- Bagging (Bootstrap Aggregating): Each tree is trained on a random subset of the training data, sampled with replacement.
- Feature Randomness: At each split in a decision tree, only a random subset of features is considered, rather than all available features.
An ensemble method is a machine learning technique that combines the predictions from multiple individual models (often called base learners or weak learners) to produce a more robust and accurate final prediction than any single model could achieve on its own. Random Forests are a prime example of an ensemble method, specifically a bagging-based approach.