Decision trees: Gini, Chi-square, Entropy, Variance
Suppose we have a data set of 30 kids, among which 15 play tennis, each student has three features (gender, height, and school), here are the details to perform split methods to measure decisions trees splits:
Split method 1: Gini impurity: the smaller the better
Split method 2: Chi-square: the larger the better
Split method 3: Entropy: the smaller the better
Split method 4: Variance reduction: the larger difference the better