budding trees
back  
Paper : Budding Trees O. Irsoy, O. T. Yildiz, E. Alpaydin ICPR 22, 2014, Stockholm, Sweden.

Abstract : We propose a new decision tree model, named the budding tree, where a node can be both a leaf and an internal decision node. Each bud node starts as a leaf node, can then grow children, but then later on, if necessary, its children can be pruned. This contrasts with traditional tree construction algorithms that only grows the tree during the training phase, and prunes it in a separate pruning phase. We use a soft tree architecture and show that the tree and its parameters can be trained using gradient-descent. Our experimental results on regression, binary classification, and multi-class classification data sets indicate that our newly proposed model has better performance than traditional trees in terms of accuracy while inducing trees of comparable size.

Code : My C++ code is here. Please cite the paper if you use it.

Data : The UCI datasets used in the paper for regression and binary classification are here. Multiclass classification datasets will be up soon. If you use the data, UCI Repository Citing Policy should apply.



Training a budding tree on sinusoidal toy 1d data. Red shows the response function of the subtree rooted at a given node. Blue shows the gating function (with a higher value giving more responsibility to the left child). Yellow shows leaf contributions (ρ in the paper). Note the smooth transition from an internal to a leaf node (denoted by a increasing transparency in yellow block and increasing opacity in blue block).

Bibtex:
 @inproceedings{irsoy2014budding,
  title={Budding trees},
  author={Irsoy, Ozan and Yildiz, Olcay Taner and Alpaydin, Ethem},
  booktitle={International Conference on Pattern Recognition},
  year={2014}
}