Abstract
Support vector machines (SVM) and relevance vector machines (RVM) constitute two state-of-the-art learning machines that are currently focus of cutting-edge research. SVM present accuracy and complexity preponderance, but are surpassed by RVM when probabilistic outputs or kernel selection come to discussion. We propose a two-level hierarchical hybrid SVM-RVM model to combine the best of both learning machines. The proposed model first level uses an RVM to determine the less confident classified examples and the second level then makes use of an SVM to learn and classify the tougher examples. We show the benefits of the hierarchical approach on a text classification task, where the two-levels outperform both learning machines