Hyperdimensional computing (HDC) is a powerful algorithmic framework at the intersection of symbolic and neural network Artificial Intelligence. In particular, HDC has received significant attention as a suitable candidate for low-resource machine learning tasks, exemplified by wearable Internet of Things. To solve classification tasks, HDC transforms input data to a high-dimensional space and uses simple component-wise vector operations to create, train, and operate the classification model. While the classical centroid model has been often used in HDC, iterative updating of centroids with wrongly classified samples improves the classification performance. In this paper, using a large and variable collection of 121 UCI datasets, we explore how confidence-driven training of centroids formed from HDC representations further improves the classification accuracy.