Smartphone-based approaches for Human Activity Recognition have become prevalent in recent years. Despite the amount of research undertaken in the field, issues such as cross-subject variability are still posing an obstacle to the deployment of solutions in large scale, free-living settings. Personalized methods (i.e. aiming to adapt a generic classifier to a specific target user) attempt to solve this problem. The lack of labeled data for training purposes, however, represents a major barrier. This is especially the case when taking into consideration that personalization generally requires labeled data to be user-specific. This paper presents a novel personalization method combining a semi-population based approach with user adaptation. Personalization is achieved through the following. Firstly, the proposed method identifies a subset of users from the available population as best candidates for initializing the classifier to the target user. Subsequently, a semi-population Neural Network classifier is trained using data from this subset of users. The classifier’s network weights are then updated using a small amount of labeled data from the target user subsequently implementing personalization. This approach was validated on a large publicly available dataset collected in a free-living scenario. The personalized approach using the proposed method has shown to improve the overall F-score to 74.4% compared to 70.9% when using a generic non-personalized approach. Results obtained, with statistical significance being confirmed on a set of 57 users, indicate that model initialization using the semi-population approach can reduce the amount of labeled data required for personalization. As such, the proposed method for model initialization could facilitate the real-world deployment of systems implementing personalization by reducing the amount of data needed for personalization.