Human activity recognition (HAR) using smartphone sensors has attracted great attention, due to its wide range of applications. A standard solution for HAR is to firstly generate some features defined based on domain knowledge (handcrafted features), and then to train an activity classification model based on these features. Very recently, deep learning with automatic feature learning from raw sensory data has also achieved great performance for HAR task. We believe that both the handcrafted features and the learned features may convey some unique information which can complement each other for HAR. In this paper, we firstly propose a feature fusion framework to combine handcrafted features with automatically learned features by a deep algorithm for HAR. Then, taking the regular dynamics of human behaviour into consideration, we develop a maximum full a posterior (MFAP) algorithm to further enhance the performance of HAR. Our extensive experimental results show the proposed approach can achieve superior performance comparing with state-of-the-art methodologies across both a public dataset and a self-collected dataset.