Pixel value differencing (PVD) and least significant bit substitution (LSB) are two widely used schemes in image steganography. These two methods do not consider different content in a cover image for hiding the secret data. The content of most digital images has different edge directions in each pixel, and the local object shape or appearance is mostly characterized by the distribution of its intensity gradients or edge directions. Exploiting these characteristics for embedding various secret information in different edge directions will eliminate sequential embedding and improve robustness. Thus, a histogram of oriented gradient (HOG) algorithm is proposed to find the dominant edge direction for each $2\times 2$ block of cover images. Blocks of interest (BOIs) are determined adaptively based on the gradient magnitude and angle of the cover image. Then, the PVD algorithm is used to hide secret data in the dominant edge direction, while the LSB substitution is utilized in the other two remaining pixels. Extensive experiments using various standard images reveal that the proposed scheme provides high embedding capacity and better visual quality compared with several other PVD- and LSB-based methods. Moreover, it resists various steganalysis techniques, such as pixel difference histogram and RS analysis.