This paper considers the leader-following synchronization problem for a network of coupled harmonic oscillators by utilizing the relative position measurements between neighboring nodes, where the node dynamics can be either identical or nonidentical. For a homogeneous network with the same node dynamics, two types of first-order observer-based protocols are proposed to achieve leader-following synchronization in the network under some necessary and sufficient conditions, including some synchronization criteria for the homogeneous network subject to parameter uncertainty. For a heterogeneous network with different node dynamics, an output regulation approach is applied to solve the leader-following synchronization problem for the nominal network, based on which the robust synchronization of the uncertain network is investigated with an allowable bound being estimated for parameter uncertainties. Numerical examples are given to illustrate the correctness and the feasibility of the theoretical analysis.