In this paper, the acoustical quality in apartment housing is studied. The purpose is to find out to what extent occupants are annoyed by indoor noise and to compare the annoyance with measured airborne and impact sound insulation. The occupants in 38 building cases in Sweden, grouped into different construction categories, were asked in a questionnaire to rate their annoyance for a variety of potentially disturbing sound sources. In total, 1230 individual responses were used for the statistical analyses. The result shows that on average, the occupants are quite satisfied and reported low annoyance. This is taken as an indication that the present National legislation for sound insulation, airborne sound insulation included, works well. However, annoyance from footstep of walking neighbours is an exception, causing significantly greater annoyance compared to any other source, especially among occupants in lightweight buildings. The commonly used impact sound insulation descriptors are unable to match subjective experience. In combination with the sensitivity of lightweight floors to low-frequency sounds, improper building designs are likely to result in poor noise protection for the occupants. To overcome this issue, a new single number quantity taking frequencies as low as 25 Hz into account is suggested.