For geotechnical construction and maintenance, assessing the shearing behavior at the soil–structure interface is significant. This study presents an experimental investigation into the effect of the grain size distribution of a sandy soil on the shearing behaviors at the soil–structure interface, using a modified direct shear apparatus. Five soil samples with different coefficients of uniformity were prepared. The normalized roughness of the structure surface (the ratio between the maximum roughness of the structure plate and the mean grain size of the soil); relative density; and maximum, mean, and minimum grain sizes of all samples were controlled in the same way. During the tests, the shear force, shear displacement, and vertical displacement were monitored. The results show that at a given shear displacement and normal stress, the sample with lower coefficient of uniformity Cu presented higher shear stress and more pronounced dilative behavior. The increase of Cu led to a decrease of the friction angle for the soil–structure interface (at both peak and ultimate states) and a decrease of the maximum vertical deformation of the soil sample during the shearing process. As Cu increased, the main force chain at the soil–structure interface turned from contact between the coarser grains to that mainly formed by the finer grains, resulting in a decrease of the shearing resistance. In comparison with the previous relevant studies, the decreasing trend of the friction angle with the increase of Cu is strongly supported.