We present a detailed investigation of the dynamic structure factor S(Q,ω) as well as of the dielectric function εM(Q,ω) of the prototypical semiconductor silicon for finite momentum transfer, combining inelastic x-ray scattering measurements and ab initio calculations. We show that, in contrast to optical spectra, for finite momentum transfer, time-dependent density-functional theory in adiabatic local-density approximation (TDLDA) together with the inclusion of lifetime effects in a modified independent-particle polarizability χ0,LT describes the physics of valence excitations with high precision. This applies to the dynamic structure factor as well as to the dielectric function, which demonstrates that TDLDA contains the short-range many-body effects that are crucial for a correct description of εM(Q,ω) in silicon at finite momentum transfer. The form of a nonlocal and energy-dependent exchange-correlation kernel is presented which provides the inclusion of the lifetime effects using the true independent-particle polarizability χ0. The description of the silicon L2,3 absorption edge has been possible by including the outer core electrons 2s and 2p in the valence electrons of the pseudopotential. The energy of the edge is underestimated but a scissor shift of the respective states by the self-energy correction for these states yields good agreement with experiment. Short-range crystal local-field effects and exchange-correlation effects become important with increasing momentum transfer. The inclusion of crystal local-field effects in the random-phase approximation is able to describe the anisotropy of the response well. Our results demonstrate the quantitative predictive power of the first-principles description.
