Inelastic X-ray scattering (IXS) is an important tool for studies of equilibrium dynamics in condensed matter. A new spectrometer recently proposed for ultra-high-resolution IXS (UHRIX) has achieved 0.6~meV and 0.25~nm$^{-1}$ spectral and momentum transfer resolutions, respectively. However, further improvements down to 0.1~meV and 0.02~nm$^{-1}$ are required to close the gap in energy-momentum space between high and low frequency probes. We show that this goal can be achieved by further optimizing the X-ray optics and by increasing the spectral flux of the incident X-ray pulses. UHRIX performs best at energies from 5 to 10 keV, where a combination of self-seeding and undulator tapering at the SASE-2 beamline of the European XFEL promises up to a hundred-fold increase in average spectral flux compared with nominal SASE pulses at saturation, or three orders of magnitude more than possible with storage-ring based radiation sources. Wave-optics propagation shows that about $7\times 10^{12}$~ph/s in a $90$-$\mu$eV bandwidth can be achieved on the sample. This will provide unique new possibilities for dynamics studies by IXS.