We present results from the SPring-8 Angstrom Compact free electron LAser (SACLA) X-ray free electron laser (XFEL) facility, using an X-ray pump, X-ray probe scheme to observe ultrafast changes in the structure of heated graphite. The 9.8 keV XFEL beam was focused to give an intensity on the order of  $∼ 10^{19} W/cm^{2}$, and the evolution of the diffraction pattern observed up to delays of 300 fs. The interplanar diffraction peaks weaken significantly within 10s of femtoseconds, but in-plane diffraction orders i.e. those with Miller Index ($hk0$), persist up to 300 fs, with the observed signal increasing. We interpret this as nonthermal damage through the breaking of interplanar bonds, which at longer timescales leads to ablation by removal of intact graphite sheets. Post-experiment examination of the graphite samples shows damage which is comparable in size to the range of the excited photoelectrons. These results highlight the challenges of accurately modelling X-ray driven heating, as it becomes a routine approach to generating high energy density states.