In short: the key of D minor have historically been regarded as the saddest, darkest and most melancholic key in music.
However, with our modern 12 tone equal temperament tuning system, the difference in “sadness” is more about the tone and tuning of the instruments themselves, not the key.
For example, if you play a guitar you know that open strings sound more rich and full. And the D minor key lacks this richness. I would therefore argue that music focused on acoustic guitar sounds the most sad in E minor.
The point is that a lower key, that sounds most rich and full on the particular instrument, will sound darker and therefore more melancholic and sad compared to other minor keys.
I believe this is why the key of D minor have historically been regarded as the saddest key in orchestral music. C is the lowest note on basses, cellos and violas. And you can not use an open string if you want to add vibrato.
The key of D minor also suits the tuning of other instruments in the orchestra.
One of the most famous film music composers of all time, Hans Zimmer, is also well known for heavily favouring the key of D minor. And you will find it used in most his scores.
So while it’s tough to argue that there even is a “saddest key in music”, D minor has historically been used as the saddest and most emotional key in orchestral music (and even in modern soundtrack music). But overall it depends on the instrumentation of the composition, and the respective tuning of all the instruments used.