In Python 3.6+, dictionaries preserve insertion order. This is done by storing the keys, values (and cached hash values) in a separate array, and the hashtable is a succinct array of indexes into this array. This results in more compact dictionaries, which are also a bit faster because of it's cache friendliness. Preserving insertion order is a happy side effect of this. This optimization can also be applied to Python sets, but it hasn't been done yet.
I am personally not a fan of details like this creeping into the language spec. CPython is not the only Python. This kind of spec creates unnecessary challenges for Micropython, for example. And of course there is Cython and others. It makes no sense to me that this should be in the language spec — you are essentially specifying a built in minefield of implementation bugs.
Cpython took this implementation detail from pypy (so it was already going to be the case in the two most used python implementations).
The reason being that the ordered dict ends up being faster than non-ordered, and people will rely on this implementation detail, so they added it to the spec to make that okay.
