This is purely my speculation, but I think it is OOP and dynamic types that is the toxic mix.

Lisp codebases tend not do it as much (CLOS notwithstanding). Data tends to be flatter and simpler. Data shape is in theory just as hard to refactor properly, but IME Lisp projects just do a lot less of that.

Of course there are also alternatives like gradual types or libs like spec in Clojure.