The decoder is the part of the CPU that maps instruction opcodes to a set of control signals. For example "LDA absolute" (opcode 0xA5) would activate the "put the result in A" signal on its last cycle while "LDX absolute" (opcode 0xA6) would activate the "put the result in X" signal. The undocumented "LAX absolute" (opcode 0xA7) simply activates both because of the decoder logic's internal wiring, causing the result to be put in both registers. For other undocumented opcodes, the "do both of these things" logic is less recognizable but it's always there. Specifically disallowing these illegal states (to make them NOPs or raise an exception, for instance) would require more die space and push the price up.
See here[1] for example to get a sense of how opcode bits form certain patterns when arranged in a specific way.
[1] https://www.nesdev.org/wiki/CPU_unofficial_opcodesWiring all the "illegal" instructions to a NOP would have taken a fair bit of extra logic, and that would have been a noticeable chunk of the transistor budget at the time.