Explain how this doesn't count as reasoning. Or this. To name just a couple examples.
Yes, they work by fuzzy logical reasoning. That is literally how neural networks, including the FFNs in Transformers, work. Every neuron is a fuzzy classifier that divides a superposition of questions formed by its input field by a fuzzy hyperplane, "answering" the superposition with an answer ranging from yes to no to anything in-between. Since the answers to each layer form the inputs to the next layer, the effective questions form grow with increasing complexity as network depth grows. Transformers works by combining DNNs with latent states (works on processing concepts, not raw data, with each FFN detecting concepts in their input and encoding resultant concepts into their output) and an attention mechanism (the FFNs of a given layer can choose what information they "want to look at" in the next FFN).