What is the connection between observed electron behavior and our mathematical formulation of quantum spin?

Introductory chapters on Pauli matrices take as a goal finding a set of transformations with −1 as an eigenvalue. Why? What is it about experiments from the lab that tells us that that is what we should be seeking?

I think the original phenomenon that required going beyond the Schrödinger equation was the fine structure of Hydrogen spectrum that indicated that electrons could be in one of two states with nearly the same energy. How did we get from there to spin notion?


SPIN PHENOMENA IN PARTICLE INTERACTIONS

Brief mention: “One of the historical experiments of the spin separation is the Stern-Gerlach experiment. They considered a particle with spin propagating through the nonuniform magnetic field. Because the derivative of the magnetic field intensity plays a role of the spin-dependent potential, particles with up and down spins are accelerated in opposite directions.”

The Stern-Gerlach Experiment In contrast to the above the Wikipedia article emphasizes an inhomogeneous magnetic field for the phenomenon. This article, however jumps directly to quantum spin dogma passing over entirely any rationale for this peculiar perspective. At least it does connect a phenomenon with a theory. It gives no feeling that this is the simplest theory consistent with Stern-Gerlach.