That's a good question.
I'd say this has to do with what happens chemically in the brain.
Neurons (brain cells) have receptors in them...there are tons of different kinds. That is where neurotransmitters (amongst other things) bind..they fit like a lock and key into the receptor. (I'm obsessively into studying all of this brain stuff and it can get very confusing and difficult to simplify...and there is a crazy amount scientists don''t even understand about it yet...but I'll try to explain the best I can).
For instance, one thing stimulants effect is the neurotransmitter dopamine. Dopamine only binds to certain cells...those with dopamine receptors. When this happens, it activates (or deactivates) the cell. These are found in various places throughout the brain and other places don't have them...so they wouldn't be effected by the presence of dopamine.
Different areas and pathways control different things....like, visual input goes to the back of the brain. The senses each have a fairly distinct location that is responsible it..
But, ADHD meds do effect other things...like blood pressure and appetite.
(Hopefully, this makes some sense)