One of my three neurology LORs this year is from a pediatric neurologist. I've had absolutely no problems and if anything, it's apparently a pretty good letter since I've gotten a bunch of compliments on it. It's been a good talking point, because it let me discuss how I've explored the different areas of neurology, including peds neuro.
As far as that being your only letter, however, I don't know if that's the best idea. Neuro is a field in which your interviewers are usually generally concerned about your level of interest and how well you understand neurology in terms of "what you're getting into." Having only one LOR from a neurologist can hurt you in that sense (think about it- if you REALLY want to be a neurologist, shouldn't you know more than one well enough to get an LOR from them?)
Tell the neuro faculty you're interested in their field and ask them if you can hang out in clinic or on rounds when you've got some spare time- it's a great way to explore the field more and you may build relationships that allow you ask them for an LOR before fourth year. That failing, make sure you schedule your neuro elective super early!
