As a Scrum Master, I want to make sure that my team doesn't "accidentally" change the priority of User Stories on my Sprints page's backlog view, so that my work is not unnecessarily increased.
How do I set the permissions so that only a select group of people can drag up/down the User Stories?
The permissions in TFS aren't granular enough to prevent this specific field from being changed.
You'd need to use witadmin to set a rule on the Microsoft.VSTS.Common.Priority field to be <READONLY not =""/> with a team/user group in the ""; e.g. READONLY not="[project]\Project Administrators" />, where you're the only user in that group.
witadmin isn't for the faint of heart, so if you're not comfortable editing work item templates, you might just need to wrap their knuckles when they change the priority.
-- or --
You're talking about the order of items on the backlog, which would be their stack rank. In which case, you'd want to do the same but on the Microsoft.VSTS.Common.StackRank field.
