No touches would be paid holding the man because a touch is not a hold. If by scrag you mean grab the player for a second as they attempt to gain possession, then, yes, that would be a free kick and so it should be. Why should a player attempting to gain possession of the ball be grabbed? You can bump them but you cannot grab them, this is how it was for the first 150 years of the game. You have always been able to bump, but not grab. If you allow a player in pursuit of the ball to be grabbed, why not allow a push in the back? If you wonder why we get so many stoppages, part of the reason is that the player who is attempting to gain possession has no chance of disposing of the ball because they are grabbed before they even get the ball.
The forwards and backs jostling is fine, it has always been fine, but holding is not fine, it has never been fine. If a backman is in front of the forward then they just have to go for the ball when it gets there. Shepherding has always been an interesting one, but being in front of a leading forward is fine, blocking their path when you are not going for the ball has never been fine. Are you saying that a defender should be able to run into the path of a leading forward to a position where there is no chance they can contest for the mark?
Plus, why make the interpretations even worse by not paying all free kicks observed by the umpires? Are you proposing that some rules be enforced differently to others, some are strict and others not strict? Recipe for inconsistent adjudication.
If they want to do what you suggest then the rules need to be changed. At least they can attempt to be honest.
DS