AD FS Claims Rules Deny Caveat

In setting up claims rules to deny groups particular claims, I noticed if I deny the "domain users" group it will deny everyone even if they're not in the "domain users" group.