I think it UV and N! where it really starts counting...
Mainly yes because the armored opponents there are countless.
Not to mention that on recent UV runs I did I almost never find full armors since they are all picked up by barons/hell knights/formers when I begin a level.
(the levels are usually swarmed by creatures anyway you never have time to get the armors first unless extremely lucky)
But anyway you'll notice the difference mostly on bosses on the low diff. levels.
