does the average use have to worry about HTTPS though?

Yes, but arguably this will make it more clear when what they are doing is insecure.

As you mentioned the majority of sites use HTTPS, which is good. Most of the time users will never notice anything. But on occasion, when a site is insecure, they’ll have a warning.

If every site were insecure, everyone would ignore the warning. If only a few rare sites are insecure, someone may notice.