Total butterfat by weight is the fat from just the milk and cream.
Total fat by weight is the percentage of normal "lipid" fat, which includes the butterfat and the fat from things like eggs and chocolate as well.
Nonfat Milk Solids, or NFMS as they refer to it in the commercial ice cream business, is basically the "solid" part of skim milk. NFMS doesn't play a big part in making homemade ice cream, but it does in making commercial ice cream, where they add some amount of dry milk powder depending on the amount of butterfat in the recipe. The less butterfat you use, the more NFMS you use to make up for it.
More info can be found in my blog post Making an Ice Cream Butterfat Calculator.
I have no idea if this is actually accurate. ;-)