Can someone tell me what the part in the red box is and should it be oriented to rub against the shaft?

Click to view attachment

Looks like the remains of the felt wik that should be inside the top shaft under the rotor.
Its purpose is to hold oil that lubricates the top shaft,most don't add oil to it very often and the shaft sticks up and the vacuum advance won't work.....common issue.

I just checked another distributor I have on the shelf, and I can see that piece in there, I can tell that mine doesn't touch the shaft either.

But I can't imagine any purpose other than to lubricate the trigger point cam. If I were where you are, I would have the felt pad lightly touch the shaft and lubricate with oil.

Where does one find replacement felt pads? I've got a disti missing the pad, and I have no idea what material/thickness is suitable.

Gary