I haven't tried using carbide on my lathe so take it for what it's worth, but there are a number of disadvantages to carbide.
I'm not sure how fast your machine will turn or if it's got the horsepower to sustain recommended feed rates at higher speeds, but I can say that mine doesn't come even close.
Carbide doesn't like sudden changes in temperature, which means running full coolant is preferable. If you stall out your spindle which carbide, you are going chip your tool. In my experience running production lathes with indexable carbide tooling, it's just a different word as far as rigidity. HSS is going to be much more forgiving of chatter, any flexation. Also the nature of using turn screws to manually feed vs even a jog wheel much less cnc control makes hss seem more at home on a toolroom style manual lathe.
HSS can be sharpened easily in whatever grinder, carbide is going to be more difficult to sharpen. At lower spindle speeds of a manual lathe, you're going to chip and dull carbide bits a lot more than if you could go 3x faster, which is where carbide shines. There may be a sweet spot for heavy cutting if you are turning large material (high surface feet per minute) where a carbide insert tool would be handy, but it will probably come down to if your lathe has the heft to maintain that. Like I said, stall the spindle and you'd be flipping the insert.
We never ground carbide when I did production. With indexable turret style tooling, you flip the insert when it gets dull. Each insert has like 2 -4 sides, and they are expensive, like $20 for each insert - and you don't buy just 1.
So problems with carbide are need to run flood coolant ideally, not enough spindle speed, not enough spindle power at higher sfm, difficult/not possible to sharpen depending on what style you get, and more expensive.
Like I said, I don't have any carbide tooling for my little lathe. Based on my reading/research, it wouldn't be worth it.