Fingers crossed wasn't good enough. The stock swingarm is stuck tight! Since I'm putting on my alloy swingarm I really don't care about destroying the stock one. I just need to be careful not to damage the shaft and sleeve inside since I'll need to reuse them.
So I drilled a few holes and pulled the grease zirk. Now I have 4 holes open to the shaft inside. I filled the chamber with carb cleaner to loosen up all of the 32 year old grease and rust. I'll let it sit for the afternoon and try again with the mallet later.