I've determined it's the wire that runs between the two switches. Seems to work fine when you roll it up or down from the passenger side. I'd put it at close to factory speeds. 10.45v during operation, even with the truck not running. It's only slow (up or down) when operating the window from the driver's side. Everything is new, even the switches. The relay mod has it's own fuse box. I did everything cause I was also doing relays for the headlights at the same time. Got the manual off another site that tells you exactly which wire is for what, so that made it a lot easier.
Gonna be a good while before I feel like running a new wire between those two switches. I may wait until I hand the truck over to the shop for a medium restoration. "Here, you fix it"