a few thoughts..
- make sure the blade is not following the angle of the nose when its going in, a common issue with beginners which immediately turns the board. imagine a line going from your rails on both sides straight past your nose, that's where the blade goes in.
- maybe just paddle on one side? either pull in from approx 1-2' away from the nose and into the rail kinda like a j-stroke or plan b, push one rail into the water slightly and paddle on that side. adjust your trim to find the sweet spot where this works for you so you're paddling straight. i can only do it on my left side.

-short strokes, pull out at your feet, sometimes going past your feet folks tend to push the blade out away from the rail as they remove it from the water thus pushing the board in the opposite direction.
- bigger or more effecient fin? check out casey gotcher's Gladiator fin.