golf putting. I take the putter outside on my backswing. I’ve tried everything to stop. Any suggestions?

The slower I take it back the less outside, but then I forget to hit it hard enough.

I like to practice putting in an open doorway (mine is a bedroom closet). My butt goes against 1 part of the frame, my head rests on the other side and I use the bottom to swing the putter along without actually touching the molding. No way to take it outside, and you stay nice and still over it.
On the course you need to stop watching the putter head while you’re putting. Concentrate on a dimple on the ball, or look for a spot underneath the ball to focus on after the ball is gone. But the fastest way to steer the putter is to look at the putter head.