Since it only happens when you get on the brakes hard, I think you can rule out a stuck caliper or similar problem. I would double-check that you don't have worn out pads, maybe try swapping pads from side to side.
After that, I would be looking for anything that lets the front suspension deflect under a heavy load to the rear. When I had a TR3A that pulled like that, it turned out that the rear lower suspension mount (on the RH side IIRC) had pulled loose from the frame. The front mount was still holding the pivot in place, but the pivot rod would bend under heavy braking, allowing the suspension to deflect and the steering to turn the wheel to the inside.
I had just gotten the car running and drove it that way for a few weeks, holding the wheel hard when I had to stop; until someone pulled out in front of me and I couldn't keep it from jumping the median! Fortunately I hadn't been moving very fast and got stopped before it veered into oncoming traffic, but the experience convinced me it had to be fixed sooner rather than later! Oddly enough, the alignment shop had not spotted the problem.