Newish pistol and first couple of times I shot it I didn't notice any issues.
Over thanksgiving, went out in the back yard with my brother and shot his P365 and my CPX-2 (I know - it's a cheap gun. Cheapest I own).

Both of us had the same issue with the CPX-2. It would shoot somewhat close to bulls-eye on the first shot and then each shot after that would move further and further to the right.

Any thoughts on a physical issue that would cause this ? If no help from you folks, I may just send it in to have them check it out at factory.

We both shot the 365 really well and had no issues with it, so don't think we were both doing anything weird regarding hold/grip etc. And all my pistols are double action only, so I'm used to the long hard trigger pull. (SP101 .357, 239DAK .40, etc.)

I am assuming you are right handed. If I remember the CPX-2 has a longer trigger pull than the 365. You may have been unconsciously changing your grip tension as the trigger is pulled shooting it. If you are yanking the long trigger you will pull to the right thereby causing the gun to roll slightly outwards to the right due to the way your fingers wrap around the grip of the gun.

Dry fire the gun and watch your front sight. Does it move? Live fire with dummy rounds to see if there is a jerk with the trigger pull and subsequent movement of the front sight.