When it comes to GPUs, the "cloud vs on-prem" debate is resolved. The answer is just do both.
While that sounds crazy, it is now possible.
The first part of the story is an emerging class of GPU enabled "data science workstations", i.e. evolved & leaner versions of gaming & design machines. For roughly the price of a high end Mac, vendors (Dell, HP & Lenovo ) now offer laptops with a powerful NVIDIA GPU that you can use whenever & wherever you want.