It turns out to be two factors:
1. Polar ice is melting.
2. We are storing water in different places and moving it around the planet using piping.
These factors redistribute the weight of the water around the planet, making the poles wander around.

We analyze space geodetic and satellite gravimetric data for the period 2003–2015 to show that all of the main features of polar motion are explained by global-scale continent-ocean mass transport. The changes in terrestrial water storage (TWS) and global cryosphere together explain nearly the entire amplitude (83 ± 23%) and mean directional shift (within 5.9° ± 7.6°) of the observed motion. We also find that the TWS variability fully explains the decadal-like changes in polar motion

Source: Climate-driven polar motion: 2003–2015