Minor Spoilers below

The only way you can go to Skellige is by paying 1000 crowns to a captain who is willing to sail. The journey is not an easy one and the ship is boarded by pirates. After fighting them off then your character will wake up on a beach. After you finish the conversation then head back to the shipwreck . Behind the southern shipwreck then you will find the captains body, complete with the 1000 crowns you paid him.

Note

Once you complete a few quests in Skellige then the gold will no longer be there. You can still loot the wrecks though. There is a considerable amount of loot so it is worth doing. Also do not forget to go to the Place of Power nearby.