How Difficult is it it learn Spoken Mandarin Chinese (not how to read and write)?

I’m 18 and don’t want to overwhelm myself too young, but want to learn spoken Chinese. Is it much easier than written Chinese? Are the four tones complicated?

Thanks,
Brian