If you right-click on a button, you can select the option of a Text, Graphic or both as a button.

Then, as I show you in my book (available on Amazon), you can swap in a custom graphic for any button by going to Button Properties and, on the Media page, replacing the Thumbnail Media (under Thumbnail Properties).