It's pretty easy. Don't let the length of my reply worry you, I just have a number of suggestions to offer.
If the package/envelope weighs under 15 ounces you don't even need a customs form for most countries.
For packages under 4 pounds, you fill out a little slip with your name/address, the buyer's name/address, what the item is, how much it's worth, and whether it's a gift or merchandise. Some buyers may ask you to mark it as a gift or indicate lower value so they don't have to pay so much when they pick it up from customs. It's up to you to decide if you'll go along with their request.
For packages over 4 pounds you fill out a slightly larger form with the same information. Only difference is that it's a 2-sheet self-carbon form so one copy stays with UPS and one goes along with the package.
If you do anything wrong, the post office clerk will tell you what to do right, so you can't really make a mistake.
BTW you can pick up blank forms in advance on your next post office trip so you can have them on hand to fill out. It's easier to fill them out at home than when you're in the post office standing in line and juggling packages. Also, you can just scrawl the information, you don't have to worry about legibility - you've got an address label on the package. The UPS keeps these on file only 30 days and then throws them out.
The rules vary slightly from country to country. You can go here to figure out international postage and find out any other conditions about shipping to specific countries:
http://ircalc.usps.gov/The way I would indicate shipping costs on an auction like yours would be to provide a set price for 1st class shipping in the US, and tell international bidders that they will pay for airmail shipping to their country. (Remember when weighing stuff to include any packaging, like cardboard reinforcing, etc. It can add a few ounces that can increase the cost more than you might expect. If you don't have a postal or food scale at home, take one of your items with packaging to the post office and check the weight there beforehand.) I would give them the option of contacting me before bidding for an estimate of shipping costs. Little items like yours will cost anywhere from $3.10 to airmail to Canada to $8.40 to airmail to Japan. UPS also offers Global Priority flat-rate envelopes - it's like Priority mail here: if your stuff will fit in the envelope, you can ship for a flat rate (usually $5.00) and save your buyer a little money.
I recommend either of these methods over surface mail, which costs half of what airmail costs but takes a lot longer to deliver. Airmail is 7-10 business days, surface mail is 4-6 weeks. Buyers who ask for surface mail often start emailing you after a month complaining that their purchase hasn't arrived yet, and you have to reassure them it's on its way. UPS has never lost any of my packages and I have shipped to every continent but Africa.
The other thing about international shipping through UPS is that you can't insure the item like you can for domestic shipping. You can buy something called registration for it, it costs around $7.50; I just skip it unless the item has sold for a whole lot.
This may all sound like a hassle but it's really not much harder than domestic shipping. The great thing about offering collectibles to international bidders is that it's harder for them than for domestic buyers to find US items. So they'll often bid on things that US buyers yawn at.
I suggest you get PayPal or other online payment account and insist that international buyers pay online. It's free for them to use; they'll charge you a small amount so a lot of sellers don't use it. But otherwise you will wait quite awhile for your buyer to mail you payment. If that doesn't bother you, accept only international money orders or well-concealed US currency. I also have found lots of domestic buyers prefer the PayPal option.
Finally, take a look at current auctions for items similar to the ones you want to sell. They'll give you an idea of what your opening bid should be, and payment/shipping language that others use in their listings may be useful for yours. Also, check eBay's forums in the Community section. There are forums dedicated to every aspect of buying and selling on eBay, and you will probably find good advice there.
Good luck, and have fun!