First, design your logo. I use Inkscape so I can later scale the icon for tshirts and printed material, but any tool will do as long as you end up with a square flattened 512x512 PNG. Open this in GIMP.
Next download this GIMP template and open it in GIMP. Copy/paste your icon into the already selected region of the template, it should "just work". Anchor your selection and merge visible layers for your 512x512 icon.
Now that you have your glossy icon with rounded edges, scale and save the image three times for 57x57, 72x72, and 114x114 as follows:
Also save your original (pre-template) logo at 57x57 as
apple-touch-icon.png. These four files should cover iPhone, iPad, Android, and Blackberry. You may also want to save either the original logo or the glossy version at 16x16 for
favicon.icoif you have not done so already.
Upload these files to your web server saved in the root directory of your website (like
robots.txt) and add the following template to the head of your website:
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon" />
<link rel="apple-touch-icon-precomposed" sizes="114x114" href="http://yourdomain.com/apple-touch-icon-114x114-precomposed.png" />
<link rel="apple-touch-icon-precomposed" sizes="72x72" href="http://yourdomain.com/apple-touch-icon-72x72-precomposed.png" />
<link rel="apple-touch-icon-precomposed" href="http://yourdomain.com/apple-touch-icon-57x57-precomposed.png" />
<link rel="apple-touch-icon" href="http://yourdomain.com/apple-touch-icon.png" />
That should take care of virtually every device that supports this feature. Test it out on your Android phone and, if you have any friends in the Cult of Mac, ask if you can check this on the half dozen iOS devices they own.
Of course you should also make sure your site works well on handheld devices, but that's a topic for another time.