Under the startup options, there is an option to browse for a different Application icon to be displayed on the title bar. However it has to be a graphic w/ a .ico extension. What program would I use to create a .ico?

You can simply use paint, open up paint under image/attributes, change to 32x32 pixels. Create your icon and save with file extension .ico (this is not in the ext)

www.download.com has demo versions of software that you can create icon files
( the demo versions have enough capabilites to create *.ico files and not have to purchase the software)

I use IconForge