Sources for Common Chemicals (Sodium Hydroxide, HCl, Ferric Chloride, Etc.) There are many great sources for chemicals online, especially if you live in the united states.
United nuclear carries a wide range of chemicals and metals as well as various bits of lab equipment. Although their prices are a bit off for the latter their stock of chemicals is a fantastic resource.
Ebay is your next best option. If you use the local version of the site (.ca for canada, .com for the states, etc) you'll likely be able to find an in-country supplier so there are no between border restrictions. Alternatively you may also find someone willing to ship to your country. Just be sure to check seller ratings to make sure you don't get screwed.
Amazon also has a great selection of stuff. From lidocaine to most kind of metal in any form. You may have to poke around but sometimes you can find things not from the above options.
Everything Else If it's rare or not a commonly used substance your options are both more expensive and limited.
Sigma Aldrich while expensive carries most things. It can be prohibitively expensive at times though and so a group buy is sometimes needed to be able to afford something weird (Ce6 for example)