Common or Simple Biological Specimen
Equipment aside, getting biological samples can be tricky. If you need them alive there are only a few suppliers that sell to individuals. Skulls and bones are easy enough but getting your hand on some rare microbe can be a pain.
Caroline Biological carries a fairly wide range of materials. Most is aimed more for schools but they do have a good assortment. They carry a variety of preserved organs, animals, fungi, bacteria and more. They're usually a good place to start.
The Thought Emporium, although still new will be carrying a variety of bioluminescent specimen as well be one of the few places to sell tardigrades. Their list of biological specimen is continually added to and if there's something you need feel free to send them an email and they can either help you source it or if it's an interesting sample they may even add it to their stock.
ATCC is a more professional supplier and carries a wide range of cell lines, microbes and other biological specimen. They may not ship to individuals but their collection is one of the best. They have thousands of cell lines for various species as well as a massive bacterial strain collection. Their prices can be a bit high but you're paying for quality.
DNA and Nucleotides
This is a much more complex problem. There are a huge variety of companies that sell various genetic pieces. Some specialize in oligonucleotides while others will carry larger segments, plasmids and more. Your best bet is to type in exactly what you're looking for in Google and search around a bit. A more useful list will be compiled here eventually as the community moves towards biological mods.