Most hard drives, have multiple layers of magnetic material to write to. So when you overwrite an area of the disk with new data, some of the old underlying data might have a chance to be around (magnetically), even though it's removed from the file table entries and there are new bytes written to that area, there are still some (expensive) methods of recovering some data (not all data, and not necessarily reliably, but a chance enough for people to try it).
To counter this paramagnetic effect, wiping utilities will overwrite the entire disk with 1's or 0's (to flip all bits on/off) and then possibly random data to ensure any underlying data is fully overwritten. It should be noted that there are various standards as to what is deemed appropriate for data removal from magnetic material by various entities.