They are
Heat:
Heating a magnet above the Curie Temperature (the temperature above which the magnetic properties of a material change from ferromagnetic to paramagnetic) causes the magnetic domains to be disrupted permanently. Mild heating causes a reduction in the magnetism. However, when it cools the full magnetism is restored.
Mechanical Shock:
A magnet that is subjected to shock such as being hit by a hammer or dropped from a height can lose its magnetism. However, modern magnets made from materials such as Samarium Cobalt and Neodymium can withstand shock.
An opposing magnetic field:
A demagnetising field or a field that acts in the opposite direction can also result in a loss of magnetism. Demagnetising fields are sometimes used to reduce the strength of a magnet to fit a specific application