Dielectrics are also insulators. But, more specifically, they are materials which can be polarized. In dielectric materials, the electrons are bound to the nucleus and have limited movement. When an external voltage is applied to the dielectric, the nucleus of the atoms is attracted to the negative side and the electros are attracted to the positive side. Hence, the material gets polarized. This is a key feature of a dielectric.
Thus a dielectric can be defined as an insulator that can be polarized. Thus all dielectrics are insulators, but all insulators are not dielectrics. A dielectric can thus store charge. This characteristic makes it very useful in the form of capacitors.
Dielectric substances conduct very little electricity but are good supporters of electric fields. They also dissipate very less energy, i.e. have low dielectric loss.