Pre-training refers to the process of training a machine learning model on a large, general dataset before it is fine-tuned on a more specific dataset. This initial training helps the model develop a broad understanding that can be refined for specific tasks.