Nutrition-rich cereal grains and oil seeds are the major sources of food and feed for human and livestock, respectively. Infected by fungi and contaminated with mycotoxins are serious problems worldwide for cereals and oil seeds before and after harvest. The growth and development activities of fungi consume seed nutrients and destroy seed structures, leading to dramatic declines of crop yield and quality. In addition, the toxic secondary metabolites produced by these fungi pose a well-known threat to both human and animals. The existence of fungi and mycotoxins has been a redoubtable problem worldwide for decades but tends to be a severe food safety issue in developing countries and regions, such as China and Africa. Detection of fungal infection at an early stage and of mycotoxin contaminants, even at a small amount, is of great significance to prevent harmful toxins from entering the food supply chains worldwide. This review focuses on the recent advancements in utilising infrared spectroscopy, Raman spectroscopy, and hyperspectral imaging to detect fungal infections and mycotoxin contaminants in cereals and oil seeds worldwide, with an emphasis on recent progress in China. Brief introduction of principles, and corresponding shortcomings, as well as latest advances of each technique, are also being presented herein.