Heavy metal pollution has become one of the crucial global environmental problems with a growing threat to the environment and human health. Therefore, research on the prevention and treatment of heavy metal pollution has attracted extensive concern. Among them, heavy metal pollution in urban rainwater runoff has become a research hotspot. Rainwater runoff is a major pathway for transporting nonpoint-source pollutants or sediment from the watersheds to other environment. In this study, the Soil & Water Assessment Tool (SWAT model) was used to assess the pollution load of manganese in the rainwater runoff discharged from a manganese mine. Results showed that with the modified SWAT model, it exhibited a higher prediction accuracy in pollution load of manganese with coefficients of R2 and Ens greater than 0.6 and 0.5, respectively. According to water quality standard for surface water in China, the key contaminated areas corresponding to the pollution load of manganese were obtained, which also reflected an actual situation that the most seriously contaminated sites were located in the mine aperture and mine tailing. This study demonstrated that the modified SWAT model was a promising tool for analysis of the pollution load of manganese in rainwater runoff from a manganese mine.