Scattered data fitting is a big issue in numerical analysis. In many applications, some of the data are contaminated by noise and some are not. It is not appropriate to interpolate the noisy data, and the traditional least squares method may lose accuracy at the points which are not contaminated. In this paper, we present least squares with interpolation method to solve this problem. The existence and uniqueness of its solution are proved and an error bound is derived. Some numerical examples are also presented to demonstrate the effectiveness of our method.