Nanoparticle flooding technology has a good oil displacement effect in low permeability reservoirs，but the problem of nanoparticles agglomeration in aqueous solution and blocking small pores in the formation has not been well solved. Low-cost oleic acid is used herein to surface modification of nano-TiO2 for reducing the agglomeration of nanoparticles，and the oil displacement system was optimized through low-permeability core simulation flooding experiment. The results showed that when the molar ratio of oleic acid to nano-TiO2 was 1∶1 and reacted at 60℃ for 4 h，the obtained modified nano-TiO2 had the longest stabilized dispersion time in aqueous solution. Infrared spectroscopy confirmed that the oleic acid group was successfully grafted onto the surface of nano-TiO2 . SEM and Zeta potential indicated that the dispersibility and stability of the modified nano-TiO2 particles was greatly improved. The particle size test results showed that the average particle size of nano-TiO2 in aqueous solution was 246.7 nm. the contact angle of unmodified nano-TiO2 and modified nano-TiO2 system with mass fracion of 0.05% on the surface of hydrophilic slide was 29.95o and 81.44 o，respectively，and the interfacial tension value was 0.475 and 0.74 mN/m，respectively，which explained the mechanism of enhancing oil recovery of the modified nano-TiO2 was still mainly relying on changing rock wettability and reducing oil-water interfacial tension. As for the low peameability reservoir with peameability of 9×10-3～12×10-3μm2，the reasonable injection parameters of the displacement system was 0.1% modified nano-TiO2+0.05% OP-10，and the reasonable injection volume was 0.3 PV，at this time，the enhanced oil recovery reached up to 15%. The nano-TiO2 solution could not only reduce the pressure of the injected water，but also improve the recovery rate of low-permeability core.