There's nothing inherently slow about selection on psychological traits. To give just one striking example, there is extremely rapid selection against intelligence in advanced societies going on right now, at a rate of about 0.25-0.3 standard deviations per century.
US: https://www.pnas.org/content/suppl/2016/05/25/1523592113.DCS... (Table S11)
Taiwan: https://www.sciencedirect.com/science/article/abs/pii/S01602...
Iceland: https://www.dropbox.com/s/9vq5t6urtu930xe/2017-kong.pdf