Skip to content
Better HN
Top
Best
Ask
Show
New
Jobs
Search
⌘K
0 points
wejick
7y ago
0 comments
Save
Share
I think adding rule for /search* to your robot is easier way to block google bot
0 comments
1 comments · 1 top-level
top
newest
oldest
dougunplugged
7y ago
Don't do this until Google sees your new robots meta directives (or canonical tag) on these pages, that way they will drop from the index. Then you can add this to your robots.txt to prevent it being crawled again.
j
/
k
navigate · click thread line to collapse