undefined | Better HN

0 pointsim3w1l5y ago0 comments

Wouldn't it be easier to first do your crawling and then construct the matrix in one go? But graphs more broadly sounds like an area where you might want to update the matrix yeah.

0 comments

amelius5y ago

Well, one exception could be that some parts of the web need to be crawled more often than others.

But tbh, I think incremental updating of a matrix only makes sense if the other parts of your pipeline are also incrementalized.

j / k navigate · click thread line to collapse

0 comments

amelius5y ago

Well, one exception could be that some parts of the web need to be crawled more often than others.

But tbh, I think incremental updating of a matrix only makes sense if the other parts of your pipeline are also incrementalized.

j / k navigate · click thread line to collapse