862
The AI company Perplexity is complaining their bots can't bypass Cloudflare's firewall
(www.searchenginejournal.com)
This is a most excellent place for technology news and articles.
They do have a point though. It would be great to let per-prompt searches go through, but not mass scrapping
I believe a lot of websites don't want both though
Does it not need to be scraped to be indexed, assuming it’s semi-typical RAG stuff?
I don't see a problem here. Maybe Perplexity should consider the reasons WHY Cloudflare have a firewall...?