Hebdomadaire Shaarli
Semaine 04 (January 20, 2025)
Discover expert reviews of the best business tools at Whoishostingthis.com. From web hosting to e-commerce solutions, our trusted recommendations empower your online success.
Largest DNS record history database, with more than 2.2 billion nameserver changes detected, daily updated. Our premium bulk domain history checker allows to lookup up to 5,000 domains at once.
email provider
cobalt lets you save what you love without ads, tracking, paywalls or other nonsense. just paste the link and you're ready to rock!
Le site pour rétablir les flux RSS de Radio France
A list of AI agents and robots to block. Contribute to ai-robots-txt/ai.robots.txt development by creating an account on GitHub.
Pour ma part, j'ai rajouté cela dans le conf-enabled/security.conf de mon apache :
# https://raw.githubusercontent.com/ai-robots-txt/ai.robots.txt/refs/heads/main/.htaccess
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^.*(AI2Bot|Ai2Bot-Dolma|Amazonbot|anthropic-ai|Applebot|Applebot-Extended|Bytespider|CCBot|ChatGPT-User|Claude-Web|ClaudeBot|cohere-ai|cohere-training-data-crawler|Crawlspace|Diffbot|DuckAssistBot|FacebookBot|FriendlyCrawler|Google-Extended|GoogleOther|GoogleOther-Image|GoogleOther-Video|GPTBot|iaskspider/2.0|ICC-Crawler|ImagesiftBot|img2dataset|ISSCyberRiskCrawler|Kangaroo\ Bot|Meta-ExternalAgent|Meta-ExternalFetcher|OAI-SearchBot|omgili|omgilibot|PanguBot|PerplexityBot|PetalBot|Scrapy|SemrushBot|Sidetrade\ indexer\ bot|Timpibot|VelenPublicWebCrawler|Webzio-Extended|YouBot).*$ [NC]
RewriteRule .* - [F,L]