# scoop.it bot

The scoop.it bot is a web crawler for the content curation and social publishing platform Scoop.it. Its purpose is to discover and index web content that is relevant to the topic-based collections curated by Scoop.it users. For website owners, having your content 'scooped' can increase visibility and drive referral traffic from audiences interested in your niche.

Breadcrumb navigation

- [Privacy-focused, simple website analytics](https://plainsignal.com/)
- [Agents](https://plainsignal.com/agents "Agents, User-Agents, Crawlers, Browsers")
- [scoop.it bot](https://plainsignal.com/agents/scoop.it-bot)


## What is the scoop.it bot?

The scoop.it bot is a web crawler for the content curation platform Scoop.it. The platform combines automated content discovery with human curation to create topic-based collections. The service employs several specialized crawlers to collect content from websites and RSS feeds. These crawlers identify themselves with user-agent strings such as `Mozilla/5.0 (compatible; scoopit-crawler/3; +https://www.scoop.it/bot.html)`. The bots are designed to be transparent and respect standard web protocols.


## Why is the scoop.it bot crawling my site?

The scoop.it bot is crawling your website to discover and index content that may be relevant to the topics being curated by its users. Your site may be visited if a user manually adds your content to their collection, if your RSS feed is being monitored, or if your content has been algorithmically identified as relevant to an existing topic. The frequency of visits depends on user interest and your content update schedule. This crawling is generally considered authorized as long as it respects `robots.txt`.


## What is the purpose of the scoop.it bot?

The purpose of the scoop.it bot is to support the Scoop.it content discovery and curation platform. For website owners, Scoop.it can provide additional visibility and traffic by exposing your content to new audiences interested in your topics. When a user 'scoops' your content, they create backlinks and social sharing opportunities that can drive referral traffic. The platform preserves attribution by maintaining links to the original source. However, as with any aggregation service, there are considerations about how much content is displayed on the platform versus what drives traffic to your site.


## How do I block the scoop.it bot?

To prevent the scoop.it bot from accessing your website, you can add a disallow rule to your `robots.txt` file. There are several user-agents for the service, so you may need to block them individually.

To block the main crawler, add the following lines to your `robots.txt` file:

```
User-agent: scoopit-crawler
Disallow: /
```


## Related agents and operators


## Canonical

Human friendly, reader version of this article is available at [scoop.it bot](https://plainsignal.com/agents/scoop.it-bot)

## Copyright

(c) 2025 [PlainSignal](https://plainsignal.com/ "Privacy-focused, simple website analytics")