Does solr do web crawling?

I have been using Nutch with Solr on my latest project and it seems to work quite nicely.

If you are using a Windows machine then I would strongly recommend following the 'No cygwin' instructions given by Jason Riffel too!

I know this question is quite old, but I'll respond anyway for the newcomer that will wonder here.

In order to use Solr, you can use a web crawler that is capable of storing documents in Solr.

For instance, The Norconex HTTP Collector is a flexible and powerful open-source web crawler that is compatible with Solr.

To use Solr with the Norconex HTTP Collector you will need the Norconex HTTP Collector which is used to crawl the website that you want to collect data from, and you will need to install the Norconex Apache Solr Committer to store collected documents into Solr. When the committer is installed, you will need to configure the XML configuration file of the crawler. I would recommend that you follow this link to get started test how the crawler works and here to know how to configure the configuration file. Finally, you will need this link to configure the committer section of the configuration file with Solr.

Note that if your goal is not to crawl web pages, Norconex also has a Filesystem Collector that can be used with the Sorl Committer as well.