Today the crawler of the search service of my MOSS-Development system stopped working from “one second to another”. The crawl-log showed me this error.
The crawler could not communicate with the server. Check that the server is available and that the firewall access is configured correctly..
The start is essential
I reconfigured everything, I checked the database-security, rebooted the virtual machine, re-created the content-sources. Nothing helped …
After some digging I found out that the start page of that site-collection (default.aspx) throws an exception due to a faulty web part…
This can’t be… Can it?! I removed the faulty web part from the default.aspx and the start page runs normal … and so does the crawler!
Because of an erroneous start page the whole crawling process for that site-collection is not working!
Conclusion
Make sure that the “entrance” to your site-collections is wide open and totally working …
PS: Excluding the default.aspx via “Crawl rules” does not work. The crawler still stops.