BlackWidow is an internet tool that will allow you to carry out various functions, among them "sniffing" a website for downloadable content, scanning for email addresses, creating site maps and detecting errors.
The app carries out these tasks via an in-built browser and a series of filters. The BlackWidow interface is average, but it's not difficult to find your way around - find the website using Browser, set the Filters, start the Scanner and, when you're done, investigate the results under Link Errors, Emails, Structure and NetSpy.
While casual users might want to use BlackWidow to access hard-to-reach content they want to download, it's more a tool aimed at developers who want to mirror websites and investigate them thoroughly. It's easy to use, but of course, you need the knowledge to interpret the results.
If you are capable of figuring them out, you'll find that BlackWidow has several useful tools, among them the ability to export both the contents and the structure of websites, a snapshot function and the ability to view source code. It's not a slick application, but if you want to get to the bottom of a website - or proxy/FTP - BlackWidow is a great start.
BlackWidow will help you get an in-depth view of a website, its structure and content.