I need to download all PDFs (or any other files) available on a given website. I have a separate download module to which I provide a link to a pdf file and it downloads the file.I need a tool that can crawl through a given website and extract hyperlinks of all the pdf files available on the site so that I can send them to my download module one by one and download only those files which match a particular pattern. Is there any such crawler (front end or back end) available free of cost? Even a paid crawler would do and even if it extracts all the hyperlinks available on site, no matter if the links correspond to a downloadable file or not.
↧