Url Address Collection
I need to collect a large number of web page (URL) adresses that are generated as a result of a typical manual Google search.
I cannot use a crawler that does an automated search and collection on the Google site. Google will flag that as an automated query.
What I need is to be able to do is go onto the Google site, manually put in some search terms, then hit the search function and have the Google search results come back to my computer screen the way a manual Google search presents itself.
After that, I need some kind of a crawler that can automatically start at page one of the manually generated Google search results and copy the URL addresses. After that, the crawler should automatically go to page two of the Google search results and copy the URL addresses, then go to page 3 of the Google Search results and copy the URL addresses, and so forth.
The typical Google search result has about 950 to 1,000 useable search results, so that would be the largest file that would be generated by this process.
These files would in effect be a file of URL links that were generated through manual Google searches.
