Url Scraping
Require small php webapplication.
Input:
- selectable (checkbox): alexa.com, google.com, yahoo.com, bing.com
- next to each from above: number from 10 to 10′000 URLs (movable indicator would be nice)
- selectable (checkbox): organic, sponsored
- selectable (radio button): scrape domain only, scrape entire URLs
- textbox for search keywords (accept google type formatting like “” for phrases, 1 keyword per line)
- output selection (radio button): with http://www. , with www. , without
Output:
- scrollable textbox holding all URLs, one per line
Function:
- go to each selected search-engine (incl. Alexa)
– for each keyword scrape URLs according to input selections
- eliminate duplicates
- use built in proxies, if necessary for speed
Done.
Provide 2nd quote for same with desktop application (must run on Windows, may run on Mac)
Contact me if you have any questions.
