I need someone to write a web crawler to retrieve information from a particular website and post to another website. The crawler will need to search through the source website by a specific category and pull data to be loaded into the the target website. The website is written in PHP and the programmer will be given FTP access to do the work. The programmer will need to use PHPMyAdmin to remotely access the database files. Do not respond unless you have experience with writing such a script.
I need help regarding web crawler
Please PM me to get details
need a crawler for a well know ad website. it is not a email grabber.
need to search for keyword and negative keyword. display results be able to see results in another frame, be able to select results and send email to them.
need a crude email template management, be able to save searches, run searches periodically and update the results. need to be able to tell if an ad as answered previously.
need to be able to flag spam or ad post (if several ads are identical or very similar. (not necessary but preferred)
need to be able to select specific categories/town/countries individually.
Able to use the tor network to do the search since that ad website is very picky as far as how much searches you can do. (optional)
raw script no design necessary.
Categories: MySQL, Perl/CGI, PHP, Python Tags: Crawler, Email, need web, php python mysql, python mysql need, results, searches, Web, web crawler
We have identified a few hundred web directories to place our local business customers in to improve their web presence. However, we need more data to help qualify these destinations. Data to be crawled and added to the database includes:
1. PageRank of domain
2. Alexa Rank
3. Crawl Freshness (how often crawled)
3. Number of Inbound Links to domain (siteexplorer.search.yahoo.com)
4. Number of pages in the website (siteexplorer.search.yahoo.com)
5. Domain age (archive.org)
6. Domain name (and extension)
7. Domain history (ie: spam infractions/penalties, etc.)
CSV or XLS is fine as output.
Web Crawler
Require a web crawler to work on a linux server
Web crawler needs to be able to stay within certain process bounds
Web crawler’s function is to extract .uk domain names and email address only and deposit them into a MYSQL database.
The crawler needs to be able to start itself should it crash and should be stable enough to run weeks on end
Crawler needs to be able to keep a log of sites visited, spawn multiple spiders and kill of these processes after certain rules occur
Language is not important
I would like a simple desktop application that ‘crawls’ specific webpages in EBay and Sitepoint for info and links based one a set of rules such as ‘status = sold’ an summarizes the data. The preferred platform is Flash for later use with Adobe Air.
I require template-based scripts with fully commented code. I will not accept your bid, regardless of your abilities, unless the code word
