I currently run my website on a single server with a single MongoDB. I am moving to Amazon EC2 and building a horizontal scale-out model where more web-servers are dynamically started up. This poses a challenge for how I use MongoDB. I have a crawler/indexer that read/writes to the DB and a web-server that read/writes to the DB. I want to change my architecture so that I have 3 server types (crawler, DB, web machines). This task is to build the MongoDB on Amazon EC2 so that it scales horizo…
I want to spend about $60 on this. I don’t think it will take more 1.5 hours if you work straight through.
My Goal:
I want to quantify how many individual listings are under each category “world wide” for this page: http://expertpages.com/all_top.htm I am not concerned with duplicates. I just want a list of URLs, so I can run a count function in excel and get an idea of where the most listings are in this directory.
What I need this script to do step by step:
1. Crawl each cate…
Categories: Crawler, Data Scraping, PHP, Programming, Script, URL Tags: scrape, scrape urls, scrape urls from, Script, urls, urls from, Website
We got the kaon price comparison script and need a skilled and experienced programmer who already worked with the script before to help me setup the categories and products
The website category structure must be set up according our example.
We need a cronjob to automate all settings , means daily update of all merchant feeds and products or setup a descent price crawler for this.
We also got the filter options, so need to set this up too, this can be automated based on keywords in des…
Categories: Automation, Crawler, Cron, Installation, Long Term, Programming, Script Tags: comparison, comparison script, Crawler, help setup, Script, setup, setup comparison
I need a web crawler that will eventually crawl all web sites in the world and index the home page. There should only be one page per url and then it should go to the next URL. The data must be stored in a SQL Server 2008 database so that it can be searched.
I am not sure where to get the starting point for the web crawler, but I would imagine that there would be many places to get urls that can act as the seeds. If you are bidding on this project, please know where you will get the seed w…
I am looking for somebody who can develop a fully operational website (vertical crawler) which crawls selected auto classifieds websites.
So when somebody comes to my site, he should choose what vehicle, model, year, mileage, fuel etc. to search, then the site should return the related results from the sites defined.
Basic information should be displayed and order by criterias like price, year,…
if a user clicks on the information he will be pointed to the original website and offer.
…
Given this directory of conferences:
http://tinyurl.com/gbrbr
The script must crawl the directory and extract the fields:
Name of conference, subject, location, dates, website and description
Script must be a command line, no user interface. Use curl. Output to a csv file
Do you love to surf the Internet? Then this job is maybe for you!
I am looking for someone who is:
-An internet pro who knows how to research and crawl such internet resources
-Able to work smart, be able to identify and seek relevant sources for such information
-Curious and Enjoy data mining and information extraction from the web
-This position is result driven and pays well. Great compensation for the successful applicant/s
-This a part-time job, so payment is contracted pe…
Categories: Crawler, Data, Data Entry, Data Mining, Research Tags: Data, entry, Internet, job, mining, Research, Web
I have a problem with a website where Google Webmaster Tools is showing pages as not found – 404 (Not found) – when the pages actually exist.
The site in question is http://www.merlinpestcontrol.co.uk
Attached are the reports from Webmaster Tools (Crawl Errors, Crawl error Sources).
I require to know:
1. Why I am getting this problem
2. What I need to do to sort it (now and in the future)
3. What robots text I should be adding to pages to ensure site is most accessible to Google.
…
Categories: Bot, Crawler, Google, Search Engine Optimization, Webmaster Tags: 404 problem, Error, Google, google webmaster, pages, problem, Webmaster
Hi I need a quick job as below:
I have a 3 html pages website
http://graphic-design-perth.com.au
WHAT YOU HAVE TO DO:
1. Copy this website to my another 84 domains (domains info will be provided)
2. Each Domain Title/keyword/description will different ( info will be provided)
3. First paragraph on home page will be different in each website ( content will be provided)
4. As you can see i have 4 services, Design, Print, Website, Marketing
what you have to do is shuffle …
Categories: Contact Form, Copy, Crawler, Google, HTML, Marketing Tags: domains, HTML, html website, Marketing, pages, provided, Website
We need a reliable developer to parse a large source of data in a few days.
The source is a website similar in structure and nature to tripadvisor.com
We need to crawl all pages like this one:
http://www.tripadvisor.com/Hotel_Review-g32655-d595189-Reviews-Garden_Cottage_B_B-Los_Angeles_California.html
And extract all relevant information (will be provided in the SPEC). The results should be organized in 1 or multiple Excel (xls) files.
Preferably the parser should be done as a separat…
Need a full service property information website and portal.
As an information service portal, the site will:
- have dynamic content with users able to create & update content, blogs, discussion forums, groups, etc in real-time
- be able to crawl other sites for property news, data, etc; as well as generate its own data, tools and information resources all arranged in a pre-defined logic
- email and phone alert service to members (e.g. property price change alerts, etc)
- store database o…
Categories: Blog, Crawler, Editing, Google, Integration, PHP, Portal, Real Estate, Search Engine, Search Engine Optimization Tags: Content, engine, information, Portal, property, Search, service
I would like a php crawling script made.
I need URLS taken and placed into a mysql database
Categories: Crawler, Data Extraction, Data Scraping, MySQL, PHP Tags: Crawler, Data, data scraping mysql, extraction data scraping, MySQL, PHP, php crawler
Below are very specific instructions. Please do read and only bid if you agree.
1. Use any programming language you want.
2. I want the source code.
3. I want to have the algorithm installed and ready to go on my server. Any hosting service is fine as long as I own the domain and hosting. (I prefer bluehost.com or arvixe.com)
Here’s how the crawler should work:
4. The user uploads a CSV file with, for example, 10,000 rows of keywords, one keyword phrase per line
5. The user specifies …
I need a browser of desktop scrap script for the website http://www.qoc.de/plaintext
I will require a demo for escrow. Smallest bid with a demo wins the project.
Feel free to contact me if u need any further information
Regards
Timon
I need a script or set of scripts to get the most viewed youtube/vimeo videos for a certain set of keywords. The script should tell me the number of views of each video and get the description for each video from the website.
Categories: Crawler, Multimedia, Programming, Python, Script, Video, YouTube Tags: Script, top, top videos, Video, videos, vimeo, YouTube
I need a script or set of scripts to get the most viewed youtube/vimeo videos for a certain set of keywords. The script should tell me the number of views of each video and get the description for each video from the website.
Categories: Crawler, Multimedia, Programming, Python, Script, Video, YouTube Tags: Script, top, top videos, Video, videos, vimeo, YouTube
Hello,
I’m looking for a company to work in the long term with. I had posted last June/2011 to find some prices for a company to help me get my clients to the first pages of Google, yahoo, Bing. I would like to use my business name also as the SEO provider, so I would need your company to remain unknown to them. Please let me know your prices and in detail what you can do for me. Dont just place…I can do it!
Thanks for your time and good luck.
ON PAGE OPTMIZATION
Keyword URL Mapping
Tar…
Categories: Article Writing, Blog, Crawler, Geo Location, Google Analytics, Long Term, Mapping, Sitemap, Social Bookmarking, Wordpress, Writing, XML Tags: company, Google, Mapping, reseller, Search Engine Optimization, term, Writing
Hello,
i need a SOFTWARE OR SCRIPT THAT CAN CRAWL A SITE AND EXTRACT ALL MOBILE NUMBERS AND EMAILS from IMAGES !
emails and mobile numbers are shown in images
It should be anonymous
Regards
We need a reliable developer to parse a large source of data in a few days.
The source is a website similar in structure and nature to tripadvisor.com
We need to crawl all pages like this one:
http://www.tripadvisor.com/Hotel_Review-g32655-d595189-Reviews-Garden_Cottage_B_B-Los_Angeles_California.html
And extract all relevant information (will be provided in the SPEC). The results should be organized in 1 or multiple Excel (xls) files.
Preferably the parser should be done as a separat…
Hello,
i need a SOFTWARE OR SCRIPT THAT CAN CRAWL A SITE AND EXTRACT ALL MOBILE NUMBERS AND EMAILS from IMAGES !
emails and mobile numbers are shown in images
It should be anonymous
Regards
I need a browser or desktop tool to scrap information from a flash website.
the lowest bid with a Demo wins the project
Regards
Timon
Need products scraped from http://tinyurl.com/6n8x4ut and installed onto my site but i only need the sub categories inside cheap jordans category such as air jordan 1, 2, 3
Categories: Crawler, Data Entry, Data Scraping, E-Commerce, Installation, MySQL, osCommerce, PHP, Website Tags: Data, images, install, osCommerce, products, scrape, site
We want a tool that searches a specific web page for some specific information This webpage are normally very small sites with maybe 10 tot 20 pages It is not automated, a user puts in a webpage and then it needs to show the information, see mock up. We then will use the info and store it in our application. It needs to run on a web server.
Search voor the word kvk or k.v.k. or Kamer van Koophandel and retreive the number behind it
Search for the word BTW or B.T.W. And retreive the number b…
Categories: .NET, Crawler, Javascript, jQuery, Search, Webpage Tags: Bot, crawl, not bot, Search, search not, Webpage, webpage crawl
Hi there
i search for some one that can clone Vusker.com
it can be practacly the same
it must have a backend admin
to see gallery’s and see reports etc
and a ilegal gallery report button i main site
it must be run on a dedicated server
with php and mysql
the source is based on the old fusker technology
so that :
Create a new fusker. There are two ways to do this: a) Enter the URL (web address) of a web page that has direct links to images. This works well with TGP pages
Categories: Clone, Crawler, Internet Explorer, MySQL, PHP Tags: Clone, com, com clone, MySQL, PHP, vusker, vusker com
I am looking to extract data from this two website’s search results
a) http://bit.ly/tNpujL 707 pages
b) http://bit.ly/ucoVoF 90 pages
The data I need are as below for search results from http://bit.ly/tNpujL :
a) Name of the real estate agent
b) Phone number
c) Email
d) Total listings
e) Locations covered
The data I need are as below for search results from http://bit.ly/ucoVoF :
a) Name of the Realtor
b) Estate agent license
c) Phone number
d) Fax number
e) State
f) Postal…
Categories: Crawler, Data, Data Scraping, Real Estate, Spider, Web Scrapping Tags: bit, Crawler, Data, estate, scraping, Search, Web
I have several thousand sites i want to scrape for email addresses.
i will provide an excel with the url’s.
I need an excel sheet back with all associated email addresses for each url in a separate column. For example
URL Email
www.sample.com info@sample.com; sales@sample.com
We are looking for an experienced programmer that can scrape every app on the Android Marketplace for specific data on the app info page. The end result will be a mysql database or csv file for each app that will show specific detail about the app. Along with that, we want the script to continuously check for new apps (on a daily basis, using cronjobs) and save new apps into the database.
We will give full details in a private message after reviewing your previous work history
Here is wha…
Need someone to use the following template and install to oscommerce. the clone needs to be identically installed without products
http://osc3.template-help.com/osc_21466/index.php
i will have person install 3 oscommerce modules, edit some header & title tags, and edit the checkout process
need someone to clone and install all shoes at…. http://tinyurl.com/6wwpbuw (need all the product colors removed from the description and append all products with a 5 digit item number
I PREFER…
Categories: Checkout, Crawler, E-Commerce, Editing, HTML, MySQL, osCommerce, PHP, Search Engine Optimization, Template Tags: Checkout, creation, install, osCommerce, Template, title, Website
Integrate the WordPress site with Facebook but facebook should not load at same time as site, deferred.
Integrate Skype that when people click on our skype name a skype telephone call or message is initiated
WP Index.php page should look like the current HTML destinations page with text boxes and pictures.4 columns with 6 text boxes per row makes 24 text boxes countries should be displayed in alphabetical order
Each Country Category (Country Packages) should collapse with the lastest 5 …
Categories: Blog, Contact Form, Conversion, Crawler, CSS, Facebook, Flash, Google, Integration, Javascript, Landing Page, OpenX, Optimization, PHP, Plugin, Sitemap, Social Networking, Widget, Wordpress Tags: boxes, Facebook, Optimization, page, skype, text, Wordpress
This is a simple job for a programmer with the write skills. Our site is written in PHP although the site displays as ASPX it is definitely in php and the address has been changed using mod rewrite.
I am wanting a script written that will check every page of our site using copyscape.com
If a page on our site appears to be duplicate content the page will be flagged and able to be viewed on a separate page of our admin system. The script will need to search copyscape in such a way that our …
Categories: CMS, Crawler, Editing, Google, Modification, PHP, Programming, Rewriting, Script, Script Installation, Search, Writing Tags: com, copyscape, page, PHP, Script, Search, site
This project is for a Coupon scraper of 3 of the largest coupon websites, including retail-me-not.
Scrape Stores
1. Description
2. Store logo/Screenshot
3. Store URL
4. Average Savings amount (If applicable)
Scrape Coupons
1. Coupon Title + actual Coupon Code (If applicable)
2. Store Name
3. Destination/Deep Link URL (Stripped of affiliate ID)
4. Expiration Date, Posted date (if applicable)
5. Tags
6. Categories
7. Coupon Type (printable, web, etc)
8. Average savings amou…
Categories: C/C++, Coupon, Crawler, Curl, Data, Data Mining, Data Scraping, Graphic Design, Logo Design, PHP, XML Tags: applicable, Coupon, Data, Design, mining, scrape, scraper
We need a small crawler or using Indeed api, to get total jobs for a company. all you have to do, provide us code which will get #jobs from indeed website.
We dont care if you do php or other script.
I have several thousand sites i want to scrape for email addresses.
i will provide an excel with the url’s.
I need an excel sheet back with all associated email addresses for each url in a separate column. For example
URL Email
www.sample.com info@sample.com; sales@sample.com
Thank you for taken the time to look at my job request.
My Website: http://www.reflexleague.com
This website is using the “Webspell Content Management System” which is “php” based mixed with html, css ect..
I am looking to have my website fully setup with SEO Optimization & Marketing. Within this process making the website fully valid via W3C and other markup tools.
Not limiting to the following:
ON-PAGE:
*Keyword analysis.
*Google Site-map Generation
*canonical issue sol…
Categories: Article Submission, Article Writing, Blog, Blog Commenting, Bot, CMS, Crawler, CSS, Google, HTML, Marketing, Optimization, PHP, Plugin, Press Release, Search Engine Optimization, Social Bookmarking, W3C, Writing Tags: Article, Blog, CMS, Optimization, Submission, Website, Writing
Requirements:
We need a robot to crawl the iTunes Podcast directory, and then
provide us with the name, website, and email address of every podcast
in the directory. (Where they are available.)
Specification:
1. The main podcast directory is here:
http://itunes.apple.com/us/genre/podcasts-arts/id1301
On the left hand side of this page, you can see all the podcast
Categories. On the right-hand side, there is a list of podcast names
for each letter in the alphabet.
Th…
Categories: Apple, Bot, Crawler, Data Scraping, Multimedia, Podcast Tags: data scraping multimedia, Directory, ITunes, itunes scraper, Podcast, scraper, scraping multimedia podcast
I need a PHP expert in SEO optimization on my site.
- original content
- 1 year old
- it has pagerank of 4 is based on http://www.prchecker.info
- based on google analytics, it keeps having crawl error on 404 page error. It has this type of error constantly.
I truly believe it’s all about the coding of the script.
Please message if you are interested. Thanks
Categories: Crawler, Google, Google Analytics, Optimization, PageRank, PHP, Programming, Search Engine Optimization Tags: Analytics, Error, expert, Google, Optimization, PHP, Search Engine Optimization
Develop an effective jobs aggregator site.
Not looking for RSS based scripts. Search engines should be able to crawl the job info. The site should be search engine friendly and fully optimized with atleast 100 backlinks from established one way pr 3 or above static pages/links (Must use and guarantee white hat SEO methods only. No backlinks from farms or other shady sites, reputable directories only).
Allow monetization through affiliate programs, contextual ads, banners etc
Categories: Backlinks, Crawler, Freelance, Links, RSS, Search Engine, Search Engine Optimization, Website Tags: aggregator, Backlinks, engine, Freelance, jobs, Search, site
I would like to rid my site of all errors including but not limited to;
1. Crawl errors – 404, 302 etc.
2. W3C validation errors
3. Google Webmasters errors
4. Clean up wordpress database (especially taking out deleted plugins)
5. others errors (if any)
At the end of the project there should be no errors whatsoever on the site – this will be confirmed by Google webmasters tool and W3C site.
Programmer is expected to have adequate experience and the only way to prove is reviews relat…
Categories: Bot, Crawler, Google, Optimization, Plugin, Programming, W3C, Website, Wordpress Tags: Clean, errors, Google, optimize, site, W3C, Wordpress
I need a bot to scrape / crawl ads from 3 sites in approx 6 or 7 categories. www.cityvibe.com/ http://backpage.com/FemaleEscorts/ And 1 or 2 other sites to be determined.
2. Then post the ads on my classifieds script (Via ADMIN)
3.The bot should not post ads that do not have phone numbers or email addresses. The bot should should post ads in appropriate location & category
The bot should post most recent ads. The bot should scrape and post pictures along with ads.
The bot should not repe…
Categories: Backpage, Bot, Classifieds, Crawler, Data Scraping Tags: Ads, Bot, classified, Classifieds, Crawler, post, scraper
This is a simple job for a programmer with the write skills. Our site is written in PHP although the site displays as ASPX it is definitely in php and the address has been changed using mod rewrite.
I am wanting a script written that will check every page of our site using copyscape.com
If a page on our site appears to be duplicate content the page will be flagged and able to be viewed on a separate page of our admin system. The script will need to search copyscape in such a way that our …
Categories: CMS, Crawler, Editing, Google, Modification, PHP, Programming, Rewriting, Script, Script Installation, Search, Writing Tags: com, copyscape, page, PHP, Script, Search, site