Get a Freelance Job

We want a tool that searches a specific web page for some specific information This webpage are normally very small sites with maybe 10 tot 20 pages It is not automated, a user puts in a webpage and then it needs to show the information, see mock up. We then will use the info and store it in our application. It needs to run on a web server.

Search voor the word kvk or k.v.k. or Kamer van Koophandel and retreive the number behind it
Search for the word BTW or B.T.W. And retreive the number b…

Hi, My google webmaster account shows “Crawl errors” and i need to rectfy the same. pleae let me know the way and i will implement the same myself . Also please see the attachement which shows the error screen

I want to email specific businesses about my printing products [canvas prints]. Businesses that have physical stores, like “Dentists” “Lawyers”, “Advertising Agencies”. They need to be in Australia.

I would prefer to grab them from the Australian Yellow Pages site, but if this too complex we can do it from google. Emails to be presented in excel, one column emails, the other saying type of business or keyword used.

If you have a better way to do this, let me know. Small budget but the work will be ongoing as we cycle through.

Hello, this project will pay $100 – $150 USD

We are looking for someone who can crawl the following:

1. Go here http://www.expocad.com/host/fx/convexx/sema11/default.html
2. Click this in red headline on right, Click here to view complete exhibitor list.
3. Very first business is 2.95 guys. IF you see it will go to a flash map of the show. I need the data for each company, about 20000 plus records in xls column format by company name, street address, city, state, zip, phone, contact name, email (if available) and web address.
4. For example, first record would read like 2.95 guys 13750 Stowe Dr.
Poway, CA, USA 92064 http://www.295guys.com 800536-5959 sales@295guys.com Lance Beesley

Thank you and please do not bid unless you really look at what is being asked of you. We have had to scraping companies win this bid previously, and both bailed out after they realized they could not provide the data. Do not waste either of our time. Check and see if you are capable to provide this data, and accurately. Thank you

We are a couple of researchers who need quick help for a small tool which can crawl the wall of Facebook groups, profiles and fan pages and transform this data into a csv.format or similar to be used for analysis in Excel etc. We need a crawler that is robust, so it can (in principle) handle several thousand wall posts.

Ideally, the crawler should crawl the pages we need crawled, if we feed it with the URL of the page in question. As an alternative, we can save the page ourselves in Firefox, then the crawler “just” needs to crawl the php.file that Firefox provides of the page.

More specifically, we need the crawler to create a csv.file that lists the following data in columns: Name of wall post poster, time and date of the post, textual content of the post, number of likes and number of comments (exact if less than 50 comments, “+50″ if more than 50 comments as Facebook now lists theses on an individual page). We would also like the name of all commenters on a post, time and date of each comment to a post, and the textual content of the comment in an affiliated csv.sheet if possible at all.

For further information, please dont hesitate to ask quetions here!

Hello guys,

I am looking for someone who can help us to remove all the crawl errors from our store, most are because of the ending; ?___store=XXX in Google webmasterprogramm

Also need someone who can help tweaking Magento for better results of our site on Google. Who can help?

Hi,

I need cron job scripts to be developed to crawl a list of websites I upload to the database.

Detailed description is available in the attachment.

I need to create a tool which will crawl predefined URLs and look for certain predefined keywords. If finds the keywords on the target webpage it should update the database accordingly. (MSQL Server 2000).

Budget up to $100 for the entire utility.

Details in PMB

I am getting hundreds of crawl errors in sitemap from the magento sitemap creator. I need someone to evaluate and fix this. Please contact me, and I will give you sitemap, and list of urls so you can evaluate before bidding.

Hello, this project should be around $50 or less in USD, We are looking for someone who can crawl the following:
1. http://www.aapexshow.com/aapex2011/public/exhibitorlist.aspx?aeid=10&ID=2434&sortMenu=105002 all pages
2. http://www.asashop.org/ a – w in each of these you need to go down a level to extract data.
3. https://tcms.aftermarket.org/personifyebusiness/MemberDirectory/tabid/143/Default.aspx each of the 9 codes (ie AEA)

I would like the data in xls column format by company name, street address, city, state, zip, phone, contact name, email (if available) and web address.

Thank you

Hello, this project should be around $50 or less in USD, We are looking for someone who can crawl the following:
1. http://www.aapexshow.com/aapex2011/public/exhibitorlist.aspx?aeid=10&ID=2434&sortMenu=105002 all pages
2. http://www.asashop.org/ a – w in each of these you need to go down a level to extract data.
3. https://tcms.aftermarket.org/personifyebusiness/MemberDirectory/tabid/143/Default.aspx each of the 9 codes (ie AEA)

I would like the data in xls column format by company name, street address, city, state, zip, phone, contact name, email (if available) and web address.

Thank you

Hello, looking for someone who can crawl the following:
1. http://www.aapexshow.com/aapex2011/public/exhibitorlist.aspx?aeid=10&ID=2434&sortMenu=105002 all pages
2. http://www.aapexshow.com/aapex2011/public/exhibitorsearch.aspx?aeid=10&ID=2435&sortMenu=105001 all pages
3. http://irce.internetretailer.com/2011/speakers/
4. http://irce.internetretailer.com/2011/exhibits/ A-Z and 0-9 at bottom of page

I would like the data in xls column format by company name, street address, city, state, zip, phone, contact name web address.

Thank you

Trying to crawl a website says … javascript disabled
i mean when i try to crawl it using a spider built in php i get javascript disabled and i get nothing out of the websites because the website needs javascript enabled so i can crawl it ….
the website it self has no javascript i am sure but it makes sure you have javascript so no one can crawl it … not because it needs javascript … i hope you understand … and also the website needs me to login before crawling …but i know how to log in into it using cURL extension
i am using php

i just need a function or anything that can fake that javascript is enabled …. i am using linux

Hi all,

i need a PHP crawler that do the following:

- Crawler have at least 3 important files: crawler.class , config.php , crawl.php (i prefer for every site a class (alexa, yahoo, bing, google))
- it should crawl websites in a specified format
- it writes all found data in the database
- crawl a site if timestamp in db is older than 7 days
- input list is a website in the format like this:
http://www.test.com
https://www.test1.com
http://www.test2.com
- HTTPS must be possible
- If a second scan runs and it detects a change, it dont delete the databse record. it creates a new record and only the newest is showed up.

What he need to crawl?

- Website Titel
- Meta Description
- Meta Keywords
- Server Banner (example Server: Apache/2.2.16 (FreeBSD) mod_hcgi/0.8.0 mod_ssl/2.2.16 OpenSSL/1.0.0c DAV/2)
- Alexa Rank (Traffic Rank 1 Month, World Traffic Rank, Review count, Average Load Time)
- Google (Pagerank, indexed Pages[site:www.test.com])
- Add wappalyzer and store the details also in our DB
- Whois Information (Nameservers, Owner, street, city)
- Ripe information (DNS Lookup ip, ripe database query)
- IPlookup of the Site

The Config File:
- possible way to activate deactivate classes ( Services )
- possible way to edit the Database configuration
- possible way to edit the expiring time of a domain
- my brain is bad today… so there could be more during the project

We need a crawler for getting news, tables and text on a web site.

These contents will be integrated our news system which uses php + mysql.

Here is what I want to be able to do with a single PHP-only script:

1. Enter a CL url like $url = ‘http://cleveland.craigslist.org/bik/’;

2. Get returned to me the 20 most recent postings for that link in this format:

a. $time
b. $linktitle
c. $linkurl
d. $email
e. $description

This script only needs to echo this is plain html so that I know it functions. This is about a 15 minute job so bid accordingly. If you exceed the max bid, you will be ignored.

Hello. I would need a little bot / script that would crawl:

1. crawl rate.ee for e-mails
2. put the e-mails into database
3. have a place where I can insert a text and design e-mail
and have a possibility to send this e-mail to all of those
people in database at once

In order to see e-mails in profiles
who have allowed it, you need to be signed in.

Username: psmaria
Password: takeit

Now some of the profiles have e-mails.
For example this one (once you have logged in):
http://www.rate.ee/userinfo.php?id=2039281
And from this kind of pages (userinfo.php) the
bot or script should go through all numbers and
put the e-mails into database.

Hello. I would need a little bot / script that would crawl:

1. crawl rate.ee for e-mails
2. put the e-mails into database
3. have a place where I can insert a text and design e-mail
and have a possibility to send this e-mail to all of those
people in database at once

In order to see e-mails in profiles
who have allowed it, you need to be signed in.

Username: psmaria
Password: takeit

Now some of the profiles have e-mails.
For example this one (once you have logged in):
http://www.rate.ee/userinfo.php?id=2039281
And from this kind of pages (userinfo.php) the
bot or script should go through all numbers and
put the e-mails into database.

I need a web crawler that will search for specific item in google, choose a single particular result link (I will provide details to the winning bid) and from the page pointed to by that link, extract a set of pictures. I would prefer the crawler to be written in Python.
You will also run this crawler for 5000 to 10000 items, save all the pictures locally and send them to me.

Write a script (Python, JavaScript, C#) to extract the Points of Interest for top 80 cities in the US using Google Maps API. (I will provide the list of cities) Each point of interest will have the following:
- name
- full address
- latitude
- longitude
- phone number (if any)

Hello. I would need a little bot that would crawl:

1. crawl rate.ee for e-mails
2. put the e-mails into database
3. have an opportunity to send one e-mail separately
to all of the people in the database.

I need a php script to scrape the contents from this site:

http://www.shopping.com/top_searches

Crawl each link and return the first 100 results from each. Store the results in a mySQL table.

Using the first one as an example this is the data I need:

2 Way Radios, 1, motorola walkie talkies
2 Way Radios, 2, motorola talkabout
2 Way Radios, 3, midland gxt900vp4
2 Way Radios, 4, motorola t9680r
2 Way Radios, 5, mile range walkie talkie
etc, etc

We renovated our website to joomla and after that our search engine rankings fell very low and our keywords too are not seen in google, when we saw the google crawl cache only half the page is getting indexed.need to correct this error.

I need an extra feature for the sphider.eu script. I want to be able to set it to crawl and index only URLs from a specific TLD. For example: I want it to index only .COM domains. The next day I only want it to index .NET domains. Etcetera.

I also want it to be able to crawl URLs with a specific word in them, for example only those URL’s with the word -net-. It will then index only the domains like for example: internet.com, netvibes.com, etc.

It should be possible to set a pause between URLs to crawl, for example crawl 1 URL, then wait 3 seconds. It should not use too much of my server resources nor from the server it is crawling.

It has to index the site title, description and keywords from the meta tags.

Hello. I would need a little bot that would crawl:

1. crawl all-links-from-dmoz.org; orkut; and myspace
for e-mails
2. put the e-mails into databases regarding what is
the ending (meaning: maria(at)email.de, girl88(at)mail.de
would go separately under Germany list, maria(at)email.be,
girl(at)mail.be under Belgium, and e-mails with .com ending
under Overall list)
3. have admin to send one e-mail separately
to all of the people in the database (for example Germany list).
The e-mail must be designable (having pictures in it)

Hello. I would need a little bot that would crawl:

1. crawl rate.ee for e-mails
2. put the e-mails into database
3. have an opportunity to send one e-mail separately
to all of the people in the database.

We need a crawler for getting news titles and text on http://www.guardian.co.uk/lifeandstyle/women

And will add a table on mysql

I want some scripts about crawling websites. I will send u details.

Hi ,
Im looking for a programmer , who will do this for me quickly and cheaper.

You will crawl articles from a website by Php Curl and add to mysql.

Simple and easy.

I need all products on air-randy.us cloned

Most products have 3+ views for each product so I will need all views available for all products.

Need a skilled programmer that can clone or crawl products. I do not want any data entry people because I need this done today.

products need to be added to my oscommerce website and added with different view options. Size attributes need to be added to all products. I only need skilled programmers that have crawl/clone experience.

Our site, www.govehicle.com, is a vehicle shopping marketplace. Car dealerships have signed up with us and they need to know that people are viewing their cars daily. Your job will just be to go to our site and view inventory. Our tracking tool will then show the number of hits to be increased.

For a better idea of what I’m talking about please visit www.govehicle.com and perform a vehicle search. Select any vehicle and view the detail page by clicking on it.

Daily we want you to spend 30 minutes browsing our cars. Please bid accordingly.

I need someone to create a desktop application which can automatically pull the embed code from all the new videos on pornhub, xvideos and xtube and post them to my website. It would make sense if it ran an update every 5 minutes or so and posted all the videos to my website. I need this to be automatic so aslong as the application is running it will pull the video, title, tags, and description and then add them to mine. I dont need it to download the videos just yet, but just embed them.

Like I said earlier I want it to pull the latest videos added to these websites.

I would like if the application could generate new tags from the title, and possible spin the title around a bit. (let me know if this is practicle)

This shouldnt be a hard project at all to complete.

I am using Adult video script which can be seen at www.adultvideoscript.com

I would like if the application used different user accounts to post the videos. Not a new account for every video but say for every 100 videos.

I need products crawled/cloned from royalsole.com and added to my website. i only need air jordans, and custom jordans category and sub category products. i believe there is no more then 600 items. royal sole is a oscommerce site and i need all product pictures and descriptions added from royaolsole.com to my oscommerce website. i need this done as soon as possible. serious bidders only. thank you

I need someone to crawl a website and add products to my existing site. I need images and product details crawled. I don’t believe there is over 600 items so Installing on my OSC site should not be difficult at all. I need this done today. I only want people that can crawl(NO DATA ENTRY BIDS) .

Serious Bidders Only please. Thanks

Hi,

I have a board here.
http://hi2pal.com/distribution

Google search doesnt show my topics there. anyone can sort out this.

need a script which will (scrape, spider, crawl) few websites, it will take input field from a file and post it and use the get. you can use either php or perl to do this. I dont really care as long as the script works.

1. people with higher ratings and low bids will be considered.
2. I will select few and message them to create a demo the one with successfull demo will win the bid.

Summary:

This project is a website archiver and spider. This archiver will store full websites it has crawled in our database. It will then crawl each website periodically, check updates to those pages (based upon our archived version), and store any pages with updated information (pages with changes).

Details:

In the database, there will be a list of URLs for the spider. The URLs would be similar to: www.scriptlance.com, www.domain.com.

The spider’s purpose is to crawl each website and retrieve it’s HTML code for replication (similar to archive.org). Every page of the website should be archived (i.e., crawl each page for internal URLs and then log the HTML to the database).

Once a website is fully crawled and archived into the database, subsequent crawls will be for the purpose of evaluating changes. If there is a change to any one of the pages on the site, then the spider will archive the latest version by storing the current HTML code.

The archiver will store the following from each page:

Page Title, Page URL, Date Archived, HTML Code (for the purpose of replication on our website).

Replication:
For our purpose, replication means that we will be able to use the information stored in the database (HTML Code) to show a version of this website to our users. The stored page should look near-exact to what the spider stored. If images are no longer around, that is fine. Take a look at the wayback machine at archiver.org to see what I mean by this.

Need to scrap data from a website – will provide URL for everyone who has an experience with data extraction from web to excel or text file. If you know how to download pictures, that would be the plus

Please read before bidding, and pm us to make sure you understand what we need.

We do not want to use any third party search engine or grabbers for this project. We simple need our own search engine for video sites only.

INFORMATION:
We have a website with a 100s of listed video sites and would like to create our own search that will crawl all listed video sites and show relevancy/results only for title, description, tags similar to ovguide. Each video site could have on average 10,000 to 60,000 plus video pages, and we want to make sure this spider/bot can crawl them all. Please exclude youtube.

Example site with specified keyword “terminator”: http://search.ovguide.com/movies_tv.php?q=terminator, as you can see in result page, it shows you multiple video sites/logos with keyword “terminator” using new technology relevis search engine: http://www.ovguide.com/relevis/ovg-relevis.ppt
Relevis is similar to google but only show results of video title, description, tags and nothing more.

Please don’t bid if you have 0 reviews.

Thank you all.