How to Develop Web Crawlers in PHP

Asked By 10 points N/A Posted on -
qa-featured

I have developed a simple website using PHP, HTML and Mysql. What I need to do now is to develop a software that will search the web for specific information and use this to update my webpage. I have learned that web crawlers are candidates to achieve that. I would like some steps needed to develop a simple web crawler to help me achieve my objective. Can I get some to buy and later modify them? How do you modify a web crawler?

  • Answer Accepted: No
  • Question Category:
SHARE
Answered By 0 points N/A #190720

How to Develop Web Crawlers in PHP

qa-featured

<?php

02  
03 /* This code contains the crawler framework that is sued to scrape the links across a specified domain and take the home page as the seed page. Also, we are storing the scraped links from an array to a MySQL database using a simple query. */
04  
05 class webCrawler
06 {
07  
08   protected $markup = '';
09  
10   public function __construct($uri)
11   {
12     $this->markup = $this->getMarkup($uri);
13   }
14  
15   public function getMarkup($uri)
16   {
17     return file_get_contents($uri);
18   }
19  
20 /*  The protected data collection method uses PCRE function preg_match_all() in order to return all tags within the markup that are accepted using our pattern of /href="([^"]*)"/'  . */
21  
22   protected function _get_links()
23   {

 

About Neil Auso
Questions
0
Answers
117
Best Answers
11
  • Posted on -
  • Question Category:

New to iPad Game Development Circle

Previous

How to Develop Script to Move Data between Mysql Tables

Next

Related Questions