Detection of change frequency in web pages to optimize server-based scheduling

Lakmal Meegahapola, Roshan Alwis, Eranga Nimalarathna, Vijini Mallawaarachchi, Dulani Meedeniya, Sampath Jayarathna

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Citations (Scopus)

Abstract

The Internet at present has become vast and dynamic with the ever increasing number of web pages. These web pages change when more content is added to them. With the availability of change detection and notification systems, keeping track of the changes occurring in web pages has become more simple and straightforward. However, most of these change detection and notification systems work based on predefined crawling schedules with static time intervals. This can become inefficient if there are no relevant changes being made to the web pages, resulting in the wastage of both temporal and computational resources. If the web pages are not crawled frequently, some of the important changes may be missed and there may be delays in notifying the subscribed users. This paper proposes a methodology to detect the frequency of change in web pages to optimize server-side scheduling of change detection and notification systems. The proposed method is based on a dynamic detection process, where the crawling schedule will be adjusted accordingly in order to result in a more efficient server-based scheduler to detect changes in web pages.

Original languageEnglish
Title of host publication17th International Conference on Advances in ICT for Emerging Regions, ICTer 2017 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages165-171
Number of pages7
ISBN (Electronic)9781538624432
DOIs
Publication statusPublished - 15 Jan 2018
Externally publishedYes
Event17th International Conference on Advances in ICT for Emerging Regions, ICTer 2017 - Colombo, Sri Lanka
Duration: 7 Sept 20178 Sept 2017

Publication series

Name17th International Conference on Advances in ICT for Emerging Regions, ICTer 2017 - Proceedings
Volume2018-January

Conference

Conference17th International Conference on Advances in ICT for Emerging Regions, ICTer 2017
Country/TerritorySri Lanka
CityColombo
Period7/09/178/09/17

Keywords

  • Change detection and notification systems
  • Change frequency
  • Crawling
  • Internet
  • Server-side scheduling
  • Web page

Fingerprint

Dive into the research topics of 'Detection of change frequency in web pages to optimize server-based scheduling'. Together they form a unique fingerprint.

Cite this