Study on Web Mining Algorithm based on Usage Mining

ABSTRACT

Web usage mining is an application of data mining technology to mining the data of the Web server log file. It can discover the browsing patterns of user and some kind of correlations between the web pages. Web usage mining provides the support for the Web site design, providing personalization server and other business making decision, etc. Web mining applies the data mining, the artificial intelligence and the chart technology and so on to the Web data and traces users¿ visiting characteristics, and then extracts the users¿ using pattern. This article will study on Web Mining Algorithm based on Usage Mining. And it also produces the design mentality of the electronic commerce website application algorithm. This algorithm is simple, effective and easy to realize, it is suitable to the Web usage mining demand of construct a low cost B2C website.

Rough set clustering for Web mining

ABSTRACT

Similar to traditional data mining, three important Web mining operations include clustering, association, and sequential analysis. Typical clustering operations in Web mining involve finding natural groupings of Web resources or Web users. Researchers have pointed out some important differences between clustering in conventional applications and clustering in Web mining. For example, the clusters and associations in Web mining do not necessarily have crisp boundaries. Moreover, due to a variety of reasons inherent in Web browsing and Web logging, the likelihood of bad or incomplete data is higher. As a result, researchers have studied the possibility of using fuzzy sets in Web mining clustering applications. The paper describes how rough set theory can also be used to develop clustering schemes for Web mining. The unsupervised classification described in the paper uses properties of rough sets along with genetic algorithms to represent clusters as interval sets. The paper also describes the design of an experiment including data collection and the clustering process. The experiment is used to create interval set representations of groups of Web visitors

Research of Web Mining Technology Based on XML

ABSTRACT

Web data mining is a new important research field in data mining. In this paper, the conception and characteristic of data mining based on Web are introduced the process and the general methods of data mining based on Web are expatiated. At present many websites are built with HTML, which is difficult to achieve real effective and accurate web mining. The appearance of XML has brought convenience for it. Based on the research of web mining, XML is used to transform semi-structured data to well structured data, and a model of web mining system which has basic data mining function and faces multi-data on the Web is built. At the same time, the problem in data mining is analyzed and studied. An example is put forward to prove the solution.

Web mining research

ABSTRACT

Web mining is a cross point of database, information retrieval and artificial intelligence. Web content mining (WCM), Web structure mining (WSM) and Web usage mining (WUM) buildup the whole Web mining. The research issues, techniques and development efforts are presented in this paper.

Visual Web Mining of Organizational Web Sites

ABSTRACT

Existing Web usage mining (WUM) tools do not indicate which data mining algorithms are used or provide effective graphical visualizations of the results obtained. WUM techniques can be used to determine typical navigation patterns in an organizational Web site. The process of combining WUM and information visualization techniques in order to discover useful information about Web usage patterns is called visual Web mining. The goal of this paper is to discuss the development of a visual Web mining prototype, called WebPatterns, which allows the user to effectively visualize Web usage patterns

Web Mining: Key Accomplishments, Applications and Future Directions

ABSTRACT

The World-Wide Web provides every internet citizen with access to an abundance of information, but it becomes increasingly difficult to identify the relevant pieces of information. Research in web mining tries to address this problem by applying techniques from data mining and machine learning to Web data and documents. The Web Mining is an application of Data Mining. Without the internet, life would have been almost impossible. The data available on the web is so voluminous and heterogeneous that it becomes an essential factor to mine this available data to make it presentable, useful, pertinent to a particular problem. Web mining deals with extracting these interesting patterns and developing useful abstracts from diversified sources. The present paper deals with a preliminary discussion of WEB mining, few key computer science contributions in the field of web mining, the prominent successful applications and outlines some promising areas of future research

Parallel Web mining for link prediction in cluster server

ABSTRACT

Many Web mining methods have recently been used to model user navigational behavior based on log files of the Web server. Cluster-based server architectures combine good performance and low cost, and are widely used for Web service. In this paper, we propose a parallel Web mining (PWM) algorithm for link prediction in the environment of Web cluster server consisting of several nodes that act as independent Web servers. According to the PWM algorithm, the transition probability matrixes are firstly obtained from the Web log flies of each node by adopting the Markov chain model, compressed under the constraint of the probability threshold c and parallel threshold a, and then sent to the central node which combine these independent results to get an integrated result by some rules. By different accuracy requirement, the PWM algorithm can be divided into simple PWM algorithm (S-PWM), faster but less accurate, and complex PWM algorithm (C-PWM), slower but more accurate. Furthermore, a related incremental parallel Web mining (I-PWM) algorithm is put forward too. The experimental results show that PWM algorithm can not only alleviate the communication cost by sending the mined transition probability matrix and decrease the time complexity by disposing in parallel but also hardly affect the accuracy of the Web mining result.

A Web Mining Architectural Model of Distributed Crawler for Internet Searches Using PageRank Algorithm

ABSTRACT

As the World Wide Web is growing rapidly and data in the present day scenario is stored in a distributed manner. The need to develop a search engine based architectural model for people to search through the Web. Broad Web search engines as well as many more specialized search tools rely on Web crawlers to acquire large collections of pages for indexing and analysis. The crawler is an important module of a web search engine. The quality of a crawler directly affects the searching quality of such Web search engines. Such a Web crawler may interact with millions of hosts over a period of weeks or months, and thus issues of robustness, flexibility, and manageability are of major importance. Given some URLs, the crawler should retrieve the Web pages of those URLs, parse the HTML files, add new URLs into its queue and go back to the first phase of this cycle. The crawler also can retrieve some other information from the HTML files as it is parsing them to get the new URLs. In this paper, we describe the design of a Web crawler that uses PageRank algorithm for distributed searches and can be run on a network of workstations. The crawler scales to several hundred pages per second, is resilient against system crashes and other events, and can be adapted to various crawling applications. We present Web mining architecture of the system and describe efficient techniques for achieving high performance.

Web mining in soft computing framework: relevance, state of the art and future directions

ABSTRACT

The paper summarizes the different characteristics of Web data, the basic components of Web mining and its different types, and the current state of the art. The reason for considering Web mining, a separate field from data mining, is explained. The limitations of some of the existing Web mining methods and tools are enunciated, and the significance of soft computing (comprising fuzzy logic (FL), artificial neural networks (ANNs), genetic algorithms (GAs), and rough sets (RSs) are highlighted. A survey of the existing literature on "soft Web mining" is provided along with the commercially available systems. The prospective areas of Web mining where the application of soft computing needs immediate attention are outlined with justification. Scope for future research in developing "soft Web mining" systems is explained. An extensive bibliography is also provided.

Web structure mining: an introduction

ABSTRACT

Due to the increasing amount of data available online, the World Wide Web has becoming one of the most valuable resources for information retrievals and knowledge discoveries. Web mining technologies are the right solutions for knowledge discovery on the Web. The knowledge extracted from the Web can be used to raise the performances for Web information retrievals, question answering, and Web based data warehousing. In this paper, we provide an introduction of Web mining as well as a review of the Web mining categories. Then we focus on one of these categories: the Web structure mining. Within this category, we introduce link mining and review two popular methods applied in Web structure mining: HITS and PageRank.

Applications of Web mining - from Web search engine to P2P fi-ltering

ABSTRACT

We have developed Japanese Web search engine "Mondou (RCAAU)", which was based on the emerging technologies of data mining. Our search engine provides associative keywords which are tightly related to focusing Web pages. We also implemented the visual interface based on the technology of information visualization. In order to improve the performance of various search strategies by using characteristics of Web systems, we try to implement the advanced Web information systems with data mining and information technologies. Firstly, we introduce various Web mining algorithm, which efficiently reduces the computing cost of Web search. We pay attention to a part of useful pages effectively and improve the performance of Web search by using our proposed algorithms. Secondly, for preserving huge volume of born-digital information in the Internet, we are focusing on technologies of Web archiving system like WARP. In order to handle monotonously increasing digital information, we have to resolve many difficult problems of long life data preservation by improving Web searching techniques. Our experiences of our Mondou Web search engine and cooperative distributed Web robots are very useful and effective. Finally, the technologies of P2P (Peer-to-Peer) distributed search systems are becoming important rapidly. For example, it is very hard to discover appropriate information resources by simple queries of Gnutella, Freenet and so on. Therefore, in order to realize the topic-driven search, we propose more intelligent search systems, which are based on the technologies of data mining.

A Web Mining Model for Real-time Webpage Personalization

ABSTRACT

Determining the size of the World Wide Web is extremely difficult. The Web can be viewed as the largest data source available and presents a challenging task for effective design and access. One proposed Web mining approach to handling the problem of effective design and access is personalization. With personalization, Web access or the contents of a Web page are modified to better fit the desires of the user. This may involve dynamically creating Web pages that are unique per user or using the desires of a user to determine what Web documents to retrieve. This paper presents a Web mining model based on dynamic clustering and hidden Markov model. The output of the model is some information for dynamically creating a Web page which can best meet the user's desires. The assumption of the dynamic clustering is that if a group of users who have the same interest trend, those pages they have visited are probably related. We propose that human should be the authority to judge the correlation of two pages. First, the model statistic a user's Web browsing records in the log file; find a group of users who have the same interest trend with the user; collect all the pages in which this group of users are interested; calculate the correlation between pages; and cluster the pages into several categories according to a predetermined threshold. Each Web page category is considered as a stochastic state variable. In the second phase, our model based on hidden Markov model is further constructed to mine the latent desires of a user given an observed sequence of Web pages that the user have browsed. In order to get the optimal parameters (transition probability matrix, the conditional probability and the initial state) in the model, we applied the Baum-Welch parameter estimation method in EM algorithm to train the model on the data set. Experimental results show that the model is practicable and efficient

Using Open Web APIs in Teaching Web Mining

ABSTRACT

With the advent of the World Wide Web, many business applications that utilize data mining and text mining techniques to extract useful business information on the Web have evolved from Web searching to Web mining. It is important for students to acquire knowledge and hands-on experience in Web mining during their education in information systems curricula. This paper reports on an experience using open Web application programming interfaces (APIs) that have been made available by major Internet companies (e.g., Google, Amazon, and eBay) in a class project to teach Web mining applications. The instructor's observations of the students' performance and a survey of the students' opinions show that the class project achieved its objectives and students acquired valuable experience in leveraging the APIs to build interesting Web mining applications.

نوشته اصلي بوسيله kiarash k (پست 17187) لطفا یه مقاله انگلیسی سال چاپ 2008 به بالا که توضیحات کلی وب کاوی داشته باشه و زیاد تخصصی نباشه معرفی کنین لطفا ممنون