This website uses cookies to ensure you have the best experience. Learn more

An Hmm Based Pre Training Approach For Sequential Data

2103 words - 9 pages

Much recent research highlighted the critical role of unsuper- vised pre-training to improve the performance of neural network models. However, extensions of those architectures to the temporal domain intro- duce additional issues, which often prevent to obtain good performance in a reasonable time. We propose a novel approach to pre-train sequential neural networks in which a simpler, approximate distribution generated by a linear model is first used to drive the weights in a better region of the parameter space. After this smooth distribution has been learned, the net- work is fine-tuned on the more complex, real dataset. The benefits of the proposed method are demonstrated on a prediction task using two datasets of polyphonic music, and the general validity of this strategy is shown by applying it to two different recurrent neural network architectures.
Even if deep learning systems reach state-of-the-art performance in several ma- chine learning tasks, their computational complexity is still a limit in many real- word scenarios. This issue has been partially tackled with the advent of new high performance parallel computing architectures, which exploit powerful graphic processors to speed-up learning algorithms [1]. However, the breakthrough that allowed to effectively train large-scale networks has been the introduction of an unsupervised pre-training phase [2], in which the network is trained to build a good generative model of the data, which can be subsequently refined using a supervised criterion (fine-tuning phase). The pre-training initializes the weights of the network in a region where optimization is somehow easier, thus helping the fine-tuning phase to reach better local optima. It might also performs some form of regularization, by introducing a bias towards good configurations of the parameter space [3]. Although the benefits of pre-training have been extensively investigated in the static domain (e.g., learning images encoded as fixed-size vec- tors), it is not yet clear how this approach should be extended to the temporal domain, where the aim is to model sequences of events. Dealing with time poses many challenges, because the temporal dependences limit the parallelization of the learning process. Despite recent advances in training recurrent networks [4], improving their convergence speed is therefore still challenging. A possible solu- tion is to pre-train only input-to-hidden connections, thereby ignoring temporal information (encoded by hidden-to-hidden connections) by considering each el- ement of the sequence as independent from the others [5].
In this paper we propose a different pre-training strategy, which is reminiscent of the idea of curriculum learning [6]. The rationale behind this approach is that complex problems should be learned by starting from simpler concepts and then increasing the difficulty level by gradually showing more complex training examples to the learning agent. To this aim, instead of using the...

Find Another Essay On An HMM-based Pre-training Approach for Sequential Data

Six-Sigma: A Data-Driven Approach and Methodology for Eliminating Defects in Any Process

1791 words - 8 pages projects started is easy; the difficult part is completing and closing those projects. Thus the right first priority is to identify the right projects to work on which will have an impact on the bottom line and generate savings for the business. Several potential projects must be identified and evaluated based on a cost and benefit analysis. A simple measure, like the project prioritization index (PPI), can be used to prioritize projects according to

An Automated Requirements Traceability: Managing Evolving Requirements Using Ontology-based Approach

1058 words - 5 pages monitoring and updating the traceability link in the requirements document. This work presents a solution for this problem by proposing an approach that uses an ontology-based knowledge representation along with information retrieval techniques. Ontology-based approach is used due to its ability to automatically generate the relationship among requirements concepts. In conclusion, the proposed approach is able to identify missing, broken or even new

An examination of the article "Cladistic analysis of languages: Indo-European classification based on lexicostatistical data"

1242 words - 5 pages be discovered.Works CitedDyen, I., Kruskal, J.B., Black, P., 1992. An Indoeuropean classification: a lexicostatistical experiment. Trans. Am. Philos. Soc. 82, 1-132.Freeman, S. and Herron, J. Evolutionary Analysis. (3rd Ed.). New Jersey: Pearson Education, Inc., 2001.Rexova, K. Frynta, D. and Zrzavy, Jan. 2003. Cladistic analysis of languages: Indo-European classification based on lexicostatistical data. Cladistics. 19: 120-127.

An Investigation of Self-Descriptions Based on Data Collected From Two Participants of Differing Age

2786 words - 11 pages An Investigation of Self-Descriptions Based on Data Collected From Two Participants of Differing Age Abstract ======== This research paper investigates the self-descriptions of two participants in the light of the findings of Morris Rosenberg (1979). Rosenberg suggested that younger children usually describe themselves in physical conditions, and older children/adults have a tendency to use character and

Selcting an Approach for a Qualitative Research Plan

597 words - 3 pages Selecting an Approach for a Qualitative Research Plan Creswell (2013) contends that “we need to identify our approach to qualitative inquiry in order to present it as a sophisticated study, to offer it as a specific type so that reviewers can properly assess it, and…to offer some way of organizing ideas that can be grounded in the scholarly literature of qualitative research” (p. 69). Although I believe all three arguments cited by the author

F/CFAT : An Approach for Combined Safety and Security Analysis

1486 words - 6 pages , when FTA is applied for the safety analysis of software system a new challenge is faced. This is because for those systems it is more desirable to carry out safety analysis with fault tress before the system design stage [2]. Such an analysis provides very useful information and concrete safety requirements for subsequent system design and implementation [4]. However such an approach raises the question about the correctness of the generated

An Advanced Grammatical Evolution Approach for Intrusion Detection in Manet

1347 words - 6 pages for communication. The maximum speed for nodes is set to 30m/sec and the pause time is 30,10,5. The ODMRP protocol is used as a routing protocol for the simulation. Using advanced grammatical algorithm the evolved rule is generated based on the audit data collected from the network under various mobility conditions with and without attacks. Works Cited [1] Sevil Sen, John A.Clark. A grammatical evolution approach to intrusion detection on

An Approach of Energy Efficient and Secure Data Transfer Using Multicast Routing Protocol in Ad Hoc Networks

585 words - 3 pages itself. Hence, in core based multicast tree, shortest path from the source node to the destination node cannot be guaranteed, but only one tree would be needed to connect the set of the source nodes to a set of the receiver nodes. And many MANET has limited energy resources (battery), and each node operates in unattended manner. So energy efficient is an import design consideration for these networks. So we proposed energy efficient paper using

An approach of Energy Efficient and secure data transfer using multicast routing protocol in Ad Hoc Networks

946 words - 4 pages An Ad-hoc network is a wireless network. It uses high frequency radio waves for nodes or device communications which increase mobility but decrease the ranges rather than using communication cables between the nodes. Ad-hoc network consists of no infrastructure and no fixed base station and centralized administrator. So, each node in this network act as router or host for direct communication with each other. Then nodes are freely move from one

Problem Based Learning as an Effective Tool for Teaching

967 words - 4 pages Problem Based Learning as an Effective Tool for Teaching Computer Network Design Overview : In this paper we discussed about challenges that are faced in teaching computer network design using problem based learning as an effective tool. Here we discussed, where it has been introduced first and the techniques that are used in teaching computer network design. A two threaded approach is used

An Improved Principal Line-Based Alignment Refinement for Palmprint Recognition

1727 words - 7 pages alignment algorithm is presented to correct the translation and rotation between the ROI images and, then, present an efficient way to incorporate the alignment result into the competitive coding method for palmprint recognition. A. Principal Line-Based Iterative Closest Point Alignment Fig. 5 Principal line-based ICP alignment refinement of two ROIs TABLE I ALGORITHM OF PRINCIPAL LINE BASED ICP ALIGNMENT Input Two principal line point sets: and

Similar Essays

Access Control And Data Updation With At Pre: Attribute Time Based Proxy Re Encryption For Shared Data In Cloud

2111 words - 9 pages CSP is relatively low and the cost of data updation by data owner and data users are relatively same. 3.3 Main Idea The main idea is to combine CPABE and attribute based PRE and also to incorporate the concept of time. Intuitively, each user is identified by a set of attributes and a set of effective time periods that denotes how long the user is eligible for these attributes, i.e., the period of validity of the user’s access right. The data

Access Control And Data Updation With At Pre: Attribute Time Based Proxy Re Encryption

700 words - 3 pages specifies ACL for a data, and then symmetric key is used to encrypt the data. This key is further encrypted with the public key of users. Users in the ACL use their secret keys to recover the symmetric key and then use this key to recover the data. 3) Identity based Encryption: In this scheme[10] an identity is viewed as set of descriptive attributes. In this scheme a private key for an identity I can decrypt a ciphertext encrypted with an identity

A Knowledge Based Approach To Innovation An Application For Project Based Firms

1055 words - 4 pages ; Sydow and Staber, 2002). The view mentioned above is also seen in similar concepts like project network and Project-Based Organizations (PBOs). Project networks, which are temporary, are created to establish healthy relations between interdependent functional groups and individual that exists to achieve project objectives. PBOs create an environment in which all the organizational structures and capabilities are built in project-centric

Team Based Approach In An Organization

972 words - 4 pages and ill-feelings and the training manager has to recognize and try to diminish such animosity. The animosity can be due to, among other reasons, the leader having less experience or less education than one or more of the other team members. As Swanepoel (2008) argues, team-based work approach means that choice-making about who should be team members become an even greater challenge than when recruiting and selecting people for more individually