How Hard Should The Test Instances Be In Instance Specific Macro Learning?

878 words - 4 pages

1 Introduction
During the instance-specific macro learning experiments [1], we faced a problem
in which there was no significant difference between the perfect model
and the other models / macro sets. I thought initially that learning in general
is not useful. But then I realized that this problem was caused partially by the
way I collect data. The test examples were so easy to capture any significant
difference in performance between the models. So, we need to make the test
instances harder to solve in general. It is also possible to fall in the other extreme,
which is making the problems too hard to the point where no model
can solve most of the instance. Practically, for a test instance to be considered,
I think I should fix a lower bound on the runtime of the empty set and an upper
bound on the runtime of the perfect model in order to have a clear view of
the differences between the models/macro sets.
2 Details
We need to find hard-enough instances to test the macro performance. It is
essential that the instances that I test are hard, because otherwise the differences
in models performances might not be clear. This is easy to check: if the
problem is really easily solvable on the empty set, I should not include it in the
results. So I add a lower bound on the time of the empty set:
T(i,m0) > MinTime
But is this assumption correct? If we set this in advance before running the
experiment, then maybe it is okay. My argument is: putting in mind that we
want to measure the significance in the difference of performance between the
models/macro sets, and given that the process switching time of current operating
systems is non zero, we should make such an assumption. This is because
there will be a small overhead in all processes, and also we cannot guarantee
that the process will take the whole CPU time, which can make the differences
in run-times unclear if the instances are small. So, MinTime is proportional to
the process switching time, the time measuring error, and whatever time that
makes us consider an instance in the relatively hard side of the run-time slope.
Test instances that do not satisfy this constraint should not be considered in the
test, because they are not hard enough.
In my experiment, when an instance times out, I register the cut-off time as
its runtime. This is obviously wrong information. But the alternatives can vary
between two techniques: (1) we can set a very high cut-off time where no instance
can timeout, or (2) we can remove instances that time out on any macro
set. Both of these options are impractical. The first is not practical because we
may need an extremely large cut-off time for the whole test set to finish. The
second option is not practical because, sometimes, there are...

Find Another Essay On How Hard Should the Test Instances Be in Instance-Specific Macro Learning?

Learning the Hard Way- Personal Narrative

894 words - 4 pages Learning the Hard Way- Personal Narrative We live our lives working in order to achieve peace within ourselves, a sense of accomplishment and happiness. The experiences and relationships that we develop along the way help to make us who we are. Weather they are good or bad, we like to believe that knowledge is gained from the people we meet and the decisions made. I have heard it said that it is suppose to be the

Should students be rewarded for learning

2777 words - 11 pages receive rewards for effort in a variety of ways; a smile returned from a stranger or flowers from a loved one but there is strong debate over wether rewards should be given for learning.Woolfolk and Margetts (2007) show two opposing views of the effects of rewards for learning, providing an interesting debate in their point/counterpoint (p.248) on the use of rewards in the classroom referring the reader to several debates publicly voiced in the

The Great Gatsby. The question is what specific instances lead to or are linked to Gatsby's eventual death. Includes direct quotations from the novel.

654 words - 3 pages The Great GatsbyIn Scott F. Fitzgerald's book, The Great Gatsby, the main character, Jay Gatsby, is eventually murdered by George Wilson. However, it would not be appropriate to cast all the blame upon Mr. Wilson. There were others who falsely fueled Mr. Wilson's rage and who covered up their involvement in the crime of killing Myrtle Wilson, Mr. Wilson's wife. The novel is based upon the ideals of treachery and deceit, and the actions of those

How alcochol should not be used in college.

1120 words - 4 pages Alcohol In CollegeAlcohol is a huge problem on most college campus's. Twenty one may be the legal drinking age, but some how minors find a way to get a hold of alcohol. People as young as fifteen are able to get their hands on an alcoholic beverage. Alcohol is said to be the chosen drug among college students. College students have a tendency to drink more then the general population. It is said that college students spend approximately $4.2

How Should International Debt be Rectified in Ghana?

1791 words - 7 pages How Should International Debt be Rectified in Ghana? International debt is a modern geographical issue which has sparked much controversy in the past and continues to affect our global community to this day. Through this essay I aim to analyse contrasting viewpoints and conclude with my own perspective of the current situation, having digested the main arguments. The debt crisis originated in the 1970s due to

How Abstinence Education Should be taught in School

1835 words - 7 pages Over the course of the last few years, there has been much opposition over how sex education should be taught in school. Abstinence should be taught in sex education classes because abstinence is moral. There wouldn't be as many unwed mothers and babies born out of wedlock. Contraceptives that are taught are not guaranteed to protect teenagers from anything, and many people support the teaching of abstinence. It has been said that teaching

Why drug test should be mandatory at schools. persuassive

543 words - 2 pages hand before the test, would you write the answers on your hand? I doubt anyone would. Well, then if students in high school knew that there would be drug tests would the students do drugs? It's the same scenario. If high schools required students to take drug tests then some of the students would quit drugs for fear of being caught. Also, the ones who skip school or freak out the day of the test would be caught, too.Over the past year, people have

Pitfalls of Standardized Testing. This essay explains how standardized tests should not be the only way that schools should look at how students improve in their acedemics.

616 words - 2 pages . Standardized test scores may be rising but this comes at the expense of other important school curriculum. For the sake of accountability, politicians have imposed high stakes on standardized tests, ignoring the warnings of testing experts. Evidence of the pitfalls of standardized tests was presented by Bob Chase in AChallenging The Almighty Test.@ Standardized tests take up valuable classroom time, have no academic value, and are biased ways to determine

Should Cigarettes Be Banned in the U.S.?

1890 words - 8 pages until recently when cigarettes became widely controversial due to the plant containing nicotine, an addictive drug to the body. Aside from containing nicotine and other hazardous chemicals to the body, cigarettes also cause a whole host of health implications to the user and the people around him or her. Aside from these health implications, cigarettes should not be banned in the U.S. because of the exaggeration of the results of secondhand smoking

Tattoos Should Be Allowed in the Workplace

1883 words - 8 pages In today’s world there are several occupations one may choose from once receiving an education. Whether a person wants to be a doctor, a teacher, or a business person, one should be able to use their education to gain access to that job. Getting a job is no easy task, for there are many people who are also searching for that same job. Employers will interview many people just to find the right one and often times will have many stand-out

How Should Science Be Done

1011 words - 4 pages , 1953). These two strands run in opposite directions. Also the bases are on the inside of the helix and the phosphates on the outside. James Watson and Francis Crick discovered that the model of the DNA structure by stealing information in order to form their conclusions. Making such a discovery cannot be taken as an offense just because of how Watson and Crick went to their conclusion; that is how science works. Whether or not their finding was

Similar Essays

Tribal Body Techniques, In Relation To One Or Two Instances Of Clothing Styles Or Body Decoration, Show How The Choice Of Certain Body Techniques And Modes Of Conduct Construct Specific Identities.

2139 words - 9 pages In relation to one or two instances of clothing styles or body decoration, show how the choice of certain body techniques and modes of conduct construct specific identities.----------------------------------------------------------Body techniques are an extremely diverse and controversial subject, being such a broad term; it is often used as a 'catch phrase' to describe any number of things relating to the body. Some of these include, people's

Divorce: How Hard Should It Be To Obtain?

2198 words - 9 pages for the no-fault law, there are also people against covenant marriage. Those people say that the government cannot make couple's relationship better and last longer(Covenant Marriages, 1999). The covenant law is there to help make things better and not let them give up right away, but if the couple does not try then there will not be a change. Many relationship problems are able to be settled no matter what the problem. In Should I Keep Trying to

This Is A Discription How Moving Can Be Hard For A Family, But In Turn, Can Become The Correct Decision.

1413 words - 6 pages my mother pondering how to bring up our move. Finally, she says, "Russ, how does it feel to be done with your first year of school?"Frowning towards her I replied, "Good, but I wish I didn't have to go back next year, my teacher was mean and there are a lot of mean kids in my class. I hate the plays." Everyday in kindergarten in Wauwatosa we had to perform in a play; the selfish kid I was, I refused.My mother sighs, "I wish we had a better

1.Find The Amount Of Reducing Sugars And Starch In Yelloe And Green Bananas By Using The Benedict's Test Nad Iodine Test. Why The Benedict's Solution Should Be Used In Excess.

1022 words - 4 pages that storing bananas at the freezing temperature can stop the ripening process. Also keep the other green bananasin a bell jar can let it become yellow.The dependent variables are the amount if sugar and starch in the 2 banana samples. The amount of reducing sugars in the 2 samples can be compared by performing the Benedict's test and finding the amount of red ppt in each sample .The amount of starch in the bananas can be compared by performing