Half of the bribery scandal was cheating on SATs, so the more you emphasize test scores the more you elicit that kind of behavior.
The College Board and its executives make millions selling test prep materials and running a monopoly over US higher education. There's a good chance we're only beginning to discover the inherent corruption here.
I disagree. In India the IIT-JEE, for admissions into the IITs, are super hard, so hard that I can bet no one can repeat performance between years, thereby removing almost any chance that someone else taking the test would be able to provide any kind of performance guarantee.
In fact the problem with the US college admission system is that it's not at all objective, rather is completely subjective, thereby giving the admission committee a lot of leeway. Students, who are good in academics, are unnecessarily (biased opinion) forced to spend time on extra curricular activities, volunteering, sports etc. Not that they are not important, but I'm not convinced that they should be part of the admission criteria.
By making admissions dependent on some standardized test you will remove all, or at least most, of human involvement which leads to corruption.
There are people offering IIT jee scores(usually at really
expensive coaching institutes). hell the best IITJEE instructors take the test every year(to see the difficulty changes) and manage to get roughly the same high score everytime
The SAT/ACT is no longer standardized testing. All the determined parents abuse this. They find a sympathetic doctor to diagnose their kid with a nonsense learning disability, then that gets them double the time for the test or more. The test vendor isn't even allowed to disclose this inequality to the colleges; there was a lawsuit over that.
So you may play by the rules, but other students are getting twice the time to complete the test. Some students are more equal than other students.
This pretty much invalidates the whole concept of the test. Unless sanity prevails, the only way for test vendors to fix the problem would be to give an absurd amount of time to everybody.
> I would be extremely pissed if that criterion was applied.
It isn't a matter of "if"...that is the standard now, lower scores for certain demographics...that is why I don't put Asian on anything...as a mixed-race Filipino, I can pass as some kind of Latino which is what I now "identify" with since it helps me in nearly every case imaginable...it kind of sucks, but that is the world we now live in.
> that is the standard now, lower scores for certain demographics
I was talking about the opposite case: belonging to a class whose scores are artificially raised. If I can get excellent scores and get admitted by myself, but then everybody thinks that I got admitted due to my ethnic background, not due to my merits. Infuriating!
> it kind of sucks, but that is the world we now live in.
That certainly does not seem to be the case in Europe.
> I was talking about the opposite case: belonging to a class whose scores are artificially raised. If I can get excellent scores and get admitted by myself, but then everybody thinks that I got admitted due to my ethnic background, not due to my merits. Infuriating!
I think I follow you...that must not be fun...having everyone think you didn't deserve the spot you earned because of other undeserving "diversity applicants"
> That certainly does not seem to be the case in Europe
I was talking about the U.S. because the author of the article was talking about the U.S...I should have been more specific
In Korea and Japan, they use the one test system and have the highest suicide rate among high school students. It also makes their high school lives hell and their education system lackluster at best. I would advise caution.
I know people who got to uni with exceptionally high marks on their HSC and absolutely no problem solving skills to speak of. Doing well on an exam does not necessarily mean someone is more intelligent or capable, it means they were good at that exam.
That seems unfair to the students who apply for admission: if supply was high the previous year and low the current year, a student in the current year may not be admitted even if they demonstrated identical academic performance (assuming "academic performance" just means standardized test score here) as an admitted student from the previous year.
Sure, but this is just life isn’t it? A university decides to change the courses it offers, or the number of places offered within a course, just because things change. There also might be a new course offered this year which the student would not have had the opportunity to apply for last year.
What a terrible idea. Ignoring the fact that not everyone can take the test for economic or mental health reasons, and the fact that being able to prep for this test effectively skews in favor of wealthier folks, there are so many experiences that shape us. How does learning to overcome adversity because you had to survive homelessness show up in the SAT? How does being raised in a multicultural background? How does music? The arts? Your ability to engage people around you and spark imaginations? Or your inventiveness or compassion?
There are so many types of people I would want to be around in college who might not do great in a standardized test. I'd rather have a rich set of people around with different ideas and experiences than filter down to a single test result.
I still agree with his main point. I got an 800 in Math SAT I & II, and I am far from a Math genius. I don't think you need to implement the Putnam exam, but the SAT is structured to make it very difficult to discern a college player from a pro basically. More than 25% of MIT has an 800 SAT and 75% has at least 780 Math SAT.
Here’s a solution. Make college free for all. Anyone with a HS diploma can get in. But, you MUST attend the classes and do the work or you’re kicked out end of semester.
Totally based on merit. No biases on income or anything else.
Who pays the utility bills? Professor and administrative salaries? Is anyone allowed in just due to having a diploma? What keeps rampant cronyism and end of year cost splurges to keep high budget levels for the next year? Who sets the budget? How does enforcement work, government employees doing enforcing?
In short how do you pay for it given the already spiraling out of control costs of higher education?
Are you saying charging high tuition is responsible for making US universities better? Correlation alone does not equal causation. Is there any evidence for this statement?
I can't speak for Switzerland but all universities in England charge the same for UK students with nothing upfront. Sure they don't have space for everybody who meets the minimum but that's inevitable. There are no legacy admissions, athletic admissions, positive discrimination etc. Everybody goes through the same process.
If someone doesn't attend or do anything why does it mater. If they never show up then they take no space and they use no teacher time so they cost nothing to "educate"
This is effectively what Czech STEM universities do. The supply is way bigger than the demand and as such getting even to the "top" study program is trivial.
Not sure it's very effective seeing the 60%+ failure rates in the first year. These people could spend that year doing something they are more suitable for.
I selfishly love standardized testing because I have mostly done well on it. but SAT prep is ABSOLUTELY effective. I have my own personal anecdotal experience as well of that of my peers, but there are countless prep courses that boast impressive scores (unless they are lying through false advertising, but I believe it). The price of private tutoring and multiple rounds of simulated live exams is prohibitive and an absolute difference maker.
Effective for passing a test, maybe, but in my view 'passing tests' (that can be prepared for especially) is well down the list of 'important life skills.'
If a good SAT score is required to get you into a good college, and a good college is required to land a well-paying job, then passing the SAT with a high score is definitely an important life skill.
Yes. Case in point, the daughter of the lady in my town who ran the SAT and SSAT prep courses got all 800s. ALL 800s. She was smart, but not that smart. It is possible to be completely prepared for the SATs.
With respect, what in the world are you talking about? A casual search on the topic shows that 2 million people took the SAT last year. Of those 2 million, fewer than 500 achieved a perfect score.
Isn't the difficulty tuned to produce a histogram close to that of a normal distribution? Yes it's true that the contents of the test have changed (removing analogies, and then recently circa 2016 revamping the writing/reading sections to be more like the ACT) but fundamentally it doesn't really look that different to me. In fact if anything now the reading section relies more on understanding and analysis of passages rather than just memorizing words and definitions which if anything seems like slightly raising the bar.
Far from universally true. IQ tests like the SAT, GRE, GMAT etc. are all designed to be as close to ungameable as possible. There’s a limited amount they can do but the limit is very high indeed. The major, and unavoidable defect is the practice effect. The more you do something the better you get at it. But IQ tests are like running or weightlifting, there are beginner gains, there are tough gains that are ground out with a great deal of effort and there’s a limit you’re not going to surpass no matter how hard you try. O matter how many steroids I take or how hard I train I’m not going to beat Usain Bolt in the 100m. I’m probably not going to beat the average NHL player.
Goodhart’s Law is the opposite of true in quality control. What gets measured gets managed. Manufacturing defects in semiconductors, automobiles and I presume, most everything else have been trending down for decades. Businesses have KPIs for employees for a reason. Monomaniacal focus over a long period on one measure may not be the best idea, but it’s a great tool to have.
This is a great example of Goodhart's Law, though. If nobody studied for the SAT, it might (arguably) work fine as a measurement of collegiate potential. But because it became the metric for collegiate success, and because the "practice effect" means that practicing makes you do better on the test, many people study hard for it and greatly increase their likely score. The folks who don't study as much or as effectively for it for whatever reason will not do as well, as so you're left with a measurement of whether people studied for the SAT and not whether the test taker would be likely to succeed at college. So a good metric becomes a bad metric because it's chosen as the metric.
Not saying it isn't a problem but those better off have the resources to be educated better before in addition to factors like nutrition. Essentially the advantages aren't bias but a byproduct. The way to fix it of course is to raise the floor.
> Predicting Success in College: SAT® Studies of Classes Graduating Since 1980
> Studies predicting success in college for students graduating since 1980 are reviewed. SAT® scores and high school records were the most common predictors, but a few studies of other predictors are included. The review establishes that SAT scores and high school records predict academic performance, nonacademic accomplishments, leadership in college, and postcollege income. The combination of high school records and SAT scores is consistently the best predictor. Academic preadmission measures contribute substantially to predicting academic success (grades, honors, acceptance and graduation from graduate or professional school); contribute moderately to predicting outcomes with both academic and nonacademic components (persistence and graduation); and make a small but significant contribution to predicting college leadership, college accomplishments (artistic, athletic, business), and post-college income. A small number of studies of nonacademic predictors (high school accomplishments, attitudes, interests) establish their importance, particularly for predicting nonacademic success.
> SAT and ACT scores as predictors of undergraduate GPA scores of construction science and management students
> The result showed relatively strong positive correlation and predictive indices for both ACT and SAT. Thus, the hypothesis of higher UGPAs being related to higher ACT or SAT scores was supported. It was concluded that the admission committees might need to reexamine their admission requirements and/or look at ACT score more than SAT during admission.
There is no better predictor for future academic success than the SAT (or an equivalent IQ test - ACT/LSAT/GRE/GMAT).
The only other predictors that are good at predicting success are either undesirable (socioeconomic status of parents, for instance), or hard to standardize between applicants (school GPA).
> Allow people to challenge students on their score forcing them to retake the test, or randomly sample 1% for a retest.
You want to implement a challenge round for the SAT? Over 2 million kids took the SAT last year[1], many of them for the second or third time. Your proposal forces a kid to be required to retest if they're arbitrarily challenged by someone else. Or it uses a lottery to select tributes who will have to retake the exam again.
The SAT is an incredibly stressful part of an incredible stressful stage of life. Kids already take it several times to improve their scores. Your proposal adds even more elements which are out of parents' and kids' control that can potentially wreak havoc on their time and mental health. It's basically the nuclear option of attempts to improve the SAT.
My point was just there are obvious ways to prevent bribery. And considering the amount we burn on higher education, spending much more on tests and verification does seem like a much much better use of money.
AGI == software capable of any economically relevant task.
If you have AGI, it is very clear you could very quickly displace the entire economy, especially as inference is much cheaper than training: which implies there will be plenty of hardware available at the time AGI is created.
People thought the same thing about nuclear energy. A popular quote from the 50s was "energy too cheap to meter." Yet here we are in a world were nuclear energy exists but unforeseen factors like hardware costs cause the costs to be much more than optimists expected.
These claims are certainly plausible to me, but they are by no means obvious. (For reference, I'm a postdoc who specializes in machine learning.)
>These claims are certainly plausible to me, but they are by no means obvious. (For reference, I'm a postdoc who specializes in machine learning.)
I don't think I disagree very much with you then.
>People thought the same thing about nuclear energy. A popular quote from the 50s was "energy too cheap to meter." Yet here we are in a world were nuclear energy exists but unforeseen factors like hardware costs cause the costs to be much more than optimists expected.
In worlds where this is true, OpenAI does not matter. So I don't really mind if they make a profit.
Or to put it another way, comparative advantage dulls the claws of capitalism such that it tends to make most people better off. Comparative advantage is much, much more powerful than most people think. But nonetheless, in a world where software can do all economically-relevant tasks, then comparative advantage breaks, at least for human workers and the Luddite fallacy becomes a non-fallacy. At this point, we have to start looking at evolutionary dynamics instead of economic ones. An unaligned AI is likely to win in such a situation. Let's call this scenerio B.
OpenAI has defined the point at which they become redistributive in the low hundreds of billions. In worlds where they are worth less than hundreds of billions (scenerio A, which is broadly what you describe above) they are not threatening so I don't care - they will help the world as most capitalist enterprises tend to, and most likely offer more good than the externalities they impose. And as in scenerio A, they will not have created cheap software that is capable of replacing all human capital, comparative advantage will work its magic.
In worlds where they are worth more, scenerio B, they have defined a plan to become explicitly redistributive and compensate everyone else, who are exposed to the extreme potential risks of AI, with a fair share of the extreme potential upsides. This seems very, very nice of them.
And should the extreme potentials be unrealizable then no problem.
This scheme allows them to leverage scenerio A in order to subsidize safety research for scenerio B. This seems to me like a really good thing, as it will allow them to compete with organizations, such as FaceBook and Baidu, that are run by people who think alignment research is unimportant.
Lowest estimate of human brain’s compute capacity is 30 TFLOPs. More reasonable estimate is 1 PFLOPS. Achieving this level of compute would cost few thousand dollars per hour. Human costs $7-$500 per hour. On the top, energy consumption bills would be outrageous. Because of slowed down Moore’s law the price goes down by 10X about 15yrs. I also think it’s fair to assume that human brain sculpted over million year of evolution with extreme frugality is fairly optimal as far as hardware requirements are concerned. So I tend to think “affordable AGI” might be around 45 years away.
I don't see the problem. If they get AGI, it will create value much larger than 100 billion. Much larger than trillions to be honest. If they fail to create AGI, then who cares?
> (AGI) — which we define as automated systems that outperform humans at most economically valuable work — [0]
I don't doubt that OpenAI will be doing absolute first class AI research (they are already doing this). It's just that I don't really find this definition of 'GI' compelling, and 'Artificial' really doesn't mean much--just because you didn't find it in a meadow somewhere doesn't mean it doesn't work. So 'A' is a pointless qualification in my opinion.
For me, the important part is what you define 'GI' to be, and I don't like the given definition. What we will have is world class task automation--which is going to be insanely profitable (congrats). But I would prefer not to confuse the idea with HLI(human-level intelligence). See [1] for a good discussion.
They will fail to create AGI--mainly because we have no measurable definition of it. What they care about is how dangerous these systems could potentially be. More than nukes? It doesn't actually matter, who will stop who from using nukes or AGI or a superbug? Only political systems and worldwide cooperation can effectively deal with this...not a startup...not now...not ever. Period.
California has some of the worst municipal governments in the country. Shockingly poorly governed. Live a year in Singapore. Live a year in San Francisco. Your opinions on many things will change.
Hearing the performative compassion of San Francisco‘s politicians and the results: streets covered in human shit, crazed heroin zombies colonizing them, a pathetic protest culture pushing zoning legislation that extracts rents from everyone; one is reminded of this Kipling poem:
http://www.kiplingsociety.co.uk/poems_copybook.htm
As someone who lives in LA, I have to agree. Really poor local government structure. Our city council just 10-2 voted to authorize the city attorney to settle a lawsuit which will now restrict the city’s ability to cleanup homeless encampments as well. So, might be getting worse, not better.
I recently moved to Encino, which has its issues but our homeless encampment is hidden down by the dam. When I lived in Beverly Grove, I had people sleeping on my sidewalk, and clusters of people setting up camp in the alleyways 100 yards from my house.
What was remarkable was the volume of trash that accumulated, along with broken glass and hypodermic needles. It was absolutely jaw dropping.
It’s truly not compassionate to let people live on the steets. The city and the state must expand conservatorship and start getting these people the help they need.
Not sure what you’re referencing. Tech isn’t a big constituency in Los Angeles, so hard to blame that. And I don’t even see how tech regulation has anything to do with homelessness management in Los Angeles.
Art is the production of veblen goods. Duchamp, in his genius, realized that though the ability to buy something painfully handcrafted does signal wealth, buying literal garbage is a much better display of wealth. Someone who can afford to buy a meticulously painted portrait, well, one may be will willing to make sacrifices for beauty. Someone able to spend millions on literal garbage - now that is a rich man.
Another aspect is photography allowed normal people access to create cheap reproductions of beautiful art. High-class people used to be able to distinguish themselves from the mob with beautiful things the mob couldn't afford.
But this does not work with cheap reproductions allowing one to signal the same taste.
In this way, there became an incentive for high-class people to acquire and inculcate a taste for art that is actively repulsive to distinguish themselves from those normal people who desire beautiful things.
Mostly this doesn't matter. They are only hurting themselves. But the effect of these incentives on public architecture has been pretty horrifying.
> Duchamp, in his genius, realized that though the ability to buy something painfully handcrafted does signal wealth, buying literal garbage is a much better display of wealth. Someone who can afford to buy a meticulously painted portrait, well, one may be will willing to make sacrifices for beauty. Someone able to spend millions on literal garbage - now that is a rich man.
> Mostly this doesn't matter. They are only hurting themselves. But the effect of these incentives on public architecture has been pretty horrifying.
One supposes a man such as him would be able to accept death with equanimity. This is backwards. As he was losing much more in death than most of us will; he was far more alive than most anyone has ever been.
Russell was 97. Feynman was 69 and was in treatment the last decade of his life. von Neumann went from diagnosis to dead in a little over a year. He was 53.
Von Neumann liked to eat and drink; his wife, Klara, said that he could count everything except calories. He enjoyed Yiddish and "off-color" humor (especially limericks).[19] He was a non-smoker.[54] In Princeton, he received complaints for regularly playing extremely loud German march music on his gramophone, which distracted those in neighboring offices, including Albert Einstein, from their work.[55] Von Neumann did some of his best work in noisy, chaotic environments, and once admonished his wife for preparing a quiet study for him to work in. He never used it, preferring the couple's living room with its television playing loudly.[56] Despite being a notoriously bad driver, he nonetheless enjoyed driving—frequently while reading a book—occasioning numerous arrests as well as accidents.
In the hope of forestalling an argument: there are cohorts of "excellence" where differentiation is often in the eye of the beholder (20 smartest people, 20 best guitarists, 20 best research universities). But I would hope it would be uncontentious to put VN in that group.
this is kind of an ironic statement on this forum when von neumann thought programming (outside of directly writing binary) to be a complete waste of time and effort.
in terms of basically being a human computer though, i think he's pretty far up there. although, i do think there's a lot more to being smart than just being a technical person. for example, i would rate noam chomsky to be pretty far up there in terms of smart people. when you hear him speak, he seems to have a photographic memory for damn near everything he has read, which is a ton.
Turing thought the same same way about programming. For example, he didn't see any point in programming languages.
Basically nobody saw at the beginning that actual computer programming could be very difficult task. You just design algorithms in abstract and some clerk inputs them into the computer in machine language.
"As soon as we started programming, we found to our surprise that it wasn't as easy to get programs right as we had thought. Debugging had to be discovered. I can remember the exact instant when I realized that a large part of my life from then on was going to be spent in finding mistakes in my own programs." – Maurice Wilkes, designer of EDSAC, on programming, 1949
ps. & edit:
Similar thing happened with AI. Dartmouth Workshop in 1956 was the beginning of systematic AI research. Tt was thought that there could be significant progress in in few months and at least during the next year in things like natural language understanding. McCarthy, Minsky, Shannon, etc. had to first discover how hard problems really were.
We're still doing that with AI today. Each step forward is seen as signs of imminent acceleration towards AGI, when we really have no idea yet how hard the problems are that lie ahead. It's encouraging to know that it isn't a new phenomenon.
Speaking of photographic memories, von Neumann had one too:
Herman Goldstine wrote "One of his remarkable abilities was his power of absolute recall. As far as I could tell, von Neumann was able on once reading a book or article to quote it back verbatim; moreover, he could do it years later without hesitation. He could also translate it at no diminution in speed from its original language into English. On one occasion I tested his ability by asking him to tell me how A Tale of Two Cities started. Whereupon, without any pause, he immediately began to recite the first chapter and continued until asked to stop after about ten or fifteen minutes."
also:
"Von Neumann was reportedly able to memorize the pages of telephone directories. He entertained friends by asking them to randomly call out page numbers; he then recited the names, addresses and numbers therein."
I think maybe it's a bit of a notation problem. They seemed to have been talking about programming as just the "writing the code" phase. We all understand programming as the last step in solving a real problem. You have to understand the problem itself, come up with the algorithm, possibly optimize it, etc.
"We all understand programming as the last step in solving a real problem. You have to understand the problem itself, come up with the algorithm, possibly optimize it, etc."
From the waterfall model to agile to test-driven programming and design to worse is better, programming has evolved away from the view that programming was just coming up with an unchanging, almost Platonic, ideal program and merely implementing it, to a much more iterative, exploratory process where code evolves in response to feedback from stakeholders and input from how it's working and meeting needs in the real world.
Many of the other candidates of that time agreed that Von Neumann was the smartest among them. He had a mind like a racecar.
Quoting Eugene Wigner:
> "I have known a great many intelligent people in my life. I knew Planck, von Laue and Heisenberg. Paul Dirac was my brother in law; Leo Szilard and Edward Teller have been among my closest friends; and Albert Einstein was a good friend, too. But none of them had a mind as quick and acute as Jansci [John] von Neumann. I have often remarked this in the presence of those men and no one ever disputed.
> But Einstein's understanding was deeper even than von Neumann's. His mind was both more penetrating and more original than von Neumann's. And that is a very remarkable statement. Einstein took an extraordinary pleasure in invention. Two of his greatest inventions are the Special and General Theories of Relativity; and for all of Jansci's brilliance, he never produced anything as original."
Here's the usual anecdote that goes along with von Neumann threads:
> The following problem can be solved the easy way or the hard way:
> "Two trains 200 miles apart are moving toward each other; each one is going at a speed of 50 miles per hour. A fly starting on the front of one of the trains flies back and forth between them at a rate of 75 miles per hour. It does this until the trains collide and crush the fly to death. What is the total distance the fly has flown?"
> In a strict mathematical sense the fly actually hits each train an infinite number of times before it gets crushed, and one could solve the problem the hard way with pencil and paper by summing an infinite series of distances. This is the way that most trained mathematicians will solve the problem. Conversely a mathematical novice will most likely solve the problem the easy way - since the trains are 200 miles apart and each train is going 50 miles an hour, it takes 2 hours for the trains to collide, therefore the fly was flying for two hours, at a rate of 75 miles per hour, and so the fly must have flown 150 miles. Easy.
> When this problem was posed to John von Neumann, he immediately replied, "150 miles."
> "Ah, I see you've heard this one before, Professor von Neumann. Nearly everyone tries to sum the infinite series."
"What do you mean?" asked von Neumann. "That's how I did it!"
George Pólya[1], author of the math classic How to Solve It[2], wrote "Von Neumann is the only student I was ever intimidated by. He was so quick. There was a seminar for advanced students in Zurich that I was teaching and von Neumann was in the class. I came to a certain theorem, and I said it was not proved and it may be difficult. Von Neumann didn’t say anything but after five minutes he raised his hand. When I called on him he went to the blackboard and proceeded to write down the proof. After that I was afraid of von Neumann."
Von Neumann was 15 when he met famous mathematician Gabor Szego who would then teach him analysis. Szego started to cry after they talked a little math during their first meet.
People are entitled to build whatever they want on their property. As Silicon Valley is ruled by rent seekers who make building illegal, people have every right to complain.
"Rent seeking" is a specific term in economics referring to people who seek to extract wealth from the economy without adding to it in any way, e.g. "make building illegal" in order to force up the value of their own property.
The term is somewhat dated, but the usage makes sense in context. Historically there has been a noble class awarded privileges via mechanisms external to the market that leveraged those privileges to extract revenue from the market. Generally this consisted of renting out some right like the right to farm certain land or draw water from certain wells or dock ships in certain harbors. From this we get the modern economic usage of the term "rents" to mean the additional cost someone can charge for something due to their exclusive or semi-exclusive ability to provide it. Think patents, copyrights, monopolies, trade secrets, and ownership of scarce finite resources (like land in SF). From this comes the term "rent seeking" to describe the behavior of obtaining market privileges through means external to the market. This (mostly) consists of trying to influence laws to be in your own favor, like patent owners trying to get patents extended or homeowners opposing upzoning.
You're welcome! I went down a wikipedia black hole on economics a while back. The most memorable term I learned was moral hazard, which also sounds confusing.
No one's proposing to build a sewage treatment plant on this land. It's just more housing, in a city where people go without a roof over their head because it's too expensive. There's no "nuisance" here, only landowners trying to assert rights over their neighbors' property.
Austin rent has gone through the roof, and they’ve got a strong nimby and zoning problem just like SF so it’s just going to get worse. It’s on the exact same track, just ten or so years behind.
Austin doesn't have Prop 13, and Texas imposes reasonable property taxes. Unless those things change, it's HIGHLY unlikely real estate in Austin will approach The Bay area in terms of prices.
To prevent bribery randomly sample 1% for a retest under much stricter security.
If you must, lower standards for protected classes to even out demographics.
Rich people can afford to burn their children’s childhood on the enrichment activities and “volunteer” work Harvard requires.
Standardized testing is by far the most fair way to do this. SAT prep is not that effective and besides SAT prep is now available online for free.