George Stigler on Education Reform
About thirty years ago a young man named Seguira became the rector of a university in a South American country in which his father had recently financed a successful revolution. Seguira, who previously had been quite the gay young blade, surprised everyone when he immediately settled down to become a serious-minded reformer—a sort of Latin Hutchins. He began casting about for reform, of which—Pinzio said—the university could stand a good deal, and eventually hit upon the merit system. He soon issued the following regulations.
In June of each year any member of the faculty could challenge the person immediately above him a rank to a competitive examination. The examination was to be made up and graded by a group of impartial professors in the United States. (Seguira told Pinzio that this country had been chosen in order to make bribery more expensive.) If the challenger won, he would exchange position and salary with his former superior. Thus an able graduate student could in successive years become an instructor, an assistant professor, an associate professor, and a professor.
There was a terrible uproar, and some shrill glee, when this announcement came out. Some of the older men were very bitter, and emphasized the fact that the rectorship was not included in the competition. But most of the younger members of the faculty were delighted at the prospect, Pinzio among them.
The announcement was made in September, and some very desirable effects were observed during the first year. The physicist Antonio bought a new pair of spectacles so he could once again read small print. Cardan the economist, who had been spending most of his time running a noodle factory, engaged an assistant professor (who could not challenge him for two years) to tutor him in the developments that had occurred in the field of economics during the previous fifteen years. The senior professor in chemistry announced in December, once he fully understood the plan, that for reasons of health he was retiring the next June, and several others followed him.
The library experienced an unprecedented rush. Learned journals—especially American—came out of dusty stacks, and hot disputes raged over the attempts of some men to draw out all of the modern treatises in a field. This, indeed, was the one clear disadvantage of the reform: men began to hoard knowledge. Few were willing to discuss their field except with better-informed people, and the exceptions were attributed to deceit as often as to arrogance. The graduate students suffered most: Filipo devoted his year course in the advanced theory of functions to a review of Euclid; Danto succeeded in getting many economics students to read Alfred instead of Adam Smith; and Ricard reviewed the Baconian theory, in painstaking detail, in his course on Shakespeare.
Yet the results of the competitions the following June were generally conceded to be beneficial. Several men began rapid, if overdue, movement toward retirement. Rumor had it that the unsuccessful and incompetent associate professor Jordan, whose wife was the daughter of the chairman of his department, was contemplating divorce. Pinzio became an associate professor.
Seguira in particular was delighted with the outcome. But he was worried by the tendency, of teachers to devote their graduate courses to empty and irrelevant subjects; so he amended the regulations to grant five points (in a hundred) to a teacher for each of his students who won a challenge. This new rule led to careful calculation: would five points outweigh a superior performance by the student in the examination? The general belief was that professors gained, and assistant professors lost, by careful instruction. The scheme led to some paradoxes the following year: Dourni was challenged by seven of his graduate students and all got higher grades in the examination, but his thirty-five point bonus kept his position for him.
By the next autumn another unanticipated result of the reform became apparent. There was a precipitous fall in enrollments of graduate students, and it was soon discovered that all who could afford it had gone to the United States for graduate work. Seguira shared the professor's indignation at this maneuver, and vowed to amend the regulations the following spring. But meanwhile a heavy gloom settled on the staff: Were not the migratory students virtually studying the examination questions?
The gloom was justified by the fact. Of the sixty-one students who spent the year in the United States, forty-six won their challenge the following spring. Nor were the results so generally approved as the year before. It is true that several fossils continued their steady march to the museum, and several able young men moved up another rank (Pinzio among them). But Storeo, the brilliant young astronomer, was defeated by a mediocre graduate student who—with the examiner—had spent the year studying some obscure variable stars. And Birnii fell because his magisterial command of political theory did not extend to the details of the New England township.
Seguira was in a quandary. To rule out migration was to invite charges of provincialism and inbreeding; to permit it on the current scale was to destroy graduate study. He finally devised a compromise: the examinations would be given by professors chosen at random from the United States, England, France, Sweden, and Germany. Now if a graduate student went abroad, four times out of five he would guess the wrong country. The amendment did stop the mass migration, but it had its own embarrassments: one sociologist had the same examination question, by chance, in two consecutive years, and each time gave the same answer. The first year he received an ignominious flunk (from Stockholm), and the second year he was offered a professorship (at Harvard).
And as the scheme entered its third year, a further effect could no longer be overlooked. Research had almost stopped. Once observed, of course, it was easy to explain. A man was likely to pass the examination with high marks if he knew what others were doing; it did not help his chances materially to do something himself. The faculty was becoming extremely well-read, and extremely unenterprising. The most prominent exceptions showed the advisability of the rule. Therespi had continued his careful researches on the fruit fly and lost in two challenges. Laboro had finished the seventh volume of his monumental history of South America, and flunked the question on the Crusades.
Another year and some serious faculty losses were required to arouse Seguira to the importance of this problem. Once persuaded, he issued still another amendment: a man was to receive two points for each article, and seven for each book. He wanted to restrict these bonuses only to current publications, but the opposition was too strong. Even the younger men, especially the successful younger men, were complaining of the baneful effects of insecurity of tenure. And it was pointed out that definitive works required time—perhaps even two years. Seguira compromised by including all works published within the previous decade.
The calculations of the faculty now became even more complex. A book (seven points), or the training of a superior student (five points)? The writing of the book might require three years, but the points were received every year thereafter, whereas the student might eventually leave. The answers at which the faculty finally arrived were various. Cimoor, whose father owned a publishing house, succeeded in getting out two books within the first year, and so influential was his father that many of the reviews were neutral. The political scientist Broze withdrew a book already in page proof, and published the nineteen chapters as nineteen articles. This, however, occasioned less complaint than Cardan's publication of a book of readings. But still, research revived somewhat.
This sequence of felt difficulty and hopeful amendment, Pinzio said, might have gone on as long as the unstable political foundations of Seguira's position permitted, had not two developments come to pass. The first was the sudden dawning on Seguira that this patchwork of rules was gradually obliterating the whole purpose of his reforms. This was brought home during the next annual competition, when four professors came out of unwilling retirement and three, with the aid of their writings of previous years, began again to climb the academic ladder. This particular development, of course, could be dealt with through a new rule—but where was it leading?
The conjunction of the second event proved decisive. Shortly after this awakening, Seguira received an invitation from the towering University of South America to become its rector. The regents wrote that his reputation for originality and enterprise was international, and that the success of his experiments indicated the need for a wider field of application. He accepted the new position, as much as a refuge as a promotion.
And what happened to Seguira? we demanded, and to the university? Seguira became as conservative as his reputation would allow, Pinzio assured us. And the merit system? Only one more amendment was added: a man could receive a permanent bonus of any number of points the department chairman deemed fit, when an offer was received from another university.
The above story is from the great economist George Stigler, from his book The Intellectual and the Marketplace originally published in 1963. Thanks to
for pointing it out to me.The story is a highly amusing illustration of the workings of Goodhart’s law within the education system which has been a topic of mine for quite some time now; see here, here, and here.
Adam Smith’s quote from The Theory of Moral Sentiments (1869) seems appropriate on this occasion:
The man of system, on the contrary, is apt to be very wise in his own conceit; and is often so enamoured with the supposed beauty of his own ideal plan of government, that he cannot suffer the smallest deviation from any part of it. He goes on to establish it completely and in all its parts, without any regard either to the great interests, or to the strong prejudices which may oppose it. He seems to imagine that he can arrange the different members of a great society with as much ease as the hand arranges the different pieces upon a chess–board. He does not consider that the pieces upon the chess–board have no other principle of motion besides that which the hand impresses upon them; but that, in the great chess–board of human society, every single piece has a principle of motion of its own, altogether different from that which the legislature might chuse to impress upon it.
