Theory-based vs. theory-driven evaluation

“Donaldson and Lipsey (2006), Leeuw and Donaldson (2015), and Weiss (1997) noted that there is a great deal of confusion today about what is meant by theory-based or theory-driven evaluation, and the differences between using program theory and social science theory to guide evaluation efforts. For example, the newcomer to evaluation typically has a very difficult time sorting through a number of closely related or sometimes interchangeable terms such as theory-oriented evaluation, theory-based evaluation, theory-driven evaluation, program theory evaluation, intervening mechanism evaluation, theoretically relevant evaluation research, program theory, program logic, logic modeling, logframes, systems maps, and the like. Rather than trying to sort out this confusion, or attempt to define all of these terms and develop a new nomenclature, a rather broad definition is offered in this book in an attempt to be inclusive.

“Program Theory–Driven Evaluation Science is the systematic use of substantive knowledge about the phenomena under investigation and scientific methods to improve, to produce knowledge and feedback about, and to determine the merit, worth, and significance of evaluands such as social, educational, health, community, and organizational programs.”

– Donaldson, S. I. (2022, p. 9). Introduction to Theory-Driven Program Evaluation (2nd ed.). Routledge.

“This is exactly what gentleness is”

“Another day, in the rain, we’re waiting for the boat at the lake; from happiness, this time, the same outburst of annihilation sweeps through me. This is how it happens sometimes, misery or joy engulfs me, without any particular tumult ensuing: nor any pathos: I am dissolved, not dismembered; I fall, I flow, I melt. Such thoughts—grazed, touched, tested (the way you test the water with your foot)—can recur. Nothing solemn about them. This is exactly what gentleness is.”

– Roland Barthes (1977), A lover’s discourse: fragments

Gorbachev

“Gorbachev’s desperate attempts to preserve socialism and the Soviet Union eventually failed utterly, turning him into an accidental hero in the West. I won’t even give him the minimal credit some offer for not sending in the proverbial tanks to crush the anti-Communist uprisings that were taking place all across the Soviet Bloc, especially since Gorbachev did send in military to Latvia and Lithuania, where he believed he could get away with it. He was hardly a risk taker where his own neck was concerned and didn’t want to end up like Romanian Communist dictator Nicolae Ceaușescu, whose rapid overthrow and execution in December 1989 was still fresh in everyone’s mind.”

– Garry Kasparov (2015), Winter Is Coming

“What is the actual difference between Russia and Ukraine?”

‘As the Director of the Ukrainian Institute London and a historian, I received numerous requests for commentary in the context of Russia’s war against Ukraine. Most began with a question asking me to elaborate on the actual difference between Russia and Ukraine. The question was well meant; it was intended to debunk Putin’s weaponised mythology. But the interviewers were oblivious to their own entrapment in the imperialist framework even as they attempted to give Ukraine a voice. […] Weary of giving a “proper” answer (starting with Volodymyr the Great and ending with Volodymyr Zelenskiy) for the umpteenth time, I asked one journalist a question in return: “What, exactly, is the difference between Ireland and England?” Instead of an answer, I heard a nervous giggle.’

– Olesya Khromeychuk (2022), Where is Ukraine?

Technology and the Task Content of Jobs across the Development Spectrum

Conclusions from Caunedo, Keller, and Shin, Y. (2021; bold added):

“Recently, technological change and rising inequality have been emphasised as two linked phenomena, particularly within developed economies. But throughout history this was not always the case. The post-World War period was one characterised by strong growth, technological advances, and improvements in income levels across the board that shrank income inequality.

Policymakers can start with removing barriers and distortions that deter the reallocation of workers driven by technological change, which will make the economy more efficient. It is also important to further study the link between frictions in the labour market and incentives to adopt new technology.

Equally important is the study of the policies that can foster human capital accumulation through schooling, job training, and re-training so that workers can fully utilise and benefit from the ongoing technological progress.

Not everyone will benefit from technological change and some will fall through the cracks. Practitioners and academics alike will need to renew our thinking on the optimal design of social safety nets. Incorporating job informality, a pervasive problem in poorer economies, should be a priority.”

References

Caunedo, J., Keller, E., & Shin, Y. (2021). Technology and the Task Content of Jobs across the Development Spectrum. NBER Working Paper Series, 28681.

The L’Oréal principle

“Starting from Disney animations that we watch as young children telling us that if we believe in ourselves, we can achieve anything, we are bombarded with the message that individuals, and they alone, are responsible for what they get in their lives. We are persuaded to accept what I call the L’Oréal principle – if some people are paid tens of millions of pounds per year, it must be because they are ‘worth it’. The implication is that, if people are poor, it must be because they are either not good enough or not trying hard enough.”
– Ha-Joon Chang (2014). Economics: The User’s Guide.

Algorithms are to programs as information is to data

‘An algorithm is a precisely-defined sequence of rules telling how to produce specified output information from given input information in a finite number of steps. A particular representation of an algorithm is called a program, just as we use the word “data” to stand for a particular representation of “information”.’

– Donald E. Knuth (1974, p. 323) [Computer Science and its Relation to Mathematics. The American Mathematical Monthly, 81(4), 323–343]

Costs and benefits in policy making

“Costs and benefits should be calculated over the lifetime of the proposal. Proposals involving infrastructure such as roads, railways and new buildings are appraised over a 60 year period. Refurbishment of existing buildings is considered over 30 years. For proposals involving administrative changes a ten year period is used as a standard measure. For interventions likely to have significant costs or benefits beyond 60 years, such as vaccination programmes, or nuclear waste storage, a suitable appraisal period should be discussed with and formally agreed by the Treasury at the start of work on the proposal.”

The Green Book (2022, p. 9)

The point of an English degree (Stewart Lee)

‘The universities minister, Michelle Donelan, wants to chop courses where “fewer than 60% of graduates are in professional employment or further study within 15 months of graduating”. She misunderstands the point of studying the arts. […] The point of an English degree is to inspire those who take it with such a love of literature that they spend the next decade serving in bars while trying to complete their Great Work. And if that doesn’t fly, they must become English teachers, handing on the same curse of loving literature to future generations, their collective misery deepening like a coastal shelf, just as our collective understanding of the works grows because of their efforts.’

– Stewart Lee (2022, 3 June). Tory contempt for the arts means we face a second dark age. The Guardian.

More magic

Photo of the infamous magic switch, by Guy L. Steele Jr; licenced under CC BY-NC-SA 3.0. Source.

“You don’t touch an unknown switch on a computer without knowing what it does, because you might crash the computer. The switch was labeled in a most unhelpful way. It had two positions, and scrawled in pencil on the metal switch body were the words ‘magic’ and ‘more magic’. The switch was in the ‘more magic’ position.

“I called another hacker over to look at it. He had never seen the switch before either. Closer examination revealed that the switch had only one wire running to it! The other end of the wire did disappear into the maze of wires inside the computer, but it’s a basic fact of electricity that a switch can’t do anything unless there are two wires connected to it. This switch had a wire connected on one side and no wire on its other side.

“It was clear that this switch was someone’s idea of a silly joke. Convinced by our reasoning that the switch was inoperative, we flipped it. The computer instantly crashed.”

Read on…