Reading Metrics Source — PISA (Programme for International Student Assessment)

[This post is a part of a series investigating the "boy crisis" in reading. Start here.]

The first source of reading data I'm looking at is PISA (Programme for International Student Assessment), which tests 15 year olds in many countries worldwide in the areas of reading, math, and science. This is a survey done every three years by OECD (Organisation for Economic Co-operation and Development). OECD is a Europe-heavy multinational organization that puts out an astounding amount of analysis and standards via their digital library, much of which is freely accessible. 

At this time, PISA 2018 results are the most recent available. Reading also happened to be the primary focus area for PISA 2018. COVID delayed the next round of assessment to be 2022 and these results are not yet available mid-2023. For my purposes here, I'll be focusing on the book-length report . Page references will refer to this report unless otherwise noted.

Parts of the 2018 reading assessment are available online. I have to say I was impressed! For example, check out these series of questions about a chicken-themed Internet forum. This is literacy in action in a relevant situation. This goes beyond surface-level interpretation to navigating a common information structure and thinking about the use of locutionary, illocutionary, and perlocutionary language. In other words, not just what someone said but also what they are implying and what they're trying to accomplish by saying it. This means we should interpret PISA reading assessment results as being about a broad approach to applied literacy, beyond decoding and even reading fluency.

"PISA assesses reading literacy, as opposed to reading. Reading is often interpreted, in a general, non-academic context, as reading aloud or simply converting text into sounds. PISA conceives of reading literacy as a broader set of competencies that allows readers to engage with written information, presented in one or more texts, for a specific purpose." PISA 2018 Results (Volume 1) What Students Know and Can Do, p. 34.

There is considerable care put into vetting the questions for international use. If you want to know more about that or how the points scale is constructed and interpreted, read here. The key take-away is that the scores only give answers about relative performance among students, which allows for comparisons among demographics within a country, among countries, and between different years of PISA testing. All of these have calculated error ranges. As far as the gender gap in reading performance on the PISA 2018 test, this is the most useful overview graph (context here):

Here we see comparative histograms of reading vs math performance by gender. Someone may well be worried about a math reading gap where boys have a bit of an edge for top performance in that area, but you can see that it's a little larger in favor of girls for reading. Plus, the lower end of math performance is almost identical while the lower end for reading performance even more strongly favors girls. In both cases, there is a somewhat larger proportion of girls around the middle.

Keep in mind that we're staring at the edges for differences. The vast majority is overlap. Edge gaps like these don't support sweeping generalizations about girls being good readers and boys being poor readers. On the other hand, this data does confirm the feeling that boys are overrepresented among students in need of intervention at the lowest end of reading proficiency.

You can see a country-by-country breakdown here. The mean reading score for all countries was 487 with a mean gender gap of about 30 points. Peru and Columbia showed the smallest gender gap of about 10 points, but their mean reading scores were only 401 and 412, respectively. The United States had a mean reading score of 505 and a gender gap of about 24 points, which is both a higher overall score and smaller gender gap than average. In addition, a histogram for the United States would not show girls scoring higher than boys on the right-side (high-side) of the curve:

"In 11 PISA-participating countries and economies, namely B-S-J-Z (China), Chile, Colombia, Costa Rica, Japan, Korea, Mexico, Panama, Peru, Chinese Taipei and the United States, there was no difference between boys and girls at the top of the distribution of reading performance. But in all countries/economies, the first decile of the performance distribution amongst boys was significantly lower than that amongst girls." (emphasis added) PISA 2018 Results (Volume II): Where All Students Can Succeed, p. 148.

From a few examples, we might be tempted to think that higher mean reading score in a country is linked to a smaller gender gap, but there is a scatter plot that shows this isn't the case. "The size of the gender gap in reading does not appear to be related to average performance (Figure II.7.2)" PISA 2018 Results (Volume II): Where All Students Can Succeed, p. 144. 

Socio-Economic Status

On average, the top quarter of students per country in terms of socio-economic status vs. the bottom quarter of students in the same country showed an average reading gap of 88 points, as compared to an average gender-based reading gap of 30 points. Not only is this a larger gap overall, but it consistently beats out the gender gap on a country-by-country basis:

"[I]n all PISA-participating countries and economies, socio-economically advantaged boys outperformed disadvantaged girls in reading (see Figure II.7.6). But in all countries, advantaged girls significantly outperformed advantaged boys in reading, while disadvantaged girls significantly outperformed disadvantaged boys. The only exception is Peru, where advantaged boys and girls performed at a similar level, on average." PISA 2018 Results (Volume II): Where All Students Can Succeed, p. 149.

Reading for Enjoyment

PISA 2018 asked a series of questions related to reading enjoyment. Even if the differences in reading performance (as discussed above) is factored in, 15 year old girls showed reported higher reading enjoyment than 15 year old boys in all participating countries. 

"When asked how much time they usually spend reading for enjoyment, more than 75% of boys reported either none at all or less than 30 minutes a day, on average across OECD countries; less than 3% reported that they read more than two hours a day. By contrast, 43% of girls reported that they read at least 30 minutes a day, and 8% of them reported reading more than 2 hours a day." PISA 2018 Results (Volume II): Where All Students Can Succeed, p. 159.

What I find most astonishing here is that the performance differences are as small as as they are with this 18% difference from 25% of boys reading at least 30 minutes per day for enjoyment and 43% of girls reporting the same.

What did the United States numbers look like for reading enjoyment questions? I gathered the following from the data tables Table II.B1.8.1 & Table II.B1.8.2.

Percentage of 15 year old students who agreed or strongly agreed with the following statements about "reading for enjoyment" (U.S. only):

I read only if I have to — 62% boys; 47% girls
Reading is one of my favorite hobbies — 25% boys; 41% girls
I like talking about books with other people — 32% boys; 48% girls
For me, reading is a waste of time — 36% boys; 20% girls
I read only to get information that I need — 61% boys; 45% girls

I don't read or I read less than 30 minutes a day — 76% boys; 63% girls
I read between 30 minutes to 2 hours a day — 21% boys; 28% girls
I read more than 2 hours a day — 3% boys; 9% girls

Whoa there, why in the world do they clump "I don't read" and "I read less than 30 minutes a day" into the same answer?! I'm a librarian who probably reads something like 20 minutes a day for enjoyment on average.

Another thing that jumps out to me is the gap between the first two questions, from "I read only if I have to" to "Reading is one of my favorite hobbies." U.S. boys gave 62% and 25%, respectively, which leaves 13% boys who take some kind of moderate view of reading. U.S. girls gave 47% and 41%, respectively, which leaves 12% girls with an in-between view. This seems very polarizing! I'm going to suggest that we should be just fine about those students in the middle; reading doesn't have to be your top thing to enjoy it.

Overall Impressions

I admire the way assessment is being done in PISA and the openness of information about the process. (You can read a lot more details via the reference links above!) I feel like I have a much better sense of scale when it comes to these gender-related reading gaps. My take-aways:

• The vast majority of 15 year olds fared similarly in reading proficiency, especially in the middle and high end. So much for "boys aren't good readers."
• There is an issue where boys are dramatically over-represented at the lowest end of reading proficiency. 
• Reading enjoyment shows a larger gender gap than reading proficiency does. In this area, I wish the questions would attempt to explore the issue where only certain types of reading—primarily fictional chapter books—are considered "reading" by many students (and, sadly, by many adults).

The next post in this series will look at another international assessment, PIRLS, which focuses on fourth graders rather than 15 year olds.

Investigating the "Boy Crisis" in Reading

It's been nine years since my last post. In the meantime, I finished grad school then took a series of library jobs with a focus on children's librarianship. I have some questions about the "boy crisis" in reading that I'd like to investigate. Writing in this format has been helpful in developing my own thinking in the past, so I'd like to try for that again. Any benefit to others is a nice bonus.

Overview of Series

1. Investigating the "Boy Crisis" in Reading (this post)
2. Reading Metrics Source — PISA 
3. Reading Metrics Source — PIRLS (forthcoming)

What "boy crisis" and why do I keep putting that in quotes?

There is, particularly in politically conservative circles, a notion that men and boys are in crisis, e.g: this David Brooks NYT op-ed "The Crisis of Men and Boys" and the book The Boy Crisis: Why Our Boys Are Struggling and What We Can Do About It by Warren Farrell and John Gray. Some major areas of concern include: lower academic test scores, higher rates of mental health conditions, devaluing of masculinity, lowering workplace earnings, absence of father figures, and lack of purpose.

I'm putting "boy crisis" in quotes to mark my reservations about the concept. First off, any claim about men and boys experiencing gender-based unfairness must be considered in the context of the long history of women and girls experiencing gender-based unfairness. Even if every "boy crisis" claim were true, it's still orders of magnitude less significant than the "girl crisis" has been. We need to maintain some perspective. Second, this all-inclusive approach is only helpful in getting people riled up. What I see in articles and social media soapboxing is a good deal of reactionary anti-feminism (patriarchy isn't real, men are biologically programmed to be fighters and job holders, etc). That doesn't mean all "boy crisis" claims are wrong or unworthy of concern, but it does immediately put off a lot of people who have that historical perspective in mind. It's a similar vibe to white Americans complaining about racial disfavoring against ourselves, where deep injustice against others is so sad, but any perceived injustice against us is an intolerable "crisis" that needs to be addressed first.

So, I want to focus specifically on boys and reading: test scores, rates of reading for pleasure, perceptions about reading, etc.

My general plan is to look into:

• what claims are being made 
• what data these claims are based on
• which other data variances exist for reading
• how this data can be interpreted
• how concerned should we be
• what we can/should do

I have a suspicion that there are no innate differences in the mental aptitude of boys or girls when it comes to reading (or anything else for that matter). I also suspect that gender essentialism is the cause rather than the cure for any reading disparities that disfavor boys. For example, it may be the case that situations generally disfavorable to girls—such as negative attitudes toward reading narratives from female perspectives—may be a cause for lower interest in reading for boys. As a children's librarian, I've heard caregivers scorn books selected by boys for gender reasons, but I've never heard caregivers scorn books selected by girls for gender reasons.

Note: This is a personal "I did my own research" kind of project, not an academic quality project, so I'm not making any attempt to be comprehensive. However, if anyone feels I'm missing something significant, I would welcome a heads-up.

For the rest of this post, here are some more-or-less representative claims about a "boy crisis" or gender gap in reading:

"Boys are 50 per cent more likely than girls to fail at all three key school subjects: maths, reading, and science. [...] A six-percentage-point gender gap in reading proficiency in fourth grade widens to an 11-percentage-point gap by the end of eighth grade."

— "Why Boys Fall Behind: The gender reversal in education has been astonishingly swift" by Richard V. Reeves, The Spectator, October 15, 2022.

"In 57 countries with data on learning poverty (not being able to read and understand a straightforward text at age 10), 10-year-old boys fare worse than girls in mastering reading skills and adolescent boys continue to fall behind girls in reading skills at the secondary level." 

— "How the Learning Crisis Affects Boys" by Matthias Eck & Justine Sass, IIEP Learning Portal, June 16, 2022.

"Gender isn’t determinative, and the differences between girls and boys aren’t always so stark, but it would be foolish to pretend they don’t exist. This is particularly true when it comes to reading and literacy. Studies consistently show that girls acquire language earlier than boys, have larger vocabularies, and use more complex sentence structures in preschool, and once they reach elementary school are 'one to one-and-a-half years ahead of boys in reading and writing.'"

— "It's a Guy Thing: How to Help Boys Become Readers" by Devon A. Corneal, Brightly.

"[T]he evidence is accumulating that by the time they go on to high school, boys have lost their interest in reading about the fictional lives, thoughts and feelings of mature individuals in works written in high-quality prose, and they are no longer motivated by an exciting plot to persist in the struggle they will have with the vocabulary that goes with it."

— "Why Johnny Won't Read" by Mark Bauerlein and Sandra, Washington Post, January 25, 2005. As recommended by boysread.org.

"Providing equitable opportunities for girls or migrant students has traditionally been at the centre of this effort; providing them for boys is a relatively recent aspect, but it is increasingly becoming an issue of outmost urgency. Throughout Europe, there is dire need for special attention to adolescent boys’ literacy development and attitudes, since this groups is more likely to be at risk."

— "About" page for BoysReading.org. 

"Boys aren’t reading as well as girls, everywhere in the world. International assessments like the PISA data show that boys read fewer books, don’t comprehend as well as girls, and demonstrate less fluency and complexity in their writing. Interestingly, boys don’t start out at a disadvantage. Kindergarten and first grade results are pretty much even for girls and boys. But by third grade the gap is evident."

— "Five Ways to Engage our Boys in Reading" by Paul Kropp, High Interest Publishing.

"Eight assessments generate valid estimates of U.S. national reading performance: the Main NAEP, given at three grades (fourth, eighth, and 12th grades); the NAEP Long Term Trend (NAEP-LTT), given at three ages (ages nine, 13, and 17); the Progress in International Reading Literacy Study (PIRLS), an international assessment given at fourth grade; and the Program for International Student Assessment (PISA), an international assessment given to 15-year-olds.  Females outscore males on the most recent administration of all eight tests.  And the gaps are statistically significant."

— "The Gender Gap in Reading" by Tom Loveless, The Brookings Institution, March 26, 2015.

In the next post of this series, I'll start looking into the specifics of these reading assessments.

On Living Life and Accepting Death

  photo by Lari Huttunen (cc by-nc-nd 2.0)

I don’t believe in an afterlife, which means I do believe in death. It shouldn’t be such a strange thing to believe life ends in death, but most people believe or at least hope for more. Death denial is an understandable impulse; sometimes it even extends to family pets, but less often to other animals. We want ourselves and those we care about to carry on. We won’t. They won’t.

Does the reality of death mean life doesn’t matter? No, it means life is the only thing that matters. You get once chance to exist and it’s happening now. Now is the time to love, the time to learn, the time to create, the time to enjoy yourself and choose to either bring comfort or suffering to others.

What about jerks who prosper in life and kind people who live hard lives? Doesn’t the reality of death mean the world is unjust? Yes. That may sound harsh, but how kind is it to tell people that the suffering and deaths of their loved ones is for the best? It can be disheartening to know we can’t make everything better, but what we can do matters all the more because there’s no other help on the way.

Besides, popular alternatives tend to be worse. At least suffering and injustice end along with life. Mainstream Christian and Muslim beliefs promise unending joy for a select few and unending suffering for most people. That’s solving a house fire with an atom bomb.

Why not just have as much pleasure in life as possible and forget about other people? Well, there’s nothing wrong with pleasure. Pleasure is great and it comes in many satisfying forms! As a loved one says: “No time enjoyed is entirely wasted.” As for ignoring the suffering of other people, moral philosophers have tried in vain to find a reason for completely selfish people to care about others. You have to start with caring a little. Thankfully, most of us do. We don’t have to solve whole categories of suffering on our own; we can cooperate with others, working within the limits of our imperfect empathy and our incomplete understanding to make our lives a little better.

“If I can stop one heart from breaking,
I shall not live in vain;
If I can ease one life the aching,
Or cool one pain,
Or help one fainting robin
Unto his nest again,
I shall not live in vain.”

— Emily Dickinson

Quote of the Day: Jensen on Library Neutrality

"Take a simple example involving the common assumption in the United States that the capitalist economic system is the only rational and morally defensible way to organize an economy. There can be, and often is, much debate about how to structure and administer a capitalist economy, but the system itself is rarely contested, despite centuries of resistance to capitalism around the world and considerable intellectual work underlying that resistance. Now, imagine that a librarian wants to produce a display of the libraryʼs resources on economics to encourage patrons to think about the subject. In many libraries such a display would include no critiques of capitalism, but simply literature that takes capitalism as a given. Such a display that ignores critical material likely would produce no controversy (except perhaps a few complaints from anti-capitalists about the absence of critique, who could easily be dismissed as cranks). It is unlikely that school boards or city councils would take up the issue of the obvious bias against socialism and other non-capitalist economic systems. Consider what might happen if a librarian charged with this task actually produced a display that carefully balanced the amount of material from as many different perspectives as s/he could identify. In many places, that display would be denounced for its 'obvious' socialist politics. Now, imagine that a librarian, observing the way in which Americans are systematically kept from being exposed to anti-capitalist ideas in the schools and mass media, decides to organize materials that compensate for that societal failure by emphasizing critiques of capitalism. That librarian could be guaranteed not only criticism and charges of political bias, but likely disciplinary action.

My point is simply that all of those decisions have a political dimension, which is unavoidable. My concern here is not which one is the right decision, but that the librarian whose display is in line with the conventional wisdom likely will escape criticism while any other choices will raise questions about 'politicizing' what should be a professional decision. Unfortunately, this neutrality game will derail rather than foster serious discussion of the issues."

- from "The Myth of the Neutral Professional" by Robert Jensen in Progressive Librarian Issue #24.

Quote of the Day: Barack Obama on Special Interests

"I've never been entirely comfortable with the term 'special-interests,' which lumps together ExxonMobil and bricklayers, the pharmaceutical lobby and the parents of special-ed kids. Most political scientists would probably disagree with me, but to my mind, there's a difference between a corporate lobby whose clout is based on money alone, and a group of like-minded individuals—whether they be textile workers, gun aficionados, veterans, or family farmers—coming together to promote their interests; between those who use their economic power to magnify their political influence far beyond what their numbers might justify, and those who are simply seeking to pool their votes to sway their representatives. The former subvert the very idea of democracy. The latter are its essence."

— Barack Obama, The Audacity of Hope: Thoughts on Reclaiming the American Dream, 2006, p. 116.

Quote of the Day: Barack Obama on Reading Critically

A 1950 edition.

‘“Sister Regina,” Marcus said. “You know Barack, don’t you? I’m trying to tell Brother Barack here about this racist tract he’s reading.”

He held up a copy of Heart of Darkness, evidence for the court. I reached over to snatch it out of his hands.

“Man, stop waving that thing around.”

“See there,” Marcus said, “Makes you embarrassed, don’t it—just being seen with a book like this. I’m telling you, man, this stuff will poison your mind.” He looked at his watch. “Damn, I’m late for class.” He leaned over and pecked Regina on the cheek. “Talk to this brother, will you? I think he can still be saved.”

Regina smiled and shook her head as we watched Marcus stride out the door. “Marcus is in one of his preaching moods, I see.” I toss the book in my backpack.

“Actually, he’s right,” I said. “It is a racist book. The way Conrad sees it, Africa’s the cesspool of the world, black folks are savages, and any contact with them breeds infection.”

Regina blew on her coffee. “So why are you reading it.”
“Because it’s assigned.” I paused, not sure if I should go on. “And because—“
“Because…”

“And because the book teaches me things,” I said. “About white people, I mean. See, the book’s not really about Africa. Or black people. It’s about the man who wrote it. The European. The American. A particular way of looking at the world. If you can keep your distance, it’s all there, in what’s said and what’s left unsaid. So I read the book to help me understand just what it is that makes white people so afraid. Their demons. The way ideas get twisted around. It helps me understand how people learn to hate.”’

— Barack Obama recounting a college experience in Dreams from My Father: A Story of Race and Inheritance.

How We Know Abstinence-Only Education Doesn't Work

Abstinence-only education does not reduce teen pregnancy. If it did, decades of research would have demonstrated this many times over. Instead, research results have been overwhelmingly inconclusive or the opposite of what its advocates would like to see.

A method of "education" characterized by limiting what is taught had better yield clear practical benefits. At least then there might be a trade-off between knowledge and behavior.

The Rise of Abstinence-Only Education In the United States

Federal funding for abstinence-only programs began in 1982 with the Adolescent Family Life Act, which was part of the previous year's omnibus spending bill (blue "AFLA" line on the chart below). This sent millions of dollars annually to programs aimed at preventing "adolescent sexual activity and adolescent pregnancy." These programs were encouraged to partner with "religious and charitable" organizations, which led to such a degree of religious involvement that a case went up to the Supreme Court by 1988. In Bowen v. Kendrick (487 U.S. 589), the Court decided that the Act was not unconstitutional on its face, but did note that there appeared to be "impermissible" specific applications, which it called for other courts to examine.

In 1998, a major welfare reform dramatically increased funding for abstinence-only programs (red "Title V" line on chart below). This bill defined an abstinence education program as one which:

1. Has as its exclusive purpose, teaching the social, psychological, and health gains to be realized by abstaining from sexual activity
2. Teaches abstinence from sexual activity outside marriage as the expected standard for all school age children
3. Teaches that abstinence from sexual activity is the only certain way to avoid out-of-wedlock pregnancy, sexually transmitted diseases, and other associated health problems
4. Teaches that a mutually faithful monogamous relationship in context of marriage is the expected standard of human sexual activity
5. Teaches that sexual activity outside of the context of marriage is likely to have harmful psychological and physical effects
6. Teaches that bearing children out-of-wedlock is likely to have harmful consequences for the child, the child’s parents, and society
7. Teaches young people how to reject sexual advances and how alcohol and drug use increases vulnerability to sexual advances
8. Teaches the importance of attaining self-sufficiency before engaging in sexual activity

In 2001, additional federal funding began flowing from the Community Based Abstinence Education program (purple "CBAE" line on the chart below). The overall progression has been from low funding in the 80s and early 90s, medium funding in the late 90s, peak funding in the 00s, and back down to medium funding through at least 2014, thanks to $50 million per year allocated as part of the Affordable Care Act. In other words, the red line holds steady for three more years than this chart depicts:

From http://www.siecus.org/index.cfm?fuseaction=page.viewPage&pageID=1340

Two decades and well over a billion tax dollars later, there is no scientific evidence that programs matching the 8-point definition are effective at reducing teen pregnancy.

An Official Investigation

As part of the Balanced Budget Act of 1997, Congress authorized funds for a scientific study of Title V abstinence-only programs. The U.S. Department of Health and Human Services (HHS) contracted this study out to Mathematica Policy Research, Inc. After years of study, Mathematica submitted its final report to HHS in April, 2007:

http://aspe.hhs.gov/hsp/abstinence07/

To maximize the opportunity for positive, reliable results, this study focused on four school programs that were especially intensive and could be especially well documented:

"All programs offered more than 50 contact hours and lasted for one or more school years, making them relatively intense among programs funded by the Title V, Section 510 grant." (Trenholm, 2007, p. 2)

"These four programs are called 'impact sites' because they had program features and staff capable of supporting a rigorous, experimental-design impact evaluation." (ibid., p. 7)

Over twelve-hundred students were involved in these four programs, with over eight-hundred students in control groups. To measure differences over time, follow-up surveys were given from 42 to 78 months after the surveys given at the beginning of each program (ibid., p. 19). It was just the sort of broad-but-detailed study that would have stood up to scrutiny from critics of abstinence-only education. Some conclusions:

"None of the individual programs had statistically significant impacts on the rate of sexual abstinence, whether measured as either always remaining abstinent or being abstinent during the last 12 months." (ibid., p. 30)

"Program and control group youth also did not differ in the number of partners with whom they had sex." (ibid., p. 31)

"Programs did not affect the age at which sexually experienced youth first engaged in sexual intercourse" (ibid., p. 31)

"Forty percent of program group youth reported that they expected to abstain from sex until marriage compared with 37 percent of control group youth, a difference that is not statistically significant" (ibid., p. 32)

"Across the individual programs, estimated impacts on unprotected sex, measured either at first intercourse or in the last 12 months, were likewise small and statistically insignificant" (ibid., p. 34)

"Ten percent of youth in both the program and control groups reported having been pregnant or gotten someone pregnant, and roughly half of them (five percent overall) reported that they had had a baby." (ibid., p. 35)

"[P]rograms raised the proportion of youth who reported that condoms never prevent HIV from an estimated 17 to 21 percent; the proportion who reported that condoms never prevent chlamydia and gonorrhea from an estimated 14 to 20 percent; and the proportion who reported that condoms never prevent herpes and HPV from an estimated 15 to 23 percent." (ibid. p. 46)

Yikes! These golden examples of abstinence-only education programs failed to alter behaviors or even attitudes. They did, however, increase the number of teens who believed condoms were useless for STI protection.

State Studies

A number of states have run evaluations on their own abstinence only programs. Unfortunately, these state studies haven't generally been models of scientific rigor. No control group, or no follow-up, or both! Although abstinence-only education failed to come out looking good in any of the studies, opponents of abstinence-only education should not rely on desired results that come from shoddy methods. Advocates for Youth put together a summary of these state studies here:

Five Years of Abstinence-Only-Until-Marriage Education: Assessing the Impact

Two of these studies did include both a control group and a follow-up. California's 17-months-later follow-up found that program students were no less likely than control students to have become sexually active, pregnant, or infected (Kirby, 1997). The program was cancelled based on these results. Missouri's smaller study had similar findings (Hauser, 2004).

Abstinence Until Ready

In 2010 a rigorous scientific study came out that showed a positive effect for abstinence education:

Efficacy of a Theory-Based Abstinence-Only Intervention Over 24 Months: A Randomized Controlled Trial With Young Adolescents

But there's a catch. This abstinence program deliberately did not match up with federal standards for abstinence-only education:

"It was not designed to meet federal criteria for abstinence-only programs. For instance, the target behavior was abstaining from vaginal, anal, and oral intercourse until a time later in life when the adolescent is more prepared to handle the consequences of sex. The intervention did not contain inaccurate information, portray sex in a negative light, or use a moralistic tone. The training and curriculum manual explicitly instructed the facilitators not to disparage the efficacy of condoms or allow the view that condoms are ineffective to go uncorrected. The results of this trial should not be taken to mean that all abstinence-only interventions are efficacious. This trial tested a theory-based abstinence-only intervention that would not meet federal criteria for abstinence programs and that is not vulnerable to many criticisms that have been leveled against interventions that meet federal criteria." (Jemmott, 2010)

The study's authors suggested a role for this kind of modified abstinence education program: an improvement over federally-defined abstinence education in communities that will not allow comprehensive sex education. It still lacks much of the information of comprehensive programs, but at least it doesn't encourage false and fearful beliefs...which evidently don't help anyway.

Correlations

Good news! Teen pregnancies, teen abortions, and births to teens have been falling:

Can this be attributed to abstinence-only programs? Perhaps the studies above are accurate within their particular contexts, but are missing out on big picture trends. If the states that require or emphasize abstinence-only education are generally the states with lower pregnancy rates, then it might be worth looking further into abstinence-only education. Someone did, in fact, look for this pattern.

Using information on state laws and policies in 2005, researchers assigned each state with relevant laws or policies a level from 3 to 0 (Stanger-Hall, 2011):

• Level Three - abstinence-only education, according to federal guidelines.
• Level Two - abstinence stressed, but discussion of contraception methods not forbidden.
• Level One - abstinence covered as part of comprehensive sex education.
• Level Zero - abstinence not specifically mentioned in sex education.

How did the states do?

So much for abstinence-only showing promise in the big picture. Here are comparison charts for pregnancies, abortions, and births (ibid.):

As far as teen pregnancy goes, abstinence-only education may actually be worse than sex ed that never mentions abstinence as an option!

Conclusion

Other issues aside, abstinence-only education does not improve teen abstinence. Its advocates should at the very least be seeking to reform it to be more like the "Theory-Based Abstinence-Only Intervention" mentioned above that went against federal guidelines and showed promise. Personally, I suspect the focus on postponing all sex until marriage is so unrealistic (and not even a worthy ideal) that teen audiences are lost to the positive message that it's OK to wait until both people are ready to make a considered, responsible choice.

References

Kirby, D., Korpi, M., Barth, R.P., Cagampang, H.H. (May/June 1997). The impact of the postponing sexual involvement curriculum among youths in California. Family Planning Perspectives, 29(3). Retrieved from http://www.guttmacher.org/pubs/journals/2910097.html

Hauser, D. (2004) Five years of abstinence-only-until-marriage education: Assessing the impact. Retrieved from http://www.advocatesforyouth.org/storage/advfy/documents/stateevaluations.pdf

Jemmott, J.B., Jemmott, L.S., Fong, G.T. (2010) Efficacy of a theory-based abstinence-only intervention over 24 months: A randomized controlled trial with young adolescents. Pediatrics & Adolescent Medicine, 164(2), 152-159. Retrieved from http://dx.doi.org/10.1001/archpediatrics.2009.267

Stanger-Hall, K.F., Hall, D.W. (October 2011). Abstinence-only education and teen pregnancy rates: Why we need comprehensive sex education in the U.S. Plos ONE. Retrieved from http://www.plosone.org/article/info:doi/10.1371/journal.pone.0024658

Trenholm, C., Devaney, B., Fortson, K., et al. (April 2007). Impacts of four Title V, Section 510 abstinence education programs final report. Retrieved from http://aspe.hhs.gov/hsp/abstinence07/