This is the second in a series of essays on the use of Arbitron ratings as a tool for Pacifica programming decision-making. While the information is specific to station KPFA in Berkeley, Pacifica's general trend of dealing with their audiences in the same manner as commercial audiences is clear in these reports.

SUBJECT: ARBITRON RATINGS AND METHODOLOGY

This message deals with the way Arbitron conducts its surveys. This information is not essential to understand the rationale behind KPFA's changes, but it is important if you wish to be able to discuss the validity of the Arbitron/AudiGraphics information and its applicability to KPFA.

Arbitron conducts four 12-week surveys per year (with one-week gaps between surveys): Spring (Apr-June), Summer (June-Sept), Fall (Sept- Dec), Winter (Dec-Apr). Exact dates vary from year to year.)


Obtaining Random Phone Numbers
The first step is to obtain a random set of phone numbers proportional to population in the various counties of the area surveyed. This includes numbers randomly selected from listed numbers, plus computer- generated possible unlisted numbers (which now make up well over half the number of residential phones, and so must be included to avoid a serious bias). The total sample for the survey is divided randomly into roughly equal weekly groups. For listed numbers, a letter is sent first to explain that an Arbitron interviewer will be calling. The initial telephone contact is used to reject anyone who admits to a media affiliation, to ask consent to participate in the survey, and to find out how many persons of age 12+ live in the household. Businesses and households with more than nine persons of age 12+ are rejected (in other words, dorms and group homes of various kinds are not included in the survey). Information on the race/ethnicity and demographics of the household are obtained. Every effort is made to reach each of the phone numbers initially selected, with calls being made at various hours of day and evening several times.

A one-week diary for each person aged 12+ is mailed. Phone followups verify that the diaries arrived and offer help if there are any questions about procedure. At the end of the week, another call is made to remind about mailing the diaries and to be sure that all instructions were understood. Bilingual (Spanish-English) diaries are provided for those who identify themselves as Hispanic or for anyone who requests one. Small payments are offered for completed diaries.

Some of the submitted diaries are rejected (incomplete diaries, those sent in before the end of the week, those lacking essential information). Entries other than call letters (slogans, frequencies, names of hosts or programs, invalid call letters, etc.) are allocated if possible to valid station call letters. The diary mentions for each quarter hour of the week are computer-entered and tabulated.

Weighting the Variables
A weighting system is used to compensate for deviations in the sample from the desired balance of demographic factors. For example, if fewer usable diaries than desired are received from Marin County or from African-American women, then each diary used from the under-represented category will be counted as slightly more than one diary.

In the fine print at the back of each report is a section on how to compute the reliability of results. For example, does a value of 2.2 mean somewhere between 2.1 and 2.3 or does it mean somewhere between 1.5 and 3.0? It takes a dedicated person with statistical training to wade through that section and actually check out the reliabilities. Believe me, none of the KPFA management do so, and the consultants have no interest in emphasizing the unreliability of the data on which they earn their livings.

Because Arbitron reports are used mainly to establish advertising rates and to sell advertising for particular stations, only commercial stations are listed in the regular Arbitron books. However, the CPB [Corporation for Public Broadcasting] has paid for each public radio station to get an Arbitron report including all the noncommercial stations once a year. . . .


Exclusions
Some things to note. Non-telephone households are excluded. Group living situations are excluded (more than nine persons of age 12 or above). People not fluent in English or Spanish are probably excluded. Although the survey is weighted to give the appropriate balance by county, age, sex, and ethnic group, it almost certainly over-counts people who like to participate in surveys and under-counts the kind of people who are afraid or hostile to such surveys. People with media affiliations can be included if they lie about it. It should also be noted that KPFA programmers often fail to mention any identifying information about the station or the program for periods of several minutes or even an hour. Thus, KPFA is likely to fail to get credit for any listeners who happen across us and listen for a brief period, but never become regular listeners.


Example - Spring 1993 Survey
The information I have at hand comes from the Spring 1993 survey. For that survey, the number of households in the Metro Survey Area was estimated at 2,326,600, with a total population estimated at 5,262,000. The total number of diaries tabulated was 4,483. For the Total Survey Area, 9,522 diaries were tabulated to represent an estimated population of 8,519,200. Thus each diary was assumed to be representative of about 1,200 people in the Metro sample, or about 900 people in the Total Survey Area. Let's say roughly that each diary represents the listening habits of 1,000 people. I think that Arbitron may have significantly increased its sample sizes since 1993, so just to be generous, let's assume that each diary represents 500 people.

This means that the 18,000 or so KPFA subscribers (I'm not sure what the current figure is) would be represented by about 36 or less diaries. You can see how well the variety of listening patterns will be reflected in this survey! More recent surveys indicate that KPFA's Cume Persons runs around 110,000 to 150,000. In other words, somewhere around 300 or fewer diaries mention KPFA at all. This is enough to give a reliable picture of our total listenership (which doesn't vary much at all from survey to survey) and of our peak listenerships for the Morning Show and the KPFA News. However, at many other times of the day, we are talking about 20 diaries or less, so that variations of several percent from survey to survey are common. Remember that when you try to get down to details (particular programs, particular age groups or ethnic groups), you're fundamentally dealing with a survey of 300 or fewer KPFA listeners --not a very big sample on which to draw conclusions.


Reading the Results
My own opinion is that you need to compare at least three or four consecutive surveys before drawing any conclusions, placing faith only in those patterns that seem consistent through nearly all surveys. If somebody wanted to wade through those complicated calculations, it would be interesting to see what kinds of error ranges we are talking about on the numbers that are being used for planning purposes.

One final note: Three of the four surveys each year will include at least a week of marathon programming, so listenership data for particular dayparts may be significantly altered by nonstandard marathon programming. Any preemptions for hearings, live coverage of marches, special holiday programming, etc., would similarly distort any attempt to draw conclusions about individual programs. In my time at KPFA, I have never known anyone (or any consultant) to bother to try to figure out how much the results are affected by these deviations from the program schedule, or to take any account of them.

Okay, there's a bit of background and a few suggestions about appropriate places to be skeptical. Next time around, we start looking at what the consultants were telling KPFA about their listenership.

MORE ON ARBITRON

We believe decisions about our stations should be made by the representatives of all the communities that have a stake in Pacifica, and should occur in an open, accountable, community-oriented process . . . something now lacking at Pacifica.