Pollsters Struggle for Accurate Picture of Mideast by Carl Bialik – The Numbers Guy – Wall Street Journal – February 26, 2011

At the height of this month’s protests in Egypt, pollsters phoned hundreds of Egyptians on their cellphones and landlines to ask them, among other things, who should be the next president of the country. Just 16% named Hosni Mubarak, who hadn’t yet stepped down.

As much as the results, the poll itself was a reflection of the growing efforts by governments, companies and others to gauge public opinion in the Middle East. Until the last decade, few major international polling organizations surveyed in the region. Today, many do, including Gallup and Pew Research Center, producing data on what Arabs and Muslims think about the U.S., religion and al Qaeda. But pollsters work under severe restrictions. Some comply with regimes that review questions and delete touchy ones. Others avoid polling during the month of Ramadan. And most conduct the interviews in person, which tends to yield better results, pollsters say, but is far more costly and time-consuming.

The challenges in the region leave many gaps in the data. Gallup polls daily in the U.S., with results available the next day. By comparison, a national, face-to-face survey in Egypt could take a month. And without a critical mass of polls, it is difficult to gauge the reliability of the numbers.

The poll in Egypt, conducted by Princeton, N.J.-based Pechter Middle East Polls among 343 Egyptians between Feb. 5 and Feb. 8, was carried out by phone using interviewers based in the United Arab Emirates, a strategy designed in part to avoid political interference. Pollsters also reasoned that Egyptians would be reluctant to open their homes to strangers during the political turmoil. It also was restricted to Cairo and Alexandria, Egypt’s two biggest cities.

Adam Pechter, who founded the polling house two years ago, says these factors might have diminished the poll’s accuracy. Still, he says, it helped shed light on what Egyptians thought during a tumultuous time.

Two other polls of Egypt conducted in 2010 underscore how geographic focus might skew results. In a survey limited to urban areas, Egyptians’ support for the U.S. appeared to be rising. But in a poll that attempted to canvass most of the country, support for the U.S. was declining.

Experienced pollsters learn to account for special factors in the region. For instance, survey researchers avoid asking questions during Ramadan, a period, several pollsters say, when it can be more difficult to persuade people to participate in surveys.

When they poll at other times of year, researchers generally find very high rates of cooperation in the Mideast: Sometimes more than 90% of people they contact participate in the survey. Participation has been particularly high in Iraq, an effect pollsters attribute to high-stakes issues and the relative infrequency of polling compared with the U.S., where most people decline to participate.

Also, pollsters who want to carry out in-person surveys can find ways to work around barriers erected by restrictive regimes. Steven Kull, director of the University of Maryland’s Program on International Policy Attitudes, says indirect questions are less likely to be censored by governments. He points to questions on a 2008 survey managed by the center asking people in 19 countries and territories, including five in the Middle East, to rate two things on a scale from one to 10: How much their country is governed according to the will of the people, and how much it should be. In every country, including the U.S., the results indicated that people wanted a government that reflects the will of the people more than the current government does. Egypt had the second-largest gap between the two sets of responses of any country, which Dr. Kull points to as evidence of popular support for the recent uprising.

Sometimes, seemingly innocuous questions are included in polls to draw attention away from political topics. Mr. Pechter says his company also has hired local market-research firms to slip political questions into mainly nonpolitical surveys, which generally aren’t screened.

Still, pollsters generally are reluctant to disclose their methodologies for Middle East polls, citing the risks of working there. That makes it difficult to evaluate their results. And some pollsters including Gallup and the University of Maryland research center, allow governments to screen polls. So, in Gallup’s latest round of polls last year there are no approval ratings in Egypt or Libya for those countries’ leaders, a question asked in many other countries around the world.

Some pollsters are hopeful that the political upheavals in the Middle East will bring more freedom to their field. “I believe this is changing right before our eyes, as people break away from the police-state mentality,” says David Pollock, a former State Department adviser and cofounder of Pechter. “You can almost see a surge of media freedom.”

Write to Carl Bialik at numbersguy@wsj.com