Oakland’s “shrinking” population and the American Community Survey

So, after much discussion last week over whether or not Oakland is experiencing an “exodus” due to crime, we now have a Trib article weighing in with Census Bureau data suggesting Oakland’s population has decreased dramatically since 2000. First, I’ll note that the Census Bureau’s official 2007 population estimate for Oakland is 401,489, not 372,000 as the article suggests. So where does the smaller figure come from?

So, there’s this thing called the American Community Survey. Wait. Let’s back up. So there’s this thing called the decennial Census. Every ten years since 1790, the government tries to count everyone living in the US. When we started, this was accomplished by sending US Marshals out on horses to look for settlers. Now, it’s this massive operation that involves tons of advisory committees, setting up regional offices, hiring hundreds of thousands of people, and costs billions of dollars. Because counting every single person in a country of over 300 million is…well, it’s really hard. And over the years, the Bureau has found that some populations are harder to count than others, particularly children, racial and ethnic minorities, and more generally, those of large cities. After an embarrassing debacle with the 1990 census, which the Bureau eventually concluded had undercounted the US population by a whopping 16 million people, we took a number of steps to ensure greater accuracy in 2000.

How to best do this was a subject of much controversy, and eventually, a Supreme Court case, which is actually a pretty fascinating story on its own, and one I might try to tell here someday during a slow news week or something if people are interested. The short version is that the Bureau wasn’t allowed to do the count the way they wanted. Instead, they relied on a whole lot of fieldwork, partnerships with local government, technology, and more user-friendly forms. The standard form, sent to 83% of housing units, asked only seven questions (the smallest number since 1820). The remaining 17% of households received a longer form (called, um, the “long form”) with an additional twenty-seven questions reflecting more detailed demographic information.

The government has traditionally used long form data to create the social and economic community profiles used in allocating federal funds. Problem is, ten years is an awful long time, and one in which significant demographic changes can occur. So the Bureau decided to address this issue by initiating a project to replace the long form with something called the American Community Survey. Surveys with the same questions as the Census long form are sent randomly to 250,000 households every month, for a total of 3 million households annually. These responses are then used to create annual demographic profiles for all states, cities, counties, and metropolitan areas with a population of 65,000 or more. The Bureau began testing the ACS in 1996 and began full implementation of the project in 2005.

So last week, the Census Bureau released a new set of ACS estimates, based on the combined survey results from 2005 to 2007. That’s what today’s Trib story, “Exodus of blacks saddens Oakland” is about. And I know I already said this, but the point is not made in the story, and it’s really important, so again, I’ll say that this is a demographic profile, not an official Census Bureau population estimate. Although the ACS three year survey results mark Oakland’s estimated population at 372,247, the Bureau’s official 2007 estimate for Oakland is 401,489.

So why would the numbers be different? For starters, the ACS numbers do not reflect group quarters populations. That’s people living in nursing homes, residential treatment centers, group homes, halfway houses, dorms, shelters, and so on. In 2000, this was estimated as 7,175 people in Oakland. The ACS is now starting to try to count group quarters, but the challenges of doing so are ongoing. Hopefully, the issue will be resolved within a few years.

The second problem, one which I think will be more difficult to resolve, is the difficulty in getting accurate responses from traditionally undercounted populations. Oakland’s significant racial and ethnic diversity is a blessing in many ways, but a curse when it comes to getting good population counts. Because the ACS does not enjoy the same level of resources when it comes to field workers and follow up as the decennial Census, it is difficult to ensure a satisfactory response rate, particularly among populations where significant distrust of the government is a common characteristic. The detailed questions on the survey are viewed by many recipients as frightening and invasive, weighting methodology is a work in progress, and ensuring adequate funding for the project has been an ongoing problem in Congress.

So what does this mean for Oakland? Well, the failure to count group quarters is a problem, and likely a significant factor in the undercounting of our African American population. How significant the problem of non-response is – well, that’s harder to tell. I think it’s fair to say that while we don’t necessarily have good reason to dismiss the estimates out of hand, we do have good reason to question them. Ultimately, we’ll have to wait until we get the results of the 2010 Census to determine how good a marker of demographic trends the ACS has turned out to be.

32 thoughts on “Oakland’s “shrinking” population and the American Community Survey

  1. dto510

    Oakland’s population and demographics are controversial, of course, because they purport to provide evidence to the otherwise counter-intuitive “gentrification” and “displacement” argument some people make against building new housing (that somehow building new housing causes people to move out of existing housing). The EBX says that Oakland’s population is shrinking because wealthier people with smaller households are “displacing” poorer people with bigger households. The Census says Oakland has 401.5k residents, a bit down from 2002, while the CA Dept of Finance says Oakland has 420k residents as of Jan 1, an increase.

    One very clear way to tell if a city is growing or shrinking is transportation. Car traffic is worse (though nowhere near Oakland’s enormous road capacity), bus ridership is up, and by one measure bike trips have doubled since 2004. Where are all those people coming from if the population isn’t increasing?

    There’s another point here, that the media’s portrayals of the ACS numbers are misleading. They present the numbers as a population estimate, when in fact they are not. There’s a disclaimer to that effect right at the beginning of the report! But that’s not as exciting a story, I suppose. Rather than “black exodus,” we’d have “demographic change” and “stable population” in the headlines. Not so sexy.

  2. Max Allstadt

    Causality is whole huge can of worms to get into on this issue.

    I could guess at a number of causes, sinister and innocent, that partially explain the change. Which of these causes are most responsible for the change is not something to guess at, but something to research, and I’m not up for that challenge.

    Here are some possible causes:

    Generational shift:
    The black families that moved to Oakland in the 40s to work in the port have been through two or three generations now. If one of those generations was successful enough to pay off a mortgage, the generation that followed would have had essentially free money in the form of a house. If that generation had more than one heir, it might be attractive to sell the house, split the proceeds, and use them as a down payment somewhere cheaper. Similarly, with only one heir, the action might be selling, relocating and using the rest as college tuition for the grandkids, or retirement funds.

    Death of Port Work:
    Past generations saw union port jobs passed along from father to son. This generation has seen port jobs passed along from son to robot.

    It happens. I could see parts of Prescott becoming very gentrified within the next decade, but it’s moving very slowly. In the five years I’ve been here, I don’t think I’ve seen any neighborhood go from poor to swanky. That’s the kind of thing that happens in NYC, but not here. Correct me if I’m wrong. Where is gentrification happening in Oakland? Where is it complete? Again, I see change, but nothing radical.

    Asian and South American immigrants seem willing to live in denser proximity to one another compared to whites and blacks. So despite the small geographic areas of Chinatown and Fruitvale, perhaps additions in these groups have changed the percentages in our pie chart. If we add non-blacks without losing black Oaklanders, percentage of black Oaklanders could change without the total number changing.

    Migrant Culture:

    If a great many of Oakland’s black families were willing to migrate in 1941, these families could have within them a culture of migration. My family does. My maternal grandparents were born in Texas, my mother in Illinois, me in NYC, and my children will likely be born in California.


    Anything I’m missing?

  3. Max Allstadt

    Forces that shift people back and forth that are worth understanding, whether or not there’s a net change.

    I do agree that raising the gentrification alarm without really good data is a problem. Unless you’re trying to sell papers.

  4. Max Allstadt

    wait, v, you had a comment, i responded to it, now it’s gone, rendering my response nonsensical.

  5. V Smoothe Post author

    What are you, some kind of machine, Max? I accidentally hit submit before I finished typing what I wanted to say, and deleted it within 15 seconds of posting!

  6. dto510

    Max, a few points: first, you seem to be assuming that Oakland’s population is shrinking, when even the Census says it’s not (though they say it’s flat, not increasing as the state says and transportation trends suggest). Second, the Port containerized in the 60s, the major job losses happened then. Overall, port employment was up until the recent layoffs (which probably aren’t actually in effect yet). I agree about the other trends.

    To answer one question – some areas of Oakland have become much nicer and more expensive. The Downtown Lake Merritt Apartment District was crime-ridden and scary until the late 90s / early 00s, when rents went from below-market to market-rate. It is a much, much nicer neighborhood now. Old Oakland basically didn’t exist ten years ago; the majority of housing is now condos. From a longer view, when I left HS in 1997 Temescal was less ghetto but now it’s a bonified desirable neighborhood (somewhat like Fruitvale). Rockridge is probably the best example of gentrification and people being priced out, though that started in the 70s. Of course, Rockridge’s gentrification happened in the absence of new housing construction, and to a great extent it’s because no new entry-level housing was built that tiny bungalows cost three-quarters of a million dollars and apartment are scarce.

  7. James H. Robinson


    I like your assessments.

    By the way, why do people assume gentrification is a bad thing?

  8. Robert

    I don’t know whether the overall population is up or down in Oakland. Traffic, however, is highly correlated with the economy. And while traffic was certainly much worse in 2007 than in 2003, that was almost certainly related to the overall economy. (I can’t really judge traffic in 2008 since my commute patterns changed.)

    I do think it interesting that the black exodus highlighted in the article is fully consistant with the decline in student enrollments in Oakland. I would take that as a confirming piece of objective evidence for the changing population.

    But as V suggests, we will have to wait until 2010 to find out. And then ‘all’ we will have to deal with is the actual counts used for representation vs. the statistically adjusted counts used for allocating monies.

  9. dto510

    Robert – Traffic is correlated with the economy, specifically with jobs, which are the main driver of population growth in our mobile society. Street traffic, bus ridership, and bike ridership, are correlated with shorter trips than freeway traffic or BART ridership (also up). While more jobs could mean more people commuting from out of the city, it also means more people living in the city.

    While students are declining overall, in areas with better schools, the public schools are overcrowded. There’s also a demographic trend – the millenials are leaving the system (half them graduated HS by or before this year), though Gen Xers are now having their kids so there are more babies but not necessarily more schoolchildren (yet). So, the picture is muddled. We will indeed have to wait until 2010 to figure it out. We’ll also have to grapple with City Council redistricting…

  10. V Smoothe Post author

    Okay, trying again.

    So, I think that before speculating on the causes of sweeping demographic changes, it’s important to establish that they’ve actually happened, which is not something that has actually been done.

    Having said that, I wouldn’t be surprised in the least if the Census 2010 reflects African Americans as a smaller percentage of Oakland’s population than in 2000, with increased representation of Asian and Latino populations.

    The beginning of the Trib article in question suggests that the high cost of housing is the primary factor in the alleged “exodus.” But looking at real estate listings in the neighborhood being described, there are tons of houses for sale for under $100,000, and few over $150,000, so I don’t think housing affordability makes that much sense right there.

    Of course, there are plenty of reasons someone wouldn’t want to purchase an $89,000 house near 89th and International. High crime, for one. And for anyone with children, the poor quality of the schools.

    We’re definitely seeing a declining African American student population at OUSD. For the 1999-2000 school year, African American students represented 48.4% of OUSD’s enrollment, with 26,650 students. For 2007-2008, that was down to 36.2% of enrollment, or 16,819 students. It seems logical to me that any family with children and with the means to afford a home in a suburb would choose to leave Oakland in favor of somewhere with better educational opportunities.

  11. dto510

    So, it’s not so much that families can’t afford to live in bad neighborhoods in East Oakland, it’s that they can’t afford to live in Montclair where the schools are good. Thus all the people in the article living in Sacto, which is more expensive than most of Oakland.

  12. Patrick

    And that’s when people like me (single, white, cash for down payment and home renovation) move in. Voila! Gentrification.

  13. Robert

    V, you probably shouldn’t be using current house prices to figure out possible causes of demographic changes in 2007. Prices, even around 98th and E. 14th were still quite high for what you got back then. On the other hand, I would agree that the c**p Oakland schools are a likely cause of a shiting population. And it is hard to know right now whether that loss is made up for by young single folks moving downtown.

  14. V Smoothe Post author

    My use of current house prices was meant as a response to the barbershop employee, a Sacramento resident, saying that the neighborhood was too expensive to live in now, not as a broader point about demographic changes over the last 8 years. I mostly just wanted to point out that affordability isn’t the only factor in people’s decisions to leave Oakland.

  15. Coolhand Luke

    I feel like in the last 5-10 years the area surrounding the West Oakland Bart has been rapidly gentrifying. Growing up, the only friends I had from West Oakland we’re black, now I have white friends who are relocating into industrial live work lofts and artist studios over there. Again, a whitening of an area does not automatically equate with gentification, but it is an indicator for sure.

    James, gentrification is a controversial issue, but I do think it is a bad thing. Community development is not bad, but displacement of populations with historical ties to an area in the interest of improvement is a bit more dicey. Immigrant communities turn over as new groups come in, but when class and race privileged folks resettle an area at the expense of others, I think it’s a bit different. (Check out this interview I did with Danny Hoch on gentrification http://wiretapmag.com/arts/43902/. Very over the top to one side, but he makes some interesting points.)

    I don’t think it’s bad for folks with money to move into struggling neighborhoods, because I think we need to pump money into struggling areas. I just have a problem when a bunch of white and Asian folks buy condos in complexes like City Limits and Citizen’s Walk in the Golden Gate neighborhood of northwest Oakland without ever intending to invest in the area. By their own admission many of those folks bought to sell (economy malfunction unforeseen).

    Though their money may help the area, their disinterest in the neighborhood’s deeper problems is frustrating. They raise hell about prostitution because hookers are on their street, but only care as in pertains to their property value. There are neighborhood activists who have been trying to make the neighborhood better for years, and though they see the new money as a positive, when their new neighbors pull their cars into their condo complex and take an elevator to their door, its hard to harness that money and influence for anything productive.

  16. Max Allstadt


    Shrinking was not an assumption, actually. I accept the theory we seem to have going: static total population, with shifts in the racial pie chart.


    Gentrification to me is bad if it wipes away all traces of poor or middle class people from a neighborhood. The result it a Mill Valley-esque lala land. You might find that appealing, but when it creates displacement I don’t like it. I also don’t like the idea of creating places where children can grow up living like sheltered little Gautamas. Isolation from the class diaspora will stunt a kid’s emotional and social growth.



    I rent my vacant lot to a Pastor who uses it for church parking on Sundays. He tells me he needs the space because his parishioners have moved as far away as Pittsburgh and Antioch, but they still commute to their old church. He’s also mentioned that some of them are starting to move back to Oakland (though not Ghost Town), because over the years the city’s gotten safer and nicer neighborhoods are less racist than they once were.

  17. len raphael

    african american population is probably undercounted, but would expect it to be consistently so over the years. undercounting for poorer families worried about losing welfare benefits, maybe ex-cons living somewhere other than registered, general belief that nothing good comes in the mail in official envelopes. but besides the school enrollment declines (i assume oakland’s awful truancy rates doesn’t affect that), anecdotal evidence supporting black exodus is easy to find. eg. note how the huge mural on the side of the grand ave condo project across from the Y dropped the black yuppie scene. or just startle the next african american comcast tech if he or she lives in oakland. all of the ones i’ve asked grew up in oakland and moved out as soon as they could because they wanted to raise families in a better place than they had. it’s one thing to be a bleeding edge yuppie risking mugging, theft, rape. you can build high fences, get big dogs, etc. But young oakland working class blacks risk getting sucked down or killed by sticking around.

    de-blackification of temescal was modest over the past 30 years compared to the decline of old ethnic white people. and there has been some number of younger blacks moving to temescal. but go below telegraph and close to Alcatraz to find entire blocks occupied by younger white families who weren’t there 10 years ago.

    i’d expect a huge undercount of latinos compared to prior counts and estimates for entire extended families careful to avoid extradition.

    -len raphael

  18. Chris Vernon

    Your comments denigrating the whole of OUSD need to be addressed. There are many parents of means in Oakland with school age children that a) are not choosing to leave Oakland and b) are sending their kids to the public schools – albeit, only some of them.

    My children went to Peralta Elementary in the 90′s when you could walk right in the first day of school each year and enroll. It was basically the same school then with a similar faculty and staff but quite a different demographic compared with 2008. It was a good school then, it now has a great deal more bells and whistles and is quite beautiful. It’s also now very difficult to land a spot for your neighborhood kindergartener there.

    Oakland Tech, where my son graduated in 2007 and my daughter started as a 9th grader this fall, is also experiencing a resurgence of interest. Before you decide, come see the remarkable and long standing Paideia classes (honors and AP humanities program) and the Engineering Academy. Year after year, graduates of these programs go to the best, and I mean the best universities and colleges in the country. Tech’s Advanced Placement pass rate is comparable to Berkeley High’s and over 400 families interested in coming next year came to tour the school this fall. The enrollment of 1800 students is the highest it’s been in years and the school is closed to additional students this year. Parent involvement and fundraising have gone way up. Parents spearheaded amazing efforts to bring back performing arts two years ago and the creation of a phenomenal new baseball field last year (come see it – behind the MacDonalds at 45th and Telegraph). This year another remarkable program was launched – an ice skating class at the Oakland Ice Center. Over 100 Tech students are skating 3 mornings a week between 7:00 and 7:50 am before school. The kids have been helped by members of the San Jose Sharks and OPD and OFD members. Last week a benefit match between OPD officers and Oakland Firemen and Firewomen netted over $6,000 towards the purchase of equipment to start an ice hockey team at Tech. I could go on, but this gives you a taste of what’s happening there.

    I don’t deny that there are continuing difficulties at Tech – many African American males are not achieving academically; a declining, but still too high dropout rate; no locker room attendants due to slow HR processing downtown so things are stolen from gym lockers; some teachers are resistant to such basic upgrades as utilization of the excellent web-site and implementation of online grading and homework assignments; no capacity for on-campus lunches and resultant problems in the surrounding neighborhood,etc.

    BUT – my kids and many other kids from around the city can get an excellent education there if they are willing to do the work. The college preparation classes are, for the most part, excellent. In addition these kids get an invaluable lesson in what it’s like to rub shoulders with people of many different races, social and economic circumstances. College admissions professionals constantly talk about the mature world-view coupled with their academic success that many Tech kids bring to their schools in comparison with kids from cloistered and exclusive private high schools.

    There are other public schools that have similar success stories in Oakland, mostly in wealthier areas, but not solely.

  19. len raphael

    cv, tech does great job for it’s top 5% or so. arguably better than berkeley high, definitely better than Orinda, and even more so if tech grads still get extra points to get into ucb.

    how is it doing for the rest of it’s students? this isn’t a leading question, but what percentage of the kids who start as freshman, graduate within say 5 years? if it’s better than the 33% or so that it was when my kids attended, that wb a welcome improvement.

    and how does it do for kids somewhere in the middle as compared to say going to school in Piedmont? (yeah, not an option for most, but a goal)

    am curious to see how much street safety has improved: do most kids in the Paideia program when your kid attended, get to school themselves or were they driven by parents?

    -len raphael

  20. Navigator

    Chris, I’m very happy to hear good things about my former high school. You’re so right, many people dismiss Oakland Tech, and many other schools in Oakland, out of hand. My sister also attended Oakland Tech, did her work, received excellent grades, and now is a very successful Civil Engineer.

    I think many parents worry about the safety issue. There’s a perception that high schools in Oakland, and in other urban communities, are not safe. Perhaps a working relationship and partnership with the local communities can go a long way in alleviating some these concerns. Also, the school needs to emphasize civic responsibility, civic pride, and teach students respect for the surrounding community.

    Chris, let’s be honest, the area along Broadway near Tech, is a trash strewn graffiti marred landscape. Students need to be taught respect for their city and the neighborhood surrounding the school. There is no excuse to treat those streets as a landfill for fast food containers. Unfortunately, these kids need to be taught the basics of good citizenship in the classroom since they don’t seem to be getting this at home. The fact that you don’t throw your trash on the ground and you don’t vandalize private and public property needs to be taught in class. Perhaps a Tech sponsored neighborhood cleanup may go a long way in making the school a valued and contributing part of the neighborhood. We need to find a way to tear down the barriers of fear and mistrust between the students and neighbors.

  21. Deckin

    I’m not sure we’re giving adequate voice to the modest idea that the estimates might actually be not so bad. It seems that the ACS is getting run down for its estimates vis-a-vis the Census Bureau’s, but absent any other data, why assume the latter is better? As for the guy in the article claiming that the numbers don’t seem right–were’s he get that? Lots of things don’t ‘seem right’ to me, but I’ll take the ACS’s data over some guy’s ‘seeming’ any day.

  22. Max Allstadt


    If broadway by Tech was a place worth caring about, maybe the kids would. A Kragen, a BK, and a tile store on the tail end of auto row’s worst piece of sprawl don’t inspire much local pride.

  23. V Smoothe Post author

    Deckin, it isn’t “vis-a-vis the Census Bureau.” The American Community Survey is a product of the Census Bureau, intended to convey “demographic characteristics.” The Census Bureau separately puts out population estimates. You cannot view an ACS fact sheet on the Census Bureau’s website without seeing a disclaimer that says the Census Bureau does not consider these numbers accurate population estimates.

  24. Deckin

    V, fair enough, but as you well know, there’s a big difference between bias and error in statistics. If the ACS surveys are consistent, then that they may be biased (by ignoring group living, etc.) doesn’t matter to an inference about population trends, so long as the bias has been consistent. If you think, on the other hand, that their survey methods are unreliable in that the produce error (misleading results but in no consistent direction), then I guess I’d want some evidence of it. As far as I can tell, the ACS results are probably a good indicator of general trends, and that’s probably all one could get with their methodology. And I’m not sure that total population is a good test of their conclusions about trends amongst sub-populations. If they accurately predicted past, well documented (i.e., backed up by Census data) population trends even while getting total populations counts wrong (which I believe is the case), then it’s really apples and oranges.

  25. V Smoothe Post author

    Well, the ACS has never accurately predicted past, well documents population trends backed up by Census data because it hasn’t existed long enough to ever be tested against those numbers.

    I can’t quite understand what you’re trying to say with respect to consistent bias. The ACS, again, is too new to have ever shown any kind of population trend. So I can’t see how you’d go about trying to evaluate it for consistency. Consistency with what?

  26. Art

    The idea behind the consistency is that if you’re counting the same way every year, then even if you’re undercounting certain groups on a regular basis (a bias), you still have comparable data sets. (I know there are x people there, but ACS says y. That’s 80% of x. Each year, I can guess the “real” number using that formula.) If it’s not a consistent undercount, it’s error and there’s not much you can do about it.

    ACS has its issues, but I work with population data on a regular basis, and going into 2009, the 2000 Census data is just about worthless. This is true for about half of every Census cycle—we won’t get 2010 data till late 2011 or early 2012, and then it will be good for 4-5 years before it’s so outdated that it’s irrelevant by 2018. ACS tries very hard to fill the gaps. They’re not perfect, but if you know what some of the pitfalls are, you can still use those data until new Census data become available. (And for what it’s worth, the Census also notoriously undercounts some groups, so I’m not sure how much there is to gain by weighing one data set against the other to determine which is right or wrong.) The best way I’ve found to look at Census/ACS data is to weigh them against good local or state measures, where these exist, to see where the discrepancies are and why. (For instance, in the North Bay, migrant farm workers are seriously undercounted. The cities know they’re there. The state and ACS sort of do. The Census doesn’t really, partly because of how they track race and ethnicity and partly because of how they guesstimate such things.)

    I’d love it if someone ever invented a good way to accurately count that had the rich data that the Census collects, but it’s incredibly hard to do that well even at the local level, let alone the national. And it’s actually getting harder—fifty years ago, people would fill out census forms or talk to census workers. Today, they largely won’t. How do you go about fixing that? If you were an undocumented worker (or homeless, or just generally distrusting of government), would you fill out a government form? How do you “sell” large-scale government data collection to the wary American? (I’m curious—and they actually hired an ad agency to do it for 2010!)

  27. V Smoothe Post author

    I understand the underlying concept of Deckin’s argument about consistency – what I don’t understand is how it would apply in this case, since there is no meaningful history of data sets for the ACS to use as comparison.

  28. Deckin

    Well, if we can all agree that the Census is data is the best hard data available (even if we acknowledge that they are missing people, what other data set is better?), then here’s the argument. We know from Census data that Oakland lost something on the order of 40K AA residents between 1990 and 2000 (I don’t have the energy to look up the exact number). Now I have no idea if they do this, but one could reverse test the current model by looking at what the currently observed trends look like, lets say, neighborhood by neighborhood, to what they saw between 1990 and 2000. I have no idea if they do that, but they could and that would be one way of testing the current model.

  29. Mike


    Not true. We’re a couple of Anglo’s and we have had it. We’ve been here five years. Always with the attitude/assumption that “It will get better”. Well we have realized, it wont. It is just getting worse. And the Oakland city government is a joke. I mean Nancy Nadel reelected? C’mon. Truth be told it could not gentrify fast enough. We’ll pay more but we’d rather live in The City.