|
2007
February || March ||
April || May ||
June || July ||
August|| September ||
November
November 2007
American Community Survey Data on MCDC Web Site
Additions to the Data Archive
Back in September we posted announcements here regarding the availability of new American Community Survey data from the Census Bureau based on the calendar year 2006 surveys. Most users with interest in these data will have been accessing it via the Census Bureau's American FactFinder web site. While this is still the easiest way for most users to access the data, for some users (such as ourselves) there is some frustration due to the limits of what data can be readily extracted using the AFF tools. This is especially true with regard to the data contained in the Data Profiles reports. These data can only be accessed one geographic area at a time via FactFinder. The MCDC has downloaded the raw data contained in both the Data Profiles data sets and the complete set of detailed tables, and has made these data available in our public data archive. Users familiar with the Uexplore/Dexter web access tools will now be able to access the 2006 ACS data using those applications.
ACS Profiles Web Application
We have also created a web application that displays our variation of the Bureau's basic profile data in a form that is similar in many ways to the profiles available via American FactFinder. Our profiles differ from the Bureau's in several ways:
- Our profiles contain both counts and percentages. You not only get the percent of persons estimated to be poor, but also the number of these persons.
- You can display data for up to 4 geographic entities at one time to assist in comparitive analyses.
- Using the dexter data extraction utility it is possible to extract the underlying data for further analysis and/or display. You can, for example, easily pull key economic indicator variables for every state, county, PUMA, city, etc. for which data are published and get it in csv, pdf, dbf or SAS dataset format per your specifications.
- We have emulated the Bureau's 4 sub profiles: basic demographics, social indicators, economic indicators and housing indicators. Unlike the AFF access, we allow you to display any or all of these four profiles at one time.
- We handle display of the sampling error estimates (margins of error) differently. We use font weights (light, normal and bold) to indicate general reliability of estimates but we do not directly display MOE values as part of the display. Instead the user can view the MOE (as a percentage of the estimate) and the confidence interval for the estimate by moving the cursor over the estimate on the display and leaving it there for 1 second.
We are still in the process of building front end menus to make it easy for users to access these profiles for specific geographies. But we have a front end that in many ways resembles the way you select your geography under AFF. To access the profiles using this geographic interface use this URL:
http://mcdc.missouri.edu/cgi-bin/broker?_PROGRAM=websas.acsskin.sas&_SERVICE=sasapp
You can also display profiles for any state, county or place (of 65,000 or more population, of course) using our applinks utility (see the "Applinks Master Menu" link in the navy blue navigation box on the MCDC home page).
Summary Files
The Census Bureau has not yet quite finalized their methodology for distributing the full collection of detailed (aka "base") tables that are created as part of the ACS data products. Last year (the 2005 vintage data) we got to download and convert a series of base table files where each table was contained in a single file and had data for the entire country. That option was not repeated this year. Late last year (well after we had already completed the tedious task of converting all the detailed base tables) the Bureau announced the availability of a new "summary file" data product, that was being tested. This new option would allow users to download all these data in a format that was similar to how they had distributed the Summary File ("sf") products from the 2000 census. This involves breaking the data down into over 140 comma-delimited "segment" files, with each segment containing a specific set of base tables. These are released in conjunction with a very large (200+ pages) pdf file of technical documentation. The whole scheme of how the data were organized and documented was designated as experimental, with the "production" version not due until December of this year. But we did not want to wait that long so we finally decided to see if we could navigate our way through the maze and access the summary files. (The big breakthrough was when we discovered that the Bureau had managed to embed an excel file with the detailed data dictionary inside the pdf file.) It took several weeks and several exchanged e-mails with Bureau staff, but we finally managed to turn the collection of .zip archives embedded within .zip archives of 140+ csv files into a series of SAS datasets that are consistent with the way we have stored decennial census SF files over the last 25 years. We have thus far only converted the files containing the estimates; there are parallel files contains values for MOE's (margins of error) and standard errors, but we have not yet done anything with those, other than download them (they come bundled within the .zip files so you have no choice but to download them if you want the estimates.)
Why do we go to all this trouble to get these summary files when such a small number of end users even understand what they are, much less plan to learn how to access them via Uexplore/Dexter? The answer is that these datasets allow us to do other things that require bulk access to these data, things that happen in the background that users do not have to understand or know much about. For example, the national state data center organization has indicated an interest in creating custom profiles based on ACS data similar to what they (we) did for SF1 and SF3 after the 2000 census. One of the keys to accomplishing this (using SAS as the language to generate the profiles) is to first get easy access to all the detailed data in a SAS dataset. The MCDC is also involved with the support of a number of custom online web sites where various collections of key indicators (many, but usually not all, census-based) can be accessed. Creating such indicators (of such things as the well-being of children or senior citizens, for example) requires easy and reliable access to these detailed data.
The detailed collection of base tables from the 2006 ACS can be accessed in the summary subdirectory of the acs2006 filetype directory of the MCDC public data archive. If that jargon does not totally frighten you, then you can access the data by pointing uexplore at this summary subdirectory. The URL for doing this is:
http://mcdc.missouri.edu/cgi-bin/uexplore?/pub/data/acs2006/summary.
(You can also get here by following the links on the archive home page, choosing the American Community Survey main category, then the acs2006 filetype directory, and then the summary directory from within that index page.) Before you go down to the summary subdirectory, a good resource to know about before you attempt such a data extraction is our annotated human-readable data dictionary page, which is stored in the acs2006 data directory. The name of this file is BasevarLabels.sas .
We are working on a new module in our xsamples library of sample Dexter extractions. This new module walks you through the complete process of locating a table with relevant data (using American FactFinder), extracting those tables via Uexplore/Dexter, and then post-processing the tabular data to produce a final completed report. (We have completed all but the postprocessing sections as of Nov. 27.) Access this module at
http://mcdc.missouri.edu/pub/xsamples/acsbasetabs1/.
September 2007
American Community Survey Data, 2006
(September 27 data release):
New data released today from the U.S. Census Bureau provide the first
social and economic characteristic profiles of the people living in
group quarters such as adult correctional facilities, college dorms
and nursing homes in nearly three decades.
This release marks the first in-depth look at the characteristics
of the nonhousehold population since the 1980 Census, said Census Bureau
Director Louis Kincannon. These are important data to understand
as decision makers grapple with policies that impact the people who
live in these facilities.
The latest data from the American Community Survey also include
profiles of more than 100 race and ethnic group iterations such as blacks,
Chinese and Mexicans. In addition, profiles for 72 ancestry
groups, from Afghani to Welsh, are also available.
Selected national highlights for todays data release include the following:
(more...)
(September 12 data release):
The Census Bureau has completed the first two phases of the 3-phase release of
data based on the American Community Survey collected in calendar year 2006.
This is the first year that ACS data cover the entire U.S. population (the 2005
data did not include persons living in group quarters). There are several ways
you can access these data:
The American FactFinder site
at the Census Bureau provides the best general access to the data as well as
related metadata. (Links are provided to access the ACS data in the 2nd
paragraph of the "Getting Detailed Data" section down the middle of the page.)
For those interested in data for the state of Missouri in Excel format,
the MCDC has downloaded a collection of xls files with complete results (detailed
base tables, the new demographic profile tables, and subject tables) for each
geographic area in a separate xls file. These can be accessed at
http://mcdc.missouri.edu/data/acs2006/profiles/Missouri/
Users wanting to access these data via our Uexplore/Dexter system will have to
wait a little longer as we work on accessing and converting the data for the
entire country. We hope to be able to post a complete collection of the profile
data for all geographies in the country within the next few days. You can monitor
what's availabe by accessing the
acs2006
data directory (and then choosing the Profiles subdirectory.) Note
that you can also access the above-referenced Missouri xls files collection
using uexplore and the profiles subdirectory.
The American Community Survey Alert, Number 53, issued by the Bureau today (9-12-07) includes
the following information regarding the September release and the upcoming final release due
later this month:
The Census Bureaus Web site now contains guidance on comparing 2006 ACS
data to 2005 ACS data, as well as comparing 2006 ACS data to Census 2000
data. This guidance may be found at:
http://www.census.gov/acs/www/UseData/compACS.htm.
The final release of 2006 ACS data will occur on Sept. 27, 2007 and
consist of selected population profiles/tables for about 200 selected
population groups by race, Hispanic origin, and ancestry. In addition,
this release will include group quarters profiles, the first such release
since the 1990 Census. Also released at that time will be workplace
geography tables.
August 2007
County Estimates With Demographic Detail
The Census Bureau completed the release of major population estimates products for this year with the county
level numbers with detail by age, race, sex and hispanic origin.
See the
Census Bureau presss release
related to these estimates.
The MCDC has downloaded these data into our public archive where they can be accessed via
Uexplore/Dexter in the popests data directory. We have also generated a series of summary reports at the county
and metropolitan/micropolitan area levels for all states. To access these use the Population Estimates Reports by State page.
(where the "by State" means you get to select your state and then have access to reports for that state which contain
state, county and metro area level data.)
For those interested in a brief summary of the data for Missouri only, see the
moagesexracehisp_2006.xls spreadsheet file
created by Evelyn Cleveland.
New Year of IRS County-to-County Migration Data
We have obtained and processed the IRS migration data for tax years 2005-2006. These data are used to generate
our county-level IRS migration profile reports. The easiest way to generate a report is to use the
Main Menu page for the application (or, if you want a Missouri county profile, go straight to the
Missouri menu page.)
You can also access these profiles using the applinks utility application if all you need is data for the most recent year(s).
See our original posting regarding these datasets and profiles on our Whats New page for
April of this year.
Population Estimates by Age Web Application Updated
Population estimates by age (single year), "bridged" race, sex and
hispanic origin (custom tab from the National Center for Health Statistics) for 2006 have been downloaded and
added to our 20XX datasets in the popests/nchsbri subdirectory of the MCDC data archive. These are the data used in
our Population Estimates by Age web application (which has been updated so that 2006 now appears and is the default on the Years select list).
These data are from the same basic source as the casrh estimates, but they differ in the greated age detail and
the collapsing of the 11 race categories on casrh to the 4 "bridged" categories which NCHS requires for their
time-series research.
July 2007
U.S. Census Bureau Releases Schedule for 2006 American Community Survey Data -- The Census Bureau will release data from the 2006 American Community Survey (ACS) in August and September according to the timetable below. This year’s data release marks the first time that group quarters (such as prisons, college dorms, military barracks and nursing homes) are included in the data products. Closer to the release date, the Census Bureau will provide guidance on making comparisons between annual ACS data products, as well as comparisons between ACS and Census 2000 data. Data will again be available for the nation, 50 states and the District of Columbia, Puerto Rico, every congressional district and all counties, places and metropolitan areas with populations of 65,000 or more.
Data Release Schedule:
Aug. 28, 2007: ACS income, earnings and poverty data, in conjunction with the Census Bureau’s annual release of income, poverty and health insurance data from the Annual Social and Economic Supplement (ASEC) to the Current Population Survey (CPS).
Sept. 12, 2007: ACS social, economic and housing characteristics, demographic and housing estimates, and Public Use Microdata Samples (PUMS) data. The data will consist of:
- Educational attainment
- Industry
- Occupation
- Class of worker
- Journey to work
- Employment status
- Work status
- Veteran status
- Housing
- Households and families
- Marital status
- Migration
- Family
Sept. 27, 2007: ACS group quarters and selected population profiles/tables for about 200 selected population groups by race, Hispanic origin and ancestry. This marks the first release of group quarters profiles since 1990. Also released at that time will be workplace geography tables.
The ACS is a key component of the Census Bureau’s 2010 Decennial Census Program, which also consists of early planning and modernization of geographic operations and a short-form only for the 2010 Census.
June 2007
Murdock Nominated to be New Head of Census Bureau -- The president has nominated Dr. Steve Murdock to be the new director of the Census Bureau.
Murdock, who is chairman of the Demography and Organization Studies program at the University of Texas at San Antonio,
is also the long-time Director of the Texas State Data Center. If approved by the Senate Murdock will replace
Louis Kincannon, who has held the post since 2002.
Site Search Engine Added -- Users of this web site will notice a new feature; we now have a Google search engine that will search this site only. The link is located at the bottom of the navigation box along the left hand side of most MCDC web pages.
County Business Patterns Data -- New data for 2005 has been added to the County Business Patterns collection in the MCDC's data archive. Users familiar with the Uexplore/Dexter software can access the data in the
cbp data directory. The collection has data for the entire country at the nation, state and county level -- 3 data sets per year, going back to 1999.
New "X Samples" Modules Document Use of Dexter -- A new collection of detailed examples of using the Dexter extraction software has been created. View the
directory page for the collection in the new xsamples directory. There is also a link to this page at the top of the Uexplore/Dexter home page. The MCDC will be
monitoring usage of these modules to see if it is worthwhile to create more.
Population Trends 2000-2006 for Counties in the Kansas City Metro Area --
Screen image of the Kansas
City county trend report
This report was generated using the latest county population estimates with components of change data
released by the Census Bureau earlier this spring. Those data were downloaded from the Bureau's web site and converted to an enhanced SAS data set (using basically the same conversion program that we used last year and the year before). It was then catalogued and documented as part of the MCDC's public data archive. This report
was created by accessing the data set using Uexplore (to navigate the archive and locate the data set) and Dexter (the data extraction module that actually generated the report). The details of the data extraction (report generation)
are described on a special web page that can be accessed at
http://mcdc.missouri.edu/pub/xsamples/popcom1/. This is just one instance of a new collection of xsample modules stored in a new xsamples directory.
The user who takes the time to follow along with the xsample module and see how the data for the report were chosen and formatted should be able to rather easily modify the query to generate a comparable report for the St. Louis MSA, or indeed for any metropolitan area (or state) in the country.
Updated Tutorial Modules for Uexplore/Dexter
The online PowerPoint modules related to the MCDC data archive and the dynamic web applications used to access it have been reorganized, revised, and relocated into a special directory of their own. The modules are ppt files but the index page is html. Access at
http://mcdc.missouri.edu/tutorials/uexploreDexter/.
May 2007
MCDC Spring Training Workshops. The MCDC will be presenting spring training workshops in Columbia, Springfield and Kansas City in May and June. There will be two workshop presentations: one dealing with the MCDC and American FactFinder web sites with special emphasis on MCDC web applications such as MABLE/Geocorr and Uexplore/Dexter, and the other an introduction to desktop GIS. John Blodgett of OSEDA and Jim Harlan of the Geographic Resources Center will be the presenters. The workshops will be held in mid to late May and mid June. For more detailed information and to sign up for one of the workshops see our training page.
IRS Migration Data -- The IRS migration files permit tracking movement at the county-to-county level for consecutive tax years. You can,
for example, see how many households ("Returns") were filed in Los Angeles county, CA in 2004 and were
then found filing their 2005 returns in Clark county, NV (Las Vegas) -- 6,129 . You can also see how many went the other way: there were 1,885 "moves" going from Clark in 2004 to Los Angeles in 2005. An Exemptions data item shows the number of exemptions claimed on these tax returns and serves as a proxy for the number of persons involved. Information (medians and means) regarding the adjusted gross income reported by these movers is reported as well. We have processed these files for the entire U.S. for the most recent six years available, the most recent data being based on returns for tax years 2004 and 2005. A "Migration Profile" report has been developed that brings together in one place all the data available for one county for one pair of tax years. To see one of these go to the main menu page (at
mcdc2.missouri.edu/cgi-bin/uexplore?/pub/data/irsmig/Migration_Profiles) and click on of the menu page for a state. Then choose a county within that state and generate the county level report.
The complete collection of data sets, including original Excel spreadsheet files for Missouri and comma-delimited ascii files for the entire country can be accessed via Uexplore at http://mcdc2.missouri.edu/cgi-bin/uexplore?/pub/data/irsmig. (From the Uexplore/Dexter home page it is included in the "Other" major category.) Note that you can also access the above-mentioned main menu for the profiles from here.
MCDC User Survey -- The first ever Missouri Census Data Center online user survey is now yours for the taking. Please help us to evaluate current
aspects of our web site, data holdings and training, and in making decisions regarding future directions.
It is a fairly long survey (47 questions), but you do not have to respond to all of them. Take the survey online at
http://mcdc2.missouri.edu/websas/mcdc_survey.shtml.
April 2007
New Version of MABLE/Geocorr2k
Version 1.3 of MABLE/Geocorr2k is now available for testing. This venerable web application is a geographic utility for viewing the relationships between many different geographic layers. The new version includes a number of new geographic layers and allows using a new weight variable reflecting 2006 county-level population estimates. Performance has been enhanced so that requests that require accessing all 50 states +DC should run in less than 4 minutes without the timing out problems encountered with earlier versions. This is still a beta test version and the links to geocorr2k on our Quick Links navigation box and from our Geography page will still point to the old version for another month or so. There is a link to a What's New in Version 1.3 page from the new application page that provides more detail regarding what has changed in this version.
IRS Migration Data
The IRS migration files permit tracking movement at the county-to-county level for consecutive tax years. You can,
for example, see how many households ("Returns") were filed in Los Angeles county, CA in 2004 and were
then found filing their 2005 returns in Clark county, NV (Las Vegas) -- 6,129 . You can also see how many went the other way: there were 1,885 "moves" going from Clark in 2004 to Los Angeles in 2005. An Exemptions data item shows the number of exemptions claimed on these tax returns and serves as a proxy for the number of persons involved. Information (medians and means) regarding the adjusted gross income reported by these movers is reported as well. We have processed these files for the entire U.S. for the most recent six years available, the most recent data being based on returns for tax years 2004 and 2005. A "Migration Profile" report has been developed that brings together in one place all the data available for one county for one pair of tax years. To see one of these go to the main menu page (at
mcdc2.missouri.edu/cgi-bin/uexplore?/pub/data/irsmig/Migration_Profiles) and click on of the menu page for a state. Then choose a county within that state and generate the county level report.
The complete collection of data sets, including original Excel spreadsheet files for Missouri and comma-delimited ascii files for the entire country can be accessed via Uexplore at http://mcdc2.missouri.edu/cgi-bin/uexplore?/pub/data/irsmig. (From the Uexplore/Dexter home page it is included in the "Other" major category.) Note that you can also access the above-mentioned main menu for the profiles from here.
MCDC Spring Training Workshops
The MCDC will be presenting spring training workshops around the state in May and June. There will be two workshop presentations - one dealing with the MCDC and American FactFinder web sites with special emphasis on MCDC web applications such as MABLE/Geocorr and Uexplore/Dexter, and the other an introduction to desktop GIS. John Blodgett of OSEDA and Jim Harlan of the Geographic Resources Center will be the presenters. The workshops will be held in Columbia, Springfield and Kansas City in mid to late May and mid June. Watch for more detailed information on our training page. We are still working on finalizing local arrangements and enrollment procedures.
MCDC Listserv
The best way to be kept informed regarding what's new at the MCDC - short of visiting the home page every day - is to subscribe to the Missouri Census Data Center listserve. This is where we post announcements concerning new data, changes (presumably enhancements) to our web site, important breaking news items related to census data, announcements regarding workshops, online training, conferences, upcoming data releases, etc. The listserv can also serve as a forum where you can provide us with feedback or ask questions. It can also be used to share your experiences and/or ask questions of other users with similar interests. To subscribe to the listserve send an email message to
listserv@listserv.sos.mo.gov .
In the body of the message type
subscribe mcdc your_firstname your_lastname.
Shortly after sending this message you should receive a confirmation that you are subscribed to the MCDC listserve along with instructions on how to post your messages to the list and/or change your subscriber status.
Note that you do not have to be affiliated with the Missouri Census Data Center or even be a resident of Missouri in order to subscribe to the list.
March 2007
New County Estimates for 2006
The Census Bureau has released new population estimates at the state and county level for the entire U.S. The estimates include components of the population change for each year from 2000 to 2006. You can access these data from the Census Bureau's Population Estimates page. For Missouri-only data you can see the new numbers in the Counties sheet within the Curmoests.xls Excel spreadsheet. (The other sheets in this workbook are still based on the 2005 vintage estimates released last year.)
Uexplore/Dexter users can access the new datasets in the popests data directory: datasets uscom06 (data for the entire country) and mocom06 (Missouri subset of uscom06).
We have updated the relevant modules in our collection of reports based on the latest estimates -- see item 3 on our Population Estimates page.
New Web Application Displays Annual Population Change Data
The MCDC has developed a new web application, nickname poptrend1, that displays a table and a bar chart depicting the change in total population for a specified state or county for each year since 2000. The best way to see a whole bunch of these reports is to go to our Population Estimates page and follow the link under item 5 :"Generate a brief report with bar chart" (or, if you're in a big hurry, you can just follow this link). This will allow you menu-based access to the application for any county in the country. The new application has also been added to our Applinks application (see the
Applinks Master Menu (to see data for any state or county in U.S.) or the Missouri County Data Map (to access applinks menus for Missouri counties only). In either case look for the poptrend1 link.
If you like these reports enough that you would like to see them for all counties in your state then you can modify the URL (displayed in your browser's Address box when you are viewing one of the reports for a county), replacing the 3-digit county code (following &co= at the far right) with the word all. For Missouri the edited URL would be
http://mcdc2.missouri.edu/cgi-bin/broker?_PROGRAM=websas.poptrend1.sas&_SERVICE=appdev&st=29&co=all.
February 2007
New Datasets and Filetypes in Data Archive
We have created several new filetypes and added new data to old ones. We have also done a little
reorganization of the archive directory structure, eliminating the Agriculture category in favor of a new
catch-all Other category.
For details see the Recent Updates to the Archive section of the Uexplore home page.
More on Missouri PUMAs
In an effort to get users more familiar with the Public Use MicroSample Area geographic entities, we have created a dataset and pdf-format report summarizing Missouri's 41 PUMA areas. The report organizes the PUMA's within the set of 10 much larger "Super PUMAs" and shows for each PUMA the 5-digit code, an MCDC-assigned mnemonic "Nickname" for the area and a list of the counties that are contained within the PUMA area. If a PUMA does not contain the entire county then a percentage figure is shown indicating what portion of the county lies within the PUMA. Finally, the report inicates a Page number value that can be used to go straight to the page in the 12-page pdf document created by the Census Bureau showing where the PUMA's are located and, specifically, what counties intersect with them.
Other "What's New" by Year:
2006 |
2005 |
2004 |
2003 |
2002
|
