How This Works

Dexter Xsamples 2010, With Frames

What, Why and Who

These modules are intended to help people with an abiding need to access the social, economic, demographic, geographic and other public data to be found in the MCDC/OSEDA public data archive at the University of Missouri. We shall be using the MCDC's Uexplore and Dexter web applications as the primary tools for accessing the data and producing the outputs shown in these modules. This application is intended for people who are interested in accessing these kinds of data and getting it in a form they can work with, as well as getting it for just the geographic areas or years or economic line codes of interest. It assumes that the user has visited the MCDC/OSEDA Public Data Archive home page and have read the important note (linked to at the top) on that page which begins with

This application is intended for use by people wanting to access data by querying a database.

This eliminates at least 90% of the world, we know. We do lots of stuff on our web site that is aimed at that 90% who don't want to know how we did it, they just want some numbers. One of the reasons for the existence of the archive and the extraction tools is that it makes it easier for us to do quick, easy and reliable extractions which can be easily shared with users who do not want to ever have to deal with a Dexter query form.

In addition to passing the "important note" test we also assume the following about users of these modules:

  1. They have some basic familiarity with the data archive. This could be based on genreal experience, or on a careful reading of the enter archive home page/directory which provides at least a feel for the scope of data available here, or it could even be based on following the link to the On-Line Tutorials and experienced the 59-slide powerpoint presentation on the MCDC Data Archive (they don't have to pay much attention to the history part though).
  2. They have some basic familiarity with using the Dexter software. This would come from using either or both of the links on the Dexter query form:
  3. They have some interest in the specific data being queried. Which means they should look through the list of Xsamples modules and just pick the ones that interest them.
.

In summary, these modules are really not so much about teaching the user the details of using Dexter (although you definitely will see some tricks or shortcuts that you may not have gathered from the other document sources), but are more about the data itself. What kind of information is hidden in the data archive, and look how relatively easy it is (most of the time) to extract it and get what I want. We want users to understand the difference between raw data, and information. The SAS data sets in the archive are closer to raw data than to information, but when combined with Uexplore/Dexter they can easily be transformed into information. That is what we want these modules to show people.

The Window Frames

Each Xsample module demonstrates how to combine the functionality of the Missouri Census Data Center's Uexplore and Dexter software modules (mostly the latter) with specific data sets in the MCDC public archive in order to create useful extracts. This is a frame-based application. When you invoke one of these modules your browser will display a window that is partitioned into 4 frames. The upper-left frame is quite small and always displays a link to this document.

The XSamples Frames Page Layout
Link to"How This Works"
Main Frame

Containing tutorial text and links to input/output modules, which will open in the Input/Output frames (on right).

Input Frame

Containing (initially) the filled-out dqf (Dexer Query Form)

Output Frame

Containing primary output, usually a report.

The rest of the left side is taken up by the Main Frame, which contains the text of the tutorial. A topic template is used for these modules to create a degree of consistency between these modules. Most of the hyperlinks in the Main Frame will cause those documents to be displayed in one of the fraems on the right side. (Although the user can override that by right-clicking on a link and specifying that they want to open it in a new window or tab.)

The two frames on the right of the screen start out each occupying half of the vertical space. The upper right frame is called the Input frame and initially contains a saved Dexter query form. The latter is what you get when you invoke Dexter and fill out the form (checking boxes and select list items, making text box entries, etc. -- all the things you do to define the query prior to using an Extract Data button to run the query). The user can browse this frame to see the details of how the query was done; they can also take advantage of this being a "live" module (unlike the static graphic images that we employed in earlier Xsample modules) where choices can be modified (or left alone as well) and an Extract Button used to run the query "live".

The right side bottom frame is called the Output frame and is initialized with an output module generated by the Xsample query. Links to all the output modules are provided in the Main Frame and those links can be used to display any or all output modules in the Output frame (just one at a time, of course).

Each frame occupies a specified portion of the window initially, but the user can drag the frame boundaries in order to modify them.

The Main Frame

The purpose of the MF is to describe the query, and provide access (hyperlinks) to various related modules such as the filled-out dexter query form, output files, metadata pages, related web sites, etc. It is structured, so that you can expect to find the same bolded item and paragraph headers, along with standard hyperlinks and other buttons that will help you navigate through the example. We won't go through each category, since they are pretty much self-explanatory. But we want to make sure you are aware of (and therefore make use of) several key items.

  1. Summary: this is where we describe the query in terms of what the query is doing. It will tend to be put in terms of what the user asked for. In many cases these will be real live queries that we actually did for real live users.

  2. Inputs: contains links to pages that were used in coding the query. It always starts with a link to the Dexter query form, which is already pre-loaded in the Input frame. Other modules that may be linked to here might be a metadata page for the dataset, maybe a screen shot of a Quick Look display, or some other relevant page that was viewed in order to help code the query. Links here always specify that the page be opened in the Input frame.

  3. Outputs: contains links to one or more output modules created by the query. Clicking on these links will usually cause them to be displayed in the Output frame. The exception to this rule is for a link to a csv file from a browser (such as most IE installs) that automatically invokes Excel to process the file, in which case Excel will be opened in a new window with the data from the csv file automatically imported. Remember, you can also right click on these links and specify that you want to open them in a new window or tab if you prefer. Some outputs are more interesting than others. The Dexter Summary log file, for example, will be of limited interest to most users. If there is a report file output it will already be displayed in the output frame so there is no point in clicking on that unless you previoulsy clicked on one of the other links here and now want to restore the original output choice.

  4. Data Set Accessed: indicates the archive dataset being referenced in the xsample. It will be displayed in the form of the filetype (i.e. the name of the /pub/data subdirectory where the dataset is stored) and the dataset name, separated by a period. If the dataset is stored in a subdirectory of a main filetype directory then we'll add a slash and the name of the subdirectory after the filetype. So, for example, if we referenced the dataset named ustabs083yr in the data directory /pub/data/acs2008/basetbls we would code this as acs2008/basetbls.ustabs083yr .

  5. What You Need to Know...: provides background information, usually focusing on the data source(s). What most often prevents users from accessing the data archive effectively is not having a good understanding of the data sets. Sometimes that lack of understanding has to do with the nitty gritty of how data sets are structured, variable naming conventions, etc. But even more common is a lack of a broader, more basic understanding of what the data collection is all about. In this section of the page we try to anticipate what the user may need to know, where we make no assumptions about prior knowledge.

    We have created a fair amount of metadata to describe many of the data sets, but there will be some that have little or no metadata associated with them. And even when there may be some, it may not be sufficient to really understand how and why these data are relevant to the problem at hand.

  6. Navigating the Archive: deals with how you would be likely to navigate your way through the data archive in order to arrive at a page where a Dexter Query Form is being displayed for the dataset from which we want to extract. Experienced Uexplore users will perhaps find this section repetitive and unnecessary, at least after a while. But better too much information than too little.

  7. What's In the Data Set: is a followup to the What You Need to Know.. section. The latter provided a broad overview of the data source. Here we zero in on a very specific file / data set that contains specific kinds of information for specific kinds of entities (usually, but not always, geographic areas) for a specific point(or points) in time, etc. In most cases it will be telling you things that can be gleaned from the detailed metadata page for the chosen data set.

  8. Coding the Query Filter:, Choosing Variables:, and Other Options: summarize what and why we did what we did in the various sections of the DQF. You may find after a while that you really don't need to read these; you can just go to the DQF module in the Input frame and see for yourself what we did. But if something doesn't make sense to you then you should be able to get an explanation from this section of the MainFrame page. Note that we shall generally not be spending time here dealing with the kind of things you can find in the Dexter Online documentation. Where we are doing something that is a bit obscure we may simply include a comment telling you to check out the Online doc for a detailed explanation.

  9. Questions, Excercises: is pretty self-explanatory. We are not planning to publish complete answers / solutions to these items. We may in some cases, however, provide links to solution DQF modules, and output modules created by these exercice solutions. While we may not always produce detailed solutions, we will respond to e-mail queries from users who are stumped, or who think they know the answer and would like to be sure. We would also encourage users to share resources amongst themselves. The best way to do that (for now -- we hope to change this soon) is to subscribe to the MCDC listserv (see instructions for doing so at the bottom of the About MCDC page, accessible from the navy blue navigation box on the MCDC home page.

Navigating Among the Frames

The use of frames to organize all the different pieces of the xsample is intended as a convenience feature. But you have to be able to navigate among the frames in order for it to work. The idea is to have the explanatory text being read in the main frame which refers to various input and output components that are displayed simultaneously just over to the right. The problem with this has to do with frame-size issues affedcting "browseability". The Dexter Query Form ("dqf"), for example, almost always needs to be viewed in a frame/window considerably larger than what is initally displayed in the Input frame quadrant. There are various ways you can view the dqf in a larger frame/tab/window:

  1. You can manually drag the borders of the Input frame to make it larger. This is OK except when you want to be able to quickly go back and forth between the other frames, since it would involve way too much time and effort dragging frame borders around. We have found that maximizing the window and then adjusting the column widths so that the left side is about a third of the screen and the right frames occupy the other 2/3 works fairly well.

  2. In some browsers (Firefox, for example, but not IE V.7) you can right click within a frame and choose "This frame" and then a series of options including "Show only this frame". When you make this choice it replaces the current 4-frame display with a simple display of the current frame page (dqf). To switch back to see the other frames just use the browser's Back button.

  3. There are links to the Input and Output modules in the Main Frame that specify that these open in the Input and Ouptut frames. But you can right click on these links and specify that you want to open them in a new window or new tab. This means you will now have to navigate among multiple windows or tabs rather than among frames. That tends to be a little trickier but the benefit is that you get to see each module at "full size".

  4. If you are one of the chosen few who are lucky enough to have dual monitors, the best solution may be to position your xsample window so that it spans bother monitors with the vertical boundary separating the Main Frame from the Input/Ouput frames occurring at the monitor split. This allows you to see both left and right side frames at full size. All you have to do now is perhaps adjust the frame boundary between the Input and Output frames depending on which one you are focusing on at the moment. Won't work so well if you want to go back and forth between Input and Output content a lot, but most of the time you are more likely to be switching between the left and right sides.

Where Can I Go For Help?

The two best basic sources for help on using the Dexter query tool are linked to from the top of the Dexter Query Form pages (and thus are typically just a click away since there is usually a DQF module visible in the upper-right Input frame). For basic help on what the Data Archive and the uexplore/dexter software are all about follow the link (from the Navy blue navigation boxes along the left side of most MCDC web pages) to the MCDC Data Archive home page. From there you will see numerous links to other documents that provide background information regarding the data archive and the software. There are also links to tutorials and on-line Help pages (including these Xsamples, which for many users may be the best source) that deal specifically with how to use the Dexter query form to define your queries.

For help with specific data collections, we suggest you peruse the archive home page to get an idea of what kinds of data are available and to navigate to the specific filetype data directories. From the pages displayed by Uexplore for these data collections look for files that appear to contain metadata. These would include files such as Readme.html files, SeeAlso.html files, and various files with extensions such as html, pdf and sometimes xls. Subdirectories with names such as Techdoc contain technical documentation. Subdirectroies named Tools appear in almost every filetype directory and these contain mostly programs and the associated log/listing files which document how the Missouri Census Data Center converted or otherwise processed the "raw data" for this particular kind of data. Tools libraries are most useful for professional programmers, espcially those who know SAS.

Be sure to look for and utilize Datasets.html files which are much easier to use when looking for a dataset than just the alphabetic list of dataabase files that you get from Uexplore. These dataset reports provide links to metadata files, the same kinds of links that you see at the top of the Dexter Query Form when a dataset has a metadata module for it. These metadata modules are perhaps the most important resource to use when trying to understand what a dataset is all about.

We do not have a forum specifically dedicated to accessing the archive (or regarding accessing this type of public data via any means) but we have pondered whether there would be sufficient interest to start one. If you would be interested in such a forum please send a note indicating so to the author (see link, below - the last thing on the page.)

And, of course, you can still get help from the human beings who are responsible for all of this. The best way to do that is by using the feedback buttons (the author's name at the end of the Questions/Comments provided at the bottom of most MCDC web pages, including this one).