How This Works (Dexter for Data Junkies, with frames)
What, Why and Who
These modules are intended to help people with an abiding need to access the social, economic, demographic, geographic and other public data to be found in the MCDC/OSEDA public data archive at the University of Missouri. We shall be using the MCDC's Uexplore and Dexter web applications as the primary tools for accessing the data and producing the outputs shown in these modules. This application is intended for people who are interested in accessing these kinds of data and getting it in a form they can work with, as well as getting it for just the geographic areas or years or economic line codes of interest. It assumes that the user has visited the
MCDC/OSEDA Public Data Archive home page and have read the important note (linked to at the top) on that page which begins with
This application is intended for use by people wanting to access data by querying a database.
This eliminates 90% of the world, we know. We do lots of stuff on our web site that is aimed at that 90% who don't want to know how we did it, they just want some numbers. One of the reasons for the existence of the archive and the extraction tools is that it makes it easier for us to do quick, easy and reliable extractions which can be easily shared with users who do not want to ever have to deal with a Dexter query form.
In addition to passing the "important note" test we also assume the following about users of these modules:
- They have some basic familiarity with the data archive. This could be based on genreal experience, or on a careful reading of the enter archive home page/directory which provides at least a feel for the scope of data available here, or it could even be based on following the link to the On-Line Tutorials and experienced the 59-slide powerpoint presentation on the MCDC Data Archive (they don't have to pay much attention to the history part though).
- They have some basic familiarity with using the Dexter software. This would come from using either or both of the links on the Dexter query form:
- the Dexter Quick Start Guide or
- the Online Documentation.
- They have some interest in the specific data being queried. Which means they should look through the list of ddj modules and just pick the ones that interest them.
In summary, these modules are really not so much about teaching the user the details of using Dexter (although you definitely will see some tricks or shortcuts that you may not have gathered from the other document sources), but are more about the data itself. What kind of information is hidden in the data archive, and look how relatively easy it is (most of the time) to extract it and get what I want. We want users to understand the difference between raw data, and information. The SAS data sets in the archive are closer to raw data than to information, but when combined with Uexplore/Dexter they can easily be transformed into information. That is what we want these modules to show people.
The Window Frames
Each ddj module demonstrates how to combine the functionality of the Missouri Census Data Center's Uexplore and Dexter
software modules (mostly the latter) with specific data sets in the MCDC public archive in order to create useful extracts.
This is a frame-based application. When you invoke a ddj module your browser will display a window that has been initially
partitioned into 4 frames. Each frame occupies a specified portion of the window initially, but the user can drag the frame
boundaries in order to modify them. There are two columns of equal width (when initially loaded, at least), each with a top and bottom frame. In the left column the top frame is quite small and has a yellow background. It's only purpose is to alert you to the existence of, and provide access to, this page. It is an important frame for new users who need to get started.
The remainder of the left column is where you'll find the main module that describes the data query. We'll be describing it in considerable more detail, below. Since it tends to be the focal point of the application we'll refer to it as the Main Frame, or sometimes just the MF.
The right column is divided into two frames of equal height. The top frame contains a scrollable graphic image of a Dexter query form. Each ddj module revolves around doing a custom data extract from the MCDC/OSEDA data archive using the Dexter extraction tool. What we have done to create these modules is to run these queries and capture the results. In the case of the DQF (Dexter Query Form) we just used a screen capture utility to save the contents of the scrollable window as a graphics image file, which gets displayed here. (Some browsers, such as Firefox, may display the entire page image in the available space, making it unreadable. Just click on the image to cause it to display at full size with a vertical scroll bar for viewing.)
The bottom frame on the right is used to display query output. It will usually contain a report in plain text, html or pdf format. Dexter queries can produce multiple outputs and we provide you with the ability to display all query outputs in this output frame, but of course just one at a time. We'll describe how this works below.
The Main Frame
The purpose of the MF is to describe the query, and provide access (hyperlinks) to various related modules such as output files, metadata pages, related web sites, etc. It is structured, so that you can expect to find the same bolded item and paragraph headers, along with standard hyperlinks and other buttons that will help you navigate through the example. We won't go through each category, since they are pretty much self-explanatory. But we want to make sure you are aware of (and therefore make use of) several key items.
- the first (top) frame contains a link to the page you are reading. We assume that you have seen this already and that it was sufficiently self-explanatory that you were able to use it to arrive here. It's the frame with the canary yellow background. You can always pull up its bottom boundary to make it essentially disappear.
- the Summary: section is where we describe the query in terms of what the query is doing. It will tend to be put in terms of what the user asked for. In many cases these will be real live queries that we actually did for real live users.
- the line titled Outputs: contains links to the various outputs created by the query. Clicking on these links will usually cause the linked-to files/pages to be displayed in the output frame (lower right side of the window). The exception to this rule is for a link to a csv file from a browser (such as most IE installs) that automatically invokes Excel to process the file, in which case Excel will be opened in a new window with the data from the csv file automatically imported. You can also right click on these links and specify that you want to open them in a new window or tab if that is your preference. Some outputs are more interesting than others. The "Saved query file" link is included for completeness but will not usually be of much interest. Similarly, the Dexter Summary log file will be of somewhat limited interest. It is the links in between to the "real" ouput that should be of most interest. If there is a report file output it will already be displayed in the output frame so there is no point in clicking on that unless you previoulsy clicked on one of the other links here and now want to restore the original output choice.
- The What You Need to Know... section provides background information, usually focusing on the data source(s). What most often prevents users from accessing the data archive effectively is not really having a good understanding of the data sets. Sometimes that lack of understanding has to do with the nitty gritty of how data sets are structured, variable naming conventions, etc. But even more common is a lack of a broader, more basic understanding of what the data collection is all about. In this section of the page we try to anticipate what the user may need to know, where we make no assumptions about prior knowledge.
We have created a fair amount of metadata to describe many of the data sets, but there will be some that have little or no metadata associated with them. And even when there may be some, it may not be sufficient to really understand how and why these data are relevant to the problem at hand.
- We are not sure if we really want to have a Navigating the Archive section in all our modules. Experienced Uexplore users will find this section repetitive and unnecessary after a while. But better too much information than too little. Note that the last bullet item in this section will always be the same and is by far the most important item in this section. It is very strongly recommended that you follow the advice given there which is to open a separate browser window where you will have a "live" Dexter query form displayed with the relevant data set selected. This is where you can try to reproduce the query by emulating what you see filled in in the DQF image displayed in the right-top frame. More importantly, this is where you can experiment with doing variations of the query that make it more interesting or useful to you. For example, if the query being described selects data for the state of Missouri, but you would rather see data for California, then you might want to substitute the appropriate code for your state in the Query Filter section (and perhaps modify a title or subtitle that may reference the name of the state).
- The What's In the Data Set section is kind of a micro version of the kind of information seen in the What You Need to Know.. section. The latter provided a broad overview of the data source. Here we zero in on a very specific file / data set that contains specific kinds of information for specific kinds of entities (usually, but not always, geographic areas) for (a) specific point(s) in time, etc. In most cases it will be telling you things that can be gleaned from the detailed metadata page for the chosen data set.
- The sections Coding the Query Filter, Choosing Variables, and Other Options summarize what and why we did what we did in the various sections of the DQF. You may find after a while that you really don't need to read these; you can just go to the DQF image and see for yourself what we did. But if something doesn't make sense to you then you should be able to get an explanation from this section of the MF. Note that we shall generally not be spending time here dealing with the kind of things you can find in the Dexter Online documentation. Where we are doing something that is a bit obscure we may simply include a comment telling you to check out the Online doc for a certain section to see an explanation of what or why we coded something in a certain way.
- The Questions, Excercises section is pretty self-explanatory. We are not planning to publish complete answers / solutions to these items, althoug in some cases we may provide links to output modules that were created when we took our own suggestion to do the exercise. But we definitely will respond to queries from users who are stumped, or who think they know the answer and would like to be sure. We would also encourage users to share resources amongst themselves. The best way to do that is to subscribe to the MCDC listserv (see instructions for doing so at the bottom of the About MCDC page, accessible from the navy blue navigation box on the MCDC home page.
Where Can I Go For Help?
For basic help on what the Data Archive and the uexplore/dexter software are all about you should follow the link (from the Navy blue navigation boxes along the left side of most MCDC web pages) to the MCDC Data Archive home page. From there you will see numerous links to other documents that provide background information regarding the data archive and the software. There are also links to various tutorials/on-line Help pages that deal specifically with how to use the Dexter query form to define your queries. And, of course, you can still get help from the human beings who are responsible for this mess. The best way to do this is by using the feedback buttons provided at the bottom of most MCDC web pages.