Posts by Tags

QGis

Making the case for hover interactions in maps

19 minute read

In keeping with my recent spate of mapping nerdiness, I decided to take an interactive map I produced last month displaying statewide annual population changes a step further by adding mouseover/hover capabilities. Here's the hover-y, nicely-colored chloropleth map I came up with. But before I get into the nitty-gritty of how I created the map –– which I'll explain step-by-step in a later post –– let me exercise a bit of self-indulgence by defending my growing belief in the need for hover capabilities when visualizing geographic data.

Not too long ago, I was an avid believer in the no-frills, less-interactivity-is-more approach to mapping geographic data, espoused by the brilliant Brian Boyer (@brianboyer), News Applications Editor for NPR and a former member of the News Apps Team at The Chicago Tribune. Boyer's argument for the need to keep maps simple –– like they used to be back in the days of ink and paper –– certainly has its merits. After all, the process of bringing a physical map closer to one's eyes to get a better view is a natural, timeless user interaction, and maps like this one, which Boyer produced during his time at The Tribune, are far more intuitive in communicating information upon first glance than many of the infoWindow-laden Google maps being produced by news organizations these days, many of them simply for the sake of being called 'interactive' (for those of you fortunate enough not to be mapping nerds, infoWindow is just Google-speak for the clickable popup boxes you see in Google maps).

But Boyer's minimalist mapping aesthetic only really works when you have one or two pieces of textual data you want to display for each geographic area. What if you have multiple pieces of information you want to display for each polygon, such as in this snazzy map from The Texas Tribune? Or, less likely but equally problematic, what if you need to bind non-textual data to your geographic polygons, such as images or Google Charts? In cases such as these, you're going to need to provide some sort of interaction that allows the user to expand and collapse the data for each area individually, or you'll just end up with a chronic case of visual overload.

Not to mention, on a more abstract level, studies have repeatedly shown that users tend to spend more time on applications that provide direct feedback based upon their actions, even if that feedback sometimes makes their ability to consume information at first glance less efficient (see Donald Norman's 2005 book Emotional Design: Why We Love (Or Hate) Everyday Things, in which Norman asserts that the feeling of emotional satisfaction and empowerment users receive from triggering an action not only puts them in a clearer state of mind, but also makes them more engaged in the information at hand). So, if we're trying to communicate geographic data to users as effectively as possible, it only makes sense that we'd want to have a certain degree of user interaction –– both for the sake of preventing visual overload and for making users feel more engaged. Such is the logic behind clickable infoWindows.

Still, clickable popups leave us with another problem: Users have to make the conscious and deliberate effort to click a polygon to see the data for that geographic area. Requiring clicks may sound like a trivial task to the designer or journalist-programmer, but for the short-attention span user, it can be an awful lot to ask for. To be fair, however, click-triggered popups may not be much of a problem for maps with only a few dozen polygons. But for maps with hundreds of small polygons –– say, census tracts or zip codes –– it can be very tedious to click the right polygon without first having to zoom into the map so far that you lose sight of the broader context.

That leads me back to a conversation I had a couple of months ago with a friend of mine from Columbia's J-School, Michael Keller (@mhkeller), who's now working as the Senior Data Reporter at The Daily Beast. Michael insisted to me after a Hacks/Hacker event that providing hover interaction for maps is almost always a good thing, because hovers require less work on the part of the user. I'll admit that I was dubious at the time, thinking of hovers as often unwanted, accidental triggers that can be distracting to the map and data at hand. But lately I've come around to his way of seeing things. If implemented correctly (i.e. no flashy interactions that cover up other parts of the map), hovers are almost always a good idea. For example, this recent map of New York Stop-and-Frisk data that Keller produced for The New York World using CartoDB and Leaflet is so detailed that it couldn't possibly have worked without infoWindows, and would have been unwieldy if it relied on click-triggered interaction. By including floating mouseover capabilities, the map allows the user to scan quickly through the chloropleth map to see individual Stop-and-Frisk data from each block, without having to attempt to click through minute geographic areas.*

I'm certainly not advocating interaction for interaction's sake (although such a case could be made, given the dynamics of the Web). But I am saying that hovers give more immediate visual feedback than click-triggered events, especially in maps. Hovers help draw users into the data without requiring them to seek it out consciously–– almost like a catchy lede would in a print narrative. So for the time-being, I'm pro-hover.

*Keller later messaged me letting me know that some examples which better illustrate the power of hovers include this map and this map, both of which use hover functionality to help highlight the effects of proposed redistricting efforts in the New York State Senate. What you'll also notice about Keller's maps is that they include hover states, which I also think is a necessity, especially for maps that include lots of small polygons.

Typography

What makes BostonGlobe.com “the world’s best designed website”

16 minute read

With the Pulitzer Price announcements coming up later this afternoon, you'd think I'd be writing about whose up for the "Best Deadline Reporting" or "Best Public Service Journalism" prizes. But instead I want to talk about a different media award doled out during the past week: BostonGlobe.com's designation as the "world's best designed website" by the Society for News Design. Put simply, I can't say I disagree.

Yet before I divulge in my effusive praise of the folks in Boston, let me point out that I'm still somewhat skeptical of the business-logic behind The Globe's decision to launch its paywall-only site last fall alongside its primary news portal, Boston.com. From a revenue standpoint, I can't see the full-paywall site, BostonGlobe.com, bringing in nearly enough subscription income to compensate for the traffic and ad dollars it will, and to some degree already has, leeched from Boston.com (at last count, in February, BostonGlobe.com had only recruited about 16,000 paid digital subscribers, many of whom had taken advantage of the site's introductory offer of 99 cents for the first four weeks but may not stick it out past the trial period). It might've been wiser for the paper to start out with a metered paywall to warm users to the idea of paying for content before erecting a full-blown ten-foot-tall paywall around its most valuable content under a new, separate domain name. But then again, who really knows – maybe it's a step in the right direction longterm? I'm not the one having to make the tough calls, so I'm in no position to judge.

At any rate, the segregation of the two sites has given the paper's parent company the freedom to make BostonGlobe.com a rich, immersive and interactive user experience that few other news organizations can match. Why, you ask? Because of a little trend gaining steam in the development world called "responsive design." See, BostonGlobe.com isn't only a standout site because of its sophisticated use of white space, its wholehearted embrace of web fonts and its visual-first approach to story art, but also because of its cross-platform capabilities. No matter what device you view BostonGlobe.com on – desktop, tablet or mobile – the site retains its same slick look and feel. The site's adaptive technology allows it to detect the screen size of the user, then adjust its layout to fit the exact resolution at hand. On desktops, that means content stretches to fill the entire browser window, and the grid restructures itself appropriately when you decrease or increase the size of the window. This eliminates the need for time-honored design standards like the 960 grid-system, which is based upon the Desktop-centric idea that all users have at least a 960px resolution. Now, the grid can be as big, or as small, as the user wants it to be.

The site's successful display of a dynamic range of real-time content on any-sized device also essentially eliminates the need for the "app-based" environment that's been the staple of the iOS5 and Android operating systems in recent years. To test it out, I gave the site a whirl on my iPad, and was pleased to see the front page neatly rearrange itself to fit the new orientation, just as any of the best platform-specific news apps out there do, including The New Yorker and Wired apps for iPad. But just because BostonGlobe.com doesn't need a native app to display its content beautifully on tablet and mobile devices doesn't mean that it won't be missing out on the growing app-centric marketplace for publishers. As Apple and Google continue to centralize digital consumption patterns into the app-based model, BostonGlobe.com may be a little too ambitious in thinking it doesn't need to play nice with the big-boy tech companies to reach its audience. On the other hand, however, it could prove to be a brilliant move for the paper, setting an example for other publishers and app-based companies in general who want to break free from the often constricting, not to mention pricey, cost of participating in the app marketplace.

For its design alone, BostonGlobe.com deserves any number of awards. Its bold, minimalist interface allows content to shine above all else, free from clutter and distraction. And with high-res horizontal photos, block-quotes, inline video and stylized headlines that grab your attention without hurting your eyes, that content stands out even more. What's more, the site has the functionality to back up its aesthetics. Its "Save" feature allows users to bookmark articles for reading offline on any device, which, even for sites without responsive design, is a brilliant feature for a news site to implement. Moreover, its "Story Flow" panel at the bottom of each article allows users to click seamlessly through to stories on similar subjects, just as readers would in an old-fashioned newspaper with physical sections. Which leaves me with one nagging question: Is BostonGlobe.com too imitative of physical newspapers to attain success in today's short attention-span digital audience? Is it too skewmorphic to succeed in an SEO-driven, hyperlinked news economy?

I guess we'll have to wait and see.

blogging

Why news organizations should stop differeniating blogs from articles

13 minute read

Andy Boyle (@andymboyle) of The Boston Globe made an┬áimpassioned┬áplea to news organizations earlier this week that they stop differentiating between blogs and articles because they’re both equally forms of content. Someone’s been needing to put this into writing for a while now, and I’m glad Andy said it so eloquently.

bostonglobe.com

What makes BostonGlobe.com “the world’s best designed website”

16 minute read

With the Pulitzer Price announcements coming up later this afternoon, you'd think I'd be writing about whose up for the "Best Deadline Reporting" or "Best Public Service Journalism" prizes. But instead I want to talk about a different media award doled out during the past week: BostonGlobe.com's designation as the "world's best designed website" by the Society for News Design. Put simply, I can't say I disagree.

Yet before I divulge in my effusive praise of the folks in Boston, let me point out that I'm still somewhat skeptical of the business-logic behind The Globe's decision to launch its paywall-only site last fall alongside its primary news portal, Boston.com. From a revenue standpoint, I can't see the full-paywall site, BostonGlobe.com, bringing in nearly enough subscription income to compensate for the traffic and ad dollars it will, and to some degree already has, leeched from Boston.com (at last count, in February, BostonGlobe.com had only recruited about 16,000 paid digital subscribers, many of whom had taken advantage of the site's introductory offer of 99 cents for the first four weeks but may not stick it out past the trial period). It might've been wiser for the paper to start out with a metered paywall to warm users to the idea of paying for content before erecting a full-blown ten-foot-tall paywall around its most valuable content under a new, separate domain name. But then again, who really knows – maybe it's a step in the right direction longterm? I'm not the one having to make the tough calls, so I'm in no position to judge.

At any rate, the segregation of the two sites has given the paper's parent company the freedom to make BostonGlobe.com a rich, immersive and interactive user experience that few other news organizations can match. Why, you ask? Because of a little trend gaining steam in the development world called "responsive design." See, BostonGlobe.com isn't only a standout site because of its sophisticated use of white space, its wholehearted embrace of web fonts and its visual-first approach to story art, but also because of its cross-platform capabilities. No matter what device you view BostonGlobe.com on – desktop, tablet or mobile – the site retains its same slick look and feel. The site's adaptive technology allows it to detect the screen size of the user, then adjust its layout to fit the exact resolution at hand. On desktops, that means content stretches to fill the entire browser window, and the grid restructures itself appropriately when you decrease or increase the size of the window. This eliminates the need for time-honored design standards like the 960 grid-system, which is based upon the Desktop-centric idea that all users have at least a 960px resolution. Now, the grid can be as big, or as small, as the user wants it to be.

The site's successful display of a dynamic range of real-time content on any-sized device also essentially eliminates the need for the "app-based" environment that's been the staple of the iOS5 and Android operating systems in recent years. To test it out, I gave the site a whirl on my iPad, and was pleased to see the front page neatly rearrange itself to fit the new orientation, just as any of the best platform-specific news apps out there do, including The New Yorker and Wired apps for iPad. But just because BostonGlobe.com doesn't need a native app to display its content beautifully on tablet and mobile devices doesn't mean that it won't be missing out on the growing app-centric marketplace for publishers. As Apple and Google continue to centralize digital consumption patterns into the app-based model, BostonGlobe.com may be a little too ambitious in thinking it doesn't need to play nice with the big-boy tech companies to reach its audience. On the other hand, however, it could prove to be a brilliant move for the paper, setting an example for other publishers and app-based companies in general who want to break free from the often constricting, not to mention pricey, cost of participating in the app marketplace.

For its design alone, BostonGlobe.com deserves any number of awards. Its bold, minimalist interface allows content to shine above all else, free from clutter and distraction. And with high-res horizontal photos, block-quotes, inline video and stylized headlines that grab your attention without hurting your eyes, that content stands out even more. What's more, the site has the functionality to back up its aesthetics. Its "Save" feature allows users to bookmark articles for reading offline on any device, which, even for sites without responsive design, is a brilliant feature for a news site to implement. Moreover, its "Story Flow" panel at the bottom of each article allows users to click seamlessly through to stories on similar subjects, just as readers would in an old-fashioned newspaper with physical sections. Which leaves me with one nagging question: Is BostonGlobe.com too imitative of physical newspapers to attain success in today's short attention-span digital audience? Is it too skewmorphic to succeed in an SEO-driven, hyperlinked news economy?

I guess we'll have to wait and see.

bubble charts

Overlaying a bubble chart onto a Google map

25 minute read

Others may hate, but I'm a big fan of using bubbles to display data. When implemented correctly (i.e. scaled in terms of area instead of diameter), bubbles can be an aesthetically appealing and concise way to represent the value of data points in an inherently visual format. Bubbles are even more useful when they include interactivity, with events like mouseover and zoom allowing users to drill down and compare similar-sized bubbles more easily than they can in static graphics. So, when I was recently working on a class project on autism diagnoses in New York City, I decided to use bubbles to represent the percentage of students with individualized education plans at all 1250 or so K-8 New York City schools.

Almost by default, I turned to Google Maps JavaScript API V3, mainly because I'm quasi-familiar with its basic functions and event handlers (as I point out later in  this post, I didn't realize that a nifty new service called CartoDB would have automated most of what I was trying to do, albeit without nearly the level of customization). Nonetheless, based on a tutorial from Karl Agius, as well as some infoWindow help from my data viz professor, Susan McGregor, I created the following interactive bubble map of NYC schools based upon the number of special needs, or IEP, students at each school. The larger the bubble, the greater percentage of special needs students a school has. Click here to see the map full-screen, or here to download a .zip of my source files for your own customization purposes.

Each bubble on this map represents one of New York City's approximately 1,250 K-8 public schools, including charters. The larger the bubble, the higher the percentage of students with individualized education plans (IEP). Click on a bubble to find out more about the school, or click anywhere within a district boundary to see an overall average IEP rate for the district. Zoom and pan to see other parts of the city.

You'll notice the opacity for the bubbles is set to 40 percent. This allows us to get a quick visual of the locations with the highest density of special needs students, given that those areas on the map will naturally be darker because they have multiple semi-opaque layers that overlap. Setting a low opacity also prevents overlapping bubbles from covering up one another. You'll also notice that the map includes polygons for each school district, which you can click on to get an average IEP rate for the entire district. I decided against setting gradated fill colors for the school district shapes so as to avoid implying causation, as well as to lessen the visual clutter.*

Preparing the data

To create the map, I first had to download the underlying data from the New York City Department of Education database as a .csv, then import it into Excel to clean it up and leave only the relevant information. Although the dataset I had only included street addresses split into multiple columns, I was able to use the concatenate function in Excel to merge the street, city, state and zip columns to get a full street address. From there, I used my favorite batch geocoding service to convert the addresses into geographic coordinates that the Google Maps API can read. Check out my resulting .csv file here for an example. Then I imported the .csv into a Google Spreadsheet, and pasted the resulting spreadsheet's URL into dataSourceURL field in the JavaScript of my main index.html file. Here's how that looked in my code:

var dataSourceUrl = "https://docs.google.com/spreadsheet/ccc?key=0Au4D8Alccn4xdHNJdXJmeTdYcEtpRXE1QXRucWtEN3c";
var onlyInfoWindow;

bubble trees

On Richard Boarman’s “Bubble Trees: The Visualization of Hierarchical Structure”

3 minute read

In his brief two-page paper "Bubble Trees: The Visualization of Hierarchical Structure," Richard Boardman proposes a new type of interactive presentation of hierarchical data that he calls the bubble tree. To bolster his argument, Boardman points out the difficulties inherent in the traditional "tree" structure, which suffers from the "breadth versus depth" problem by leading to information overload and taking up too much screen real estate. As a solution, he proposes a clickable bubble tree that leads to child and grandchild bubbles. Because of its interactive nature and nested structure, Boardman's bubble tree would "naturally allow the user to explore and work out relationships for themselves," he says.

Since the publication of Boardman's paper, this style of bubbletree has become somewhat ad nouveau in the information design community, with popular JavaScript libraries such as Bubbletree.js putting the creation of complex, hierarchical bubble trees into the hands of the general web development public. As its popular use has demonstrated, Bubbletree.js can be particularly handy when it comes to displaying Open Spending data.

bubbletree

What we can learn about charts from The WSJ Guide to Information Graphics

12 minute read

Although geared primarily toward the production of static graphics for print publications, Dona M. Wong's The Wall Street Journal Guide to Information Graphics (2010provides a wealth of salient and time-honored tips and guidelines that any student of data visualization would be well-advised to follow. At the heart of Wong's book is the notion that data integrity trumps all else, and no matter how aesthetically pleasing or visually powerful an information graphic may be, if it doesn't communicate clear and accurate data to the reader/user, it doesn't do its job.

In the first two chapters of The WSJ Guide, Wong, a former student of data viz extraordinaire Edward Tufte, addresses the topic of charting. From a theoretical standpoint, Wong lays out four principle steps to the charting process:

  1. Research: Find your data source, and ensure that it's timely, authoritative and free of bias.
  2. Edit:  Figure out what the data says (essentially, determine what your story is), and conceive of how best to boil that data down in a way that's simple enough for your intended audience to understand without skewing its meaning.
  3. Plot: Determine the appropriate chart type for your data (e.g. bar, column, line, pie, stacked bar, etc.), choose the right settings (scale, increments, axes, etc.), labeling the chart (e.g. legends and source lines) and pick the best color and typography combinations to accentuate your key message.
  4. Review: When you're done, ask yourself the following questions: Does the data match up with what external sources say? Are there any outliers? Does the chart make sense? What would the average user/reader think upon first seeing the chart?

Regarding the finer points of charting, Wong does an excellent job at pointing out the various dos and don'ts of the presentation process.  She sets forth clear guidelines about when to use what type of chart. For example, when dealing with change over time, Wong says to always use a line chart instead of a bar chart, as bar charts should ideally be reserved for comparing several different series of data. Also, Wong asserts, pie charts usually aren't as good of a choice for displaying complex data as bar or line charts, primarily because they make it harder to discern discrete differences in size (later, she flat-out dismisses the donut pie chart for the same reason). A few of her other tips I found particularly relevant included: (a) avoiding high-contrast color schemes that draw attention away from the data, (b) shying away from icons with high detail so as to avoid visual overload, (c) never, under any circumstances, add cloying shadow or 3D-effects and (d) never rely on zebra patterns, dotted lines or other fancy methods of labeling. "A chart is not a piece of fine art," Wong says.

Most importantly, Wong sets forth some other general principles to help designers avoid creating misleading charts. For example, when creating a bubble chart, always plot the bubbles by area, not radi. Also, never plot two different data series on noncomparable scales, and when creating bar charts, always start at the zero baseline. Other steps to ensuring data integrity include putting numbers into their appropriate context (comparing apples to apples), holding off on rounding until the end of the data analysis process and avoiding charting predictive numbers alongside actual ones. As Wong so eloquently puts it, "Unlike a misspelled word in a story, one wrong number discredits a whole chart."

Although the new addition of interactivity to chart design adds another layer of complexity to the visualization process that Wong doesn't address here, most of the guidelines she sets forth hold true in both static and dynamic mediums. Yet it would be interesting to hear what she has to say about the vexing question of when and when not to add static labels to interactive charts...

business insider

business models

“Newspapers are the new startups”

1 minute read

Newspapers are the new startups . . . we’re starting to see a lot of great changes as technologies improve and cultures change."
-John Levitt, Director of Sales and Marketing,  Parse.ly
</p>

Levitt's is one of the most insightful takes on the publishing industry I've heard in a while. It's going to take a lot of restructuring and a ground-up approach, but I'm excited to be a part of it as we embrace the start-up culture in Savannah.

cartodb

chloropleth

Making the case for hover interactions in maps

19 minute read

In keeping with my recent spate of mapping nerdiness, I decided to take an interactive map I produced last month displaying statewide annual population changes a step further by adding mouseover/hover capabilities. Here's the hover-y, nicely-colored chloropleth map I came up with. But before I get into the nitty-gritty of how I created the map –– which I'll explain step-by-step in a later post –– let me exercise a bit of self-indulgence by defending my growing belief in the need for hover capabilities when visualizing geographic data.

Not too long ago, I was an avid believer in the no-frills, less-interactivity-is-more approach to mapping geographic data, espoused by the brilliant Brian Boyer (@brianboyer), News Applications Editor for NPR and a former member of the News Apps Team at The Chicago Tribune. Boyer's argument for the need to keep maps simple –– like they used to be back in the days of ink and paper –– certainly has its merits. After all, the process of bringing a physical map closer to one's eyes to get a better view is a natural, timeless user interaction, and maps like this one, which Boyer produced during his time at The Tribune, are far more intuitive in communicating information upon first glance than many of the infoWindow-laden Google maps being produced by news organizations these days, many of them simply for the sake of being called 'interactive' (for those of you fortunate enough not to be mapping nerds, infoWindow is just Google-speak for the clickable popup boxes you see in Google maps).

But Boyer's minimalist mapping aesthetic only really works when you have one or two pieces of textual data you want to display for each geographic area. What if you have multiple pieces of information you want to display for each polygon, such as in this snazzy map from The Texas Tribune? Or, less likely but equally problematic, what if you need to bind non-textual data to your geographic polygons, such as images or Google Charts? In cases such as these, you're going to need to provide some sort of interaction that allows the user to expand and collapse the data for each area individually, or you'll just end up with a chronic case of visual overload.

Not to mention, on a more abstract level, studies have repeatedly shown that users tend to spend more time on applications that provide direct feedback based upon their actions, even if that feedback sometimes makes their ability to consume information at first glance less efficient (see Donald Norman's 2005 book Emotional Design: Why We Love (Or Hate) Everyday Things, in which Norman asserts that the feeling of emotional satisfaction and empowerment users receive from triggering an action not only puts them in a clearer state of mind, but also makes them more engaged in the information at hand). So, if we're trying to communicate geographic data to users as effectively as possible, it only makes sense that we'd want to have a certain degree of user interaction –– both for the sake of preventing visual overload and for making users feel more engaged. Such is the logic behind clickable infoWindows.

Still, clickable popups leave us with another problem: Users have to make the conscious and deliberate effort to click a polygon to see the data for that geographic area. Requiring clicks may sound like a trivial task to the designer or journalist-programmer, but for the short-attention span user, it can be an awful lot to ask for. To be fair, however, click-triggered popups may not be much of a problem for maps with only a few dozen polygons. But for maps with hundreds of small polygons –– say, census tracts or zip codes –– it can be very tedious to click the right polygon without first having to zoom into the map so far that you lose sight of the broader context.

That leads me back to a conversation I had a couple of months ago with a friend of mine from Columbia's J-School, Michael Keller (@mhkeller), who's now working as the Senior Data Reporter at The Daily Beast. Michael insisted to me after a Hacks/Hacker event that providing hover interaction for maps is almost always a good thing, because hovers require less work on the part of the user. I'll admit that I was dubious at the time, thinking of hovers as often unwanted, accidental triggers that can be distracting to the map and data at hand. But lately I've come around to his way of seeing things. If implemented correctly (i.e. no flashy interactions that cover up other parts of the map), hovers are almost always a good idea. For example, this recent map of New York Stop-and-Frisk data that Keller produced for The New York World using CartoDB and Leaflet is so detailed that it couldn't possibly have worked without infoWindows, and would have been unwieldy if it relied on click-triggered interaction. By including floating mouseover capabilities, the map allows the user to scan quickly through the chloropleth map to see individual Stop-and-Frisk data from each block, without having to attempt to click through minute geographic areas.*

I'm certainly not advocating interaction for interaction's sake (although such a case could be made, given the dynamics of the Web). But I am saying that hovers give more immediate visual feedback than click-triggered events, especially in maps. Hovers help draw users into the data without requiring them to seek it out consciously–– almost like a catchy lede would in a print narrative. So for the time-being, I'm pro-hover.

*Keller later messaged me letting me know that some examples which better illustrate the power of hovers include this map and this map, both of which use hover functionality to help highlight the effects of proposed redistricting efforts in the New York State Senate. What you'll also notice about Keller's maps is that they include hover states, which I also think is a necessity, especially for maps that include lots of small polygons.

chloropleth maps

INTERACTIVE: Why is the South the most obese part of the country? Five theories

17 minute read

This map displays the obesity rate of each U.S. state in 2010. The darker shade red represents a higher percentage of obese residents, while the green represents states with lower obesity rates. Click on each state to see the exact totals of each state's obesity rate.* 

Southerners need to lay off of the Crisco, cut back on the processed foods and start spending more time on the treadmill to fight the growing epidemic of obesity, experts say.

According to 2010 data released by the Centers for Disease Control and Prevention, the South is the most obese region in the nation, with about one in three of its residents classifying as chronically obese. That's far greater than the entire nation, where the figure is closer to one in four.

Of the 10 states with the highest rates of adult obesity, eight of them are in the South: Mississippi, Alabama, South Carolina, Kentucky, Louisiana, Texas and Tennessee. And that's only assuming you don't count West Virginia as being "southern."

Across the nation, the epidemic has grown worse in recent years. Twelve states now have obesity rates higher than 30 percent, compared to four years ago when only one state, Mississippi, ranked above the 30 percent threshold. The only state in the Deep South without an obesity rate of more than 30 percent today is Georgia, but that appears to be primarily because the more physically fit population of metro Atlanta offsets the rest of the state's obesity.

But what's making the South –– the region CDC Dr. William Dietz has dubbed "the heart disease and stroke belt" –– more chubby than the rest of the nation? Here are five possible explanations:

1. High poverty -– The South may be obese, but it's also poor. With a poverty rate of 14 percent, the South is easily the most impoverished region in the country. And according to data from the USDA, states with a higher poverty rate also tend to have a higher number of obese citizens. Experts say that's because people with a low income are more likely to purchase high-calorie inexpensive processed foods, which contribute to weight gain. "If you overlay a map of obesity onto a map of poverty, the two very clearly correspond," said David A. Davis, a professor of Southern Studies at Mercer University who has conducted extensive research on southern foodways. "The southern diet is a diet of poverty, and it's one based on cheap, fatty processed foods."

2. The "grocery gap" – Because the South is largely rural, many residents don't have quality access to fresh fruits and vegetables, and are forced to drive long distances to find anything healthier than potato chips and sodas at roadside gas stations. All five states with above average of what the USDA calls "food insecurity" levels are located in the South: Texas, Arkansas, Mississippi, Georgia and North Carolina. What's more, it's significantly more expensive to purchase low-fat items in the South than in the rest of the nation. For example, Georgia, Mississippi, North Carolina and Virginia all topped the list of states where it costs the most to buy low-fat milk, USDA data says.

3. The grease-fed "southern" culinary tradition -– One of the easiest explanations for the South's staggering obesity rates is the region's tradition of fried chicken, sweet tea and gravy on top of everything – or what's commonly referred to by non-southerners as the "Paula Deen" effect. "To me, it's simply a cultural habit regarding what we eat, not an issue of poverty," says Andy Breck, director of the Center for a Better South, a nonprofit group based out of the University of South Carolina that seeks to raise awareness about ongoing issues facing the region. "People are fat in Mississippi. People are fat in South Carolina. People are fat in Alabama. There's got to be something going on. And it's not just poor people. It's middle and upper-class folks who grotesquely overeat, because that's all they've ever known to do."

4. Lack of physical activity–– Southerners also tend to be less physically active than the rest of the country, burning off fewer calories and retaining more body fat, USDA data says. All five U.S. states where less than 60 percent of adults met the USDA's recommended physical activity guidelines in 2008 were located below the Mason Dixon Line: Louisiana, Mississippi, Alabama, Tennessee and Kentucky. Some researchers have speculated that the South's lack of physical activity may not be so much sheer laziness as it is a lack of access to places to exercise. Few rural areas have fancy private gyms for southerners to burn off their extra calories, and most of the year it's just too plain hot in the South to exercise outdoors.

5. Lack of quality education – Perhaps at the heart of the southern obesity epidemic, however, is the region's crippling lack of quality public education. "I don't buy the fact that the South is fat because of traditional southern foodways," said Davis, who teaches classes on southern poverty and culture and has written numerous articles on the subject. "To me, it's more of a poverty and an educational problem. If we don't educate people, especially in terms of health education, we're going to keep having obese citizens."

*Source: The Centers for Disease Control and Prevention, April 3, 2012

css

Building a responsive site in less than 20 minutes

6 minute read

An ever-so-sleek responsive portfolio site I designed for a friend in less than 20 minutes using Skeleton as a foundation.[/caption]

With all this talk lately of the new era of responsive design, I realized today that I've yet to create anything that's actually responsive. Given that I've only pondered using it in the implementation of complex, database-driven news sites, the task of tweaking every level of CSS to fit perfectly into a responsive grid system has so far seemed too daunting to tackle.

But I got curious this afternoon, and found my way upon a cool new library called Skeleton, which bills itself as a "a beautiful boilerplate for responsive, mobile-friendly development." Essentially, Skeleton is a collection of CSS and JS files that makes the mystery of responsive design seem a little less illusive. Upon uploading the package to my server, I was pleased to find a neatly-coded, easy-to-understand responsive site that I could play around with and tweak to my own liking. I ended up adjusting the grid size and performing some minor customizations to the underlying Skeleton structural CSS, but other than that, the development kit was, as it promised, "style agnostic."

Since I didn't have any specific projects I needed to be working on today, I decided to give Skeleton a whirl by designing a new online portfolio as a favor for my friend Daniel Medina. I'm not kidding – within 10 minutes I'd coded a fully satisfactory responsive portfolio site that looks beautiful on my iPhone and tablet.

So, if you're feeling experimental, try Skeleton out. It's exciting to see how quickly this technology is catching steam, and to see the actual workings of the responsive CSS in action firsthand. My next goal is to integrate a responsive layout into a more-powerful database-driven site, perhaps one designed in Drupal.

data

Should data viz be a specialty or a commodity skill in the newsroom?

8 minute read

An interesting question came up at last Wednesday's Doing Data Journalism (#doingdataj) panel hosted by the Tow Center for Digital Journalism here at Columbia's J-School: Should there be data specialists in the newsroom, or can everyone be a data journalist? For New York Times interactive editor Aron Pilholfer, who participated in the panel, the question is not so much should everyone do data as will everyone do data. And for Pilholfer, the answer to that question clearly seems to be no:

I kind of naively thought that at one time you could train everybody to be at least a base level of competency with something like Excel, but I'm not of that belief anymore. I think you do need specialists.

I've always hated the idea of having technology or innovation 'specialists' in a work environment that should ideally be collaborative. So, at first I tended to disagree with Pilholfer's argument. But what won me over was the reasoning behind his claim. For Pilholfer, it's not that the technology, human talent or open source tools aren't there for everyone to scrape, analyze and process data –– in fact, it's now easier than ever to organize messy data with simple and often free desktop applications like Excel and Google Refine. The problem is that there's a cultural lack of interest within newsrooms, often from an editorial level, to produce data-driven stories. As Pilholfer says in what appears to be an indictment of upper-level editors for disregarding the value of data,

The problem is that we continue to reward crap journalism that's based on anecdotal evidence alone . . . But truly if it's not a priority at the top to reward good data-driven journalism, it's going to be impossible to get people into data because they just don't think it's worth it.

I totally agree, but with one lurking suspicion. As with the top-level editors, many traditional users –– or 'readers,' as one might call them –– still at least think they like to read pretty, anecdotal narratives, and tend not to care as much whether the hard data backs them up. In other words, it’s an audience problem just as much as it is a managerial or institutional one. Some legacy news consumers just still aren't data literate. Because they're not accustomed to even having such data freely available to them, they don't even value having it. As the old saying goes, "You can't miss what you never had." Yet as traffic and engagement statistics continually confirm, as soon users have open data readily available to them through news apps and data visualizations, they spend more time accessing the data than they do reading the print narrative.

data visualization

Overlaying a bubble chart onto a Google map

25 minute read

Others may hate, but I'm a big fan of using bubbles to display data. When implemented correctly (i.e. scaled in terms of area instead of diameter), bubbles can be an aesthetically appealing and concise way to represent the value of data points in an inherently visual format. Bubbles are even more useful when they include interactivity, with events like mouseover and zoom allowing users to drill down and compare similar-sized bubbles more easily than they can in static graphics. So, when I was recently working on a class project on autism diagnoses in New York City, I decided to use bubbles to represent the percentage of students with individualized education plans at all 1250 or so K-8 New York City schools.

Almost by default, I turned to Google Maps JavaScript API V3, mainly because I'm quasi-familiar with its basic functions and event handlers (as I point out later in  this post, I didn't realize that a nifty new service called CartoDB would have automated most of what I was trying to do, albeit without nearly the level of customization). Nonetheless, based on a tutorial from Karl Agius, as well as some infoWindow help from my data viz professor, Susan McGregor, I created the following interactive bubble map of NYC schools based upon the number of special needs, or IEP, students at each school. The larger the bubble, the greater percentage of special needs students a school has. Click here to see the map full-screen, or here to download a .zip of my source files for your own customization purposes.

Each bubble on this map represents one of New York City's approximately 1,250 K-8 public schools, including charters. The larger the bubble, the higher the percentage of students with individualized education plans (IEP). Click on a bubble to find out more about the school, or click anywhere within a district boundary to see an overall average IEP rate for the district. Zoom and pan to see other parts of the city.

You'll notice the opacity for the bubbles is set to 40 percent. This allows us to get a quick visual of the locations with the highest density of special needs students, given that those areas on the map will naturally be darker because they have multiple semi-opaque layers that overlap. Setting a low opacity also prevents overlapping bubbles from covering up one another. You'll also notice that the map includes polygons for each school district, which you can click on to get an average IEP rate for the entire district. I decided against setting gradated fill colors for the school district shapes so as to avoid implying causation, as well as to lessen the visual clutter.*

Preparing the data

To create the map, I first had to download the underlying data from the New York City Department of Education database as a .csv, then import it into Excel to clean it up and leave only the relevant information. Although the dataset I had only included street addresses split into multiple columns, I was able to use the concatenate function in Excel to merge the street, city, state and zip columns to get a full street address. From there, I used my favorite batch geocoding service to convert the addresses into geographic coordinates that the Google Maps API can read. Check out my resulting .csv file here for an example. Then I imported the .csv into a Google Spreadsheet, and pasted the resulting spreadsheet's URL into dataSourceURL field in the JavaScript of my main index.html file. Here's how that looked in my code:

var dataSourceUrl = "https://docs.google.com/spreadsheet/ccc?key=0Au4D8Alccn4xdHNJdXJmeTdYcEtpRXE1QXRucWtEN3c";
var onlyInfoWindow;

Why calculus matters when it comes to data-driven stories

7 minute read

A quick refresher from my data visualization professor here at Columbia a couple of weeks ago reminded me why I was forced to spend all those grueling hours calculating standard deviation back in high school.

See, when you're using a data set to tell a story, the first step is to understand what that data says. And to do that, you've got to have a good idea of the range and variation of the values at hand. Not only can figuring that information out help you determine whether there's any statistical significance to your data set, but it can also pinpoint outliers and possible errors that may exist within the data before you begin the work of visualizing it.

Thanks to powerful processing programs like Excel, we can figure out the variability of data sets pretty easily using the program's built-in standard deviation function (remember this intimidating-looking equation from calculus class?). Still, it always helps to know how to calculate the information out by hand, if only to get a conceptual idea of why numbers such as the standard deviation (the variability of a data-set) and the z-value (the number of standard deviations a given value is away from the mean) even matter in the first place when it comes to data visualization.

So, to brush up on my formulas and also better understand the numbers behind an actual story assignment for one of my classes, I recently hand-calculated the standard deviation and z-values for a set of data on state-by-state obesity rates. From my calculations, I was able to use the standard deviation (3.24) to determine that, on average, most states fell within the middle of the bell-curve for the average national obesity rate (27.1 percent) . In addition, the z-values helped me understand which states stood out from the pack as possible outliers (Mississippi is by far the most obese with a 2.13 z-value, Colorado the least obese with a -1.9 z-value). To get an idea of how those formulas look hand-calculated in Excel, check out my spreadsheet here. And keep these formulas in mind while working on your next data story. They can potentially save you time and effort by helping you figure out what your data set says before you have to go through the often-lengthy process of visualizing it.

What makes BostonGlobe.com “the world’s best designed website”

16 minute read

With the Pulitzer Price announcements coming up later this afternoon, you'd think I'd be writing about whose up for the "Best Deadline Reporting" or "Best Public Service Journalism" prizes. But instead I want to talk about a different media award doled out during the past week: BostonGlobe.com's designation as the "world's best designed website" by the Society for News Design. Put simply, I can't say I disagree.

Yet before I divulge in my effusive praise of the folks in Boston, let me point out that I'm still somewhat skeptical of the business-logic behind The Globe's decision to launch its paywall-only site last fall alongside its primary news portal, Boston.com. From a revenue standpoint, I can't see the full-paywall site, BostonGlobe.com, bringing in nearly enough subscription income to compensate for the traffic and ad dollars it will, and to some degree already has, leeched from Boston.com (at last count, in February, BostonGlobe.com had only recruited about 16,000 paid digital subscribers, many of whom had taken advantage of the site's introductory offer of 99 cents for the first four weeks but may not stick it out past the trial period). It might've been wiser for the paper to start out with a metered paywall to warm users to the idea of paying for content before erecting a full-blown ten-foot-tall paywall around its most valuable content under a new, separate domain name. But then again, who really knows – maybe it's a step in the right direction longterm? I'm not the one having to make the tough calls, so I'm in no position to judge.

At any rate, the segregation of the two sites has given the paper's parent company the freedom to make BostonGlobe.com a rich, immersive and interactive user experience that few other news organizations can match. Why, you ask? Because of a little trend gaining steam in the development world called "responsive design." See, BostonGlobe.com isn't only a standout site because of its sophisticated use of white space, its wholehearted embrace of web fonts and its visual-first approach to story art, but also because of its cross-platform capabilities. No matter what device you view BostonGlobe.com on – desktop, tablet or mobile – the site retains its same slick look and feel. The site's adaptive technology allows it to detect the screen size of the user, then adjust its layout to fit the exact resolution at hand. On desktops, that means content stretches to fill the entire browser window, and the grid restructures itself appropriately when you decrease or increase the size of the window. This eliminates the need for time-honored design standards like the 960 grid-system, which is based upon the Desktop-centric idea that all users have at least a 960px resolution. Now, the grid can be as big, or as small, as the user wants it to be.

The site's successful display of a dynamic range of real-time content on any-sized device also essentially eliminates the need for the "app-based" environment that's been the staple of the iOS5 and Android operating systems in recent years. To test it out, I gave the site a whirl on my iPad, and was pleased to see the front page neatly rearrange itself to fit the new orientation, just as any of the best platform-specific news apps out there do, including The New Yorker and Wired apps for iPad. But just because BostonGlobe.com doesn't need a native app to display its content beautifully on tablet and mobile devices doesn't mean that it won't be missing out on the growing app-centric marketplace for publishers. As Apple and Google continue to centralize digital consumption patterns into the app-based model, BostonGlobe.com may be a little too ambitious in thinking it doesn't need to play nice with the big-boy tech companies to reach its audience. On the other hand, however, it could prove to be a brilliant move for the paper, setting an example for other publishers and app-based companies in general who want to break free from the often constricting, not to mention pricey, cost of participating in the app marketplace.

For its design alone, BostonGlobe.com deserves any number of awards. Its bold, minimalist interface allows content to shine above all else, free from clutter and distraction. And with high-res horizontal photos, block-quotes, inline video and stylized headlines that grab your attention without hurting your eyes, that content stands out even more. What's more, the site has the functionality to back up its aesthetics. Its "Save" feature allows users to bookmark articles for reading offline on any device, which, even for sites without responsive design, is a brilliant feature for a news site to implement. Moreover, its "Story Flow" panel at the bottom of each article allows users to click seamlessly through to stories on similar subjects, just as readers would in an old-fashioned newspaper with physical sections. Which leaves me with one nagging question: Is BostonGlobe.com too imitative of physical newspapers to attain success in today's short attention-span digital audience? Is it too skewmorphic to succeed in an SEO-driven, hyperlinked news economy?

I guess we'll have to wait and see.

INTERACTIVE: Why is the South the most obese part of the country? Five theories

17 minute read

This map displays the obesity rate of each U.S. state in 2010. The darker shade red represents a higher percentage of obese residents, while the green represents states with lower obesity rates. Click on each state to see the exact totals of each state's obesity rate.* 

Southerners need to lay off of the Crisco, cut back on the processed foods and start spending more time on the treadmill to fight the growing epidemic of obesity, experts say.

According to 2010 data released by the Centers for Disease Control and Prevention, the South is the most obese region in the nation, with about one in three of its residents classifying as chronically obese. That's far greater than the entire nation, where the figure is closer to one in four.

Of the 10 states with the highest rates of adult obesity, eight of them are in the South: Mississippi, Alabama, South Carolina, Kentucky, Louisiana, Texas and Tennessee. And that's only assuming you don't count West Virginia as being "southern."

Across the nation, the epidemic has grown worse in recent years. Twelve states now have obesity rates higher than 30 percent, compared to four years ago when only one state, Mississippi, ranked above the 30 percent threshold. The only state in the Deep South without an obesity rate of more than 30 percent today is Georgia, but that appears to be primarily because the more physically fit population of metro Atlanta offsets the rest of the state's obesity.

But what's making the South –– the region CDC Dr. William Dietz has dubbed "the heart disease and stroke belt" –– more chubby than the rest of the nation? Here are five possible explanations:

1. High poverty -– The South may be obese, but it's also poor. With a poverty rate of 14 percent, the South is easily the most impoverished region in the country. And according to data from the USDA, states with a higher poverty rate also tend to have a higher number of obese citizens. Experts say that's because people with a low income are more likely to purchase high-calorie inexpensive processed foods, which contribute to weight gain. "If you overlay a map of obesity onto a map of poverty, the two very clearly correspond," said David A. Davis, a professor of Southern Studies at Mercer University who has conducted extensive research on southern foodways. "The southern diet is a diet of poverty, and it's one based on cheap, fatty processed foods."

2. The "grocery gap" – Because the South is largely rural, many residents don't have quality access to fresh fruits and vegetables, and are forced to drive long distances to find anything healthier than potato chips and sodas at roadside gas stations. All five states with above average of what the USDA calls "food insecurity" levels are located in the South: Texas, Arkansas, Mississippi, Georgia and North Carolina. What's more, it's significantly more expensive to purchase low-fat items in the South than in the rest of the nation. For example, Georgia, Mississippi, North Carolina and Virginia all topped the list of states where it costs the most to buy low-fat milk, USDA data says.

3. The grease-fed "southern" culinary tradition -– One of the easiest explanations for the South's staggering obesity rates is the region's tradition of fried chicken, sweet tea and gravy on top of everything – or what's commonly referred to by non-southerners as the "Paula Deen" effect. "To me, it's simply a cultural habit regarding what we eat, not an issue of poverty," says Andy Breck, director of the Center for a Better South, a nonprofit group based out of the University of South Carolina that seeks to raise awareness about ongoing issues facing the region. "People are fat in Mississippi. People are fat in South Carolina. People are fat in Alabama. There's got to be something going on. And it's not just poor people. It's middle and upper-class folks who grotesquely overeat, because that's all they've ever known to do."

4. Lack of physical activity–– Southerners also tend to be less physically active than the rest of the country, burning off fewer calories and retaining more body fat, USDA data says. All five U.S. states where less than 60 percent of adults met the USDA's recommended physical activity guidelines in 2008 were located below the Mason Dixon Line: Louisiana, Mississippi, Alabama, Tennessee and Kentucky. Some researchers have speculated that the South's lack of physical activity may not be so much sheer laziness as it is a lack of access to places to exercise. Few rural areas have fancy private gyms for southerners to burn off their extra calories, and most of the year it's just too plain hot in the South to exercise outdoors.

5. Lack of quality education – Perhaps at the heart of the southern obesity epidemic, however, is the region's crippling lack of quality public education. "I don't buy the fact that the South is fat because of traditional southern foodways," said Davis, who teaches classes on southern poverty and culture and has written numerous articles on the subject. "To me, it's more of a poverty and an educational problem. If we don't educate people, especially in terms of health education, we're going to keep having obese citizens."

*Source: The Centers for Disease Control and Prevention, April 3, 2012

data viz

What we can learn about charts from The WSJ Guide to Information Graphics

12 minute read

Although geared primarily toward the production of static graphics for print publications, Dona M. Wong's The Wall Street Journal Guide to Information Graphics (2010provides a wealth of salient and time-honored tips and guidelines that any student of data visualization would be well-advised to follow. At the heart of Wong's book is the notion that data integrity trumps all else, and no matter how aesthetically pleasing or visually powerful an information graphic may be, if it doesn't communicate clear and accurate data to the reader/user, it doesn't do its job.

In the first two chapters of The WSJ Guide, Wong, a former student of data viz extraordinaire Edward Tufte, addresses the topic of charting. From a theoretical standpoint, Wong lays out four principle steps to the charting process:

  1. Research: Find your data source, and ensure that it's timely, authoritative and free of bias.
  2. Edit:  Figure out what the data says (essentially, determine what your story is), and conceive of how best to boil that data down in a way that's simple enough for your intended audience to understand without skewing its meaning.
  3. Plot: Determine the appropriate chart type for your data (e.g. bar, column, line, pie, stacked bar, etc.), choose the right settings (scale, increments, axes, etc.), labeling the chart (e.g. legends and source lines) and pick the best color and typography combinations to accentuate your key message.
  4. Review: When you're done, ask yourself the following questions: Does the data match up with what external sources say? Are there any outliers? Does the chart make sense? What would the average user/reader think upon first seeing the chart?

Regarding the finer points of charting, Wong does an excellent job at pointing out the various dos and don'ts of the presentation process.  She sets forth clear guidelines about when to use what type of chart. For example, when dealing with change over time, Wong says to always use a line chart instead of a bar chart, as bar charts should ideally be reserved for comparing several different series of data. Also, Wong asserts, pie charts usually aren't as good of a choice for displaying complex data as bar or line charts, primarily because they make it harder to discern discrete differences in size (later, she flat-out dismisses the donut pie chart for the same reason). A few of her other tips I found particularly relevant included: (a) avoiding high-contrast color schemes that draw attention away from the data, (b) shying away from icons with high detail so as to avoid visual overload, (c) never, under any circumstances, add cloying shadow or 3D-effects and (d) never rely on zebra patterns, dotted lines or other fancy methods of labeling. "A chart is not a piece of fine art," Wong says.

Most importantly, Wong sets forth some other general principles to help designers avoid creating misleading charts. For example, when creating a bubble chart, always plot the bubbles by area, not radi. Also, never plot two different data series on noncomparable scales, and when creating bar charts, always start at the zero baseline. Other steps to ensuring data integrity include putting numbers into their appropriate context (comparing apples to apples), holding off on rounding until the end of the data analysis process and avoiding charting predictive numbers alongside actual ones. As Wong so eloquently puts it, "Unlike a misspelled word in a story, one wrong number discredits a whole chart."

Although the new addition of interactivity to chart design adds another layer of complexity to the visualization process that Wong doesn't address here, most of the guidelines she sets forth hold true in both static and dynamic mediums. Yet it would be interesting to hear what she has to say about the vexing question of when and when not to add static labels to interactive charts...

On Richard Boarman’s “Bubble Trees: The Visualization of Hierarchical Structure”

3 minute read

In his brief two-page paper "Bubble Trees: The Visualization of Hierarchical Structure," Richard Boardman proposes a new type of interactive presentation of hierarchical data that he calls the bubble tree. To bolster his argument, Boardman points out the difficulties inherent in the traditional "tree" structure, which suffers from the "breadth versus depth" problem by leading to information overload and taking up too much screen real estate. As a solution, he proposes a clickable bubble tree that leads to child and grandchild bubbles. Because of its interactive nature and nested structure, Boardman's bubble tree would "naturally allow the user to explore and work out relationships for themselves," he says.

Since the publication of Boardman's paper, this style of bubbletree has become somewhat ad nouveau in the information design community, with popular JavaScript libraries such as Bubbletree.js putting the creation of complex, hierarchical bubble trees into the hands of the general web development public. As its popular use has demonstrated, Bubbletree.js can be particularly handy when it comes to displaying Open Spending data.

development

Building a responsive site in less than 20 minutes

6 minute read

An ever-so-sleek responsive portfolio site I designed for a friend in less than 20 minutes using Skeleton as a foundation.[/caption]

With all this talk lately of the new era of responsive design, I realized today that I've yet to create anything that's actually responsive. Given that I've only pondered using it in the implementation of complex, database-driven news sites, the task of tweaking every level of CSS to fit perfectly into a responsive grid system has so far seemed too daunting to tackle.

But I got curious this afternoon, and found my way upon a cool new library called Skeleton, which bills itself as a "a beautiful boilerplate for responsive, mobile-friendly development." Essentially, Skeleton is a collection of CSS and JS files that makes the mystery of responsive design seem a little less illusive. Upon uploading the package to my server, I was pleased to find a neatly-coded, easy-to-understand responsive site that I could play around with and tweak to my own liking. I ended up adjusting the grid size and performing some minor customizations to the underlying Skeleton structural CSS, but other than that, the development kit was, as it promised, "style agnostic."

Since I didn't have any specific projects I needed to be working on today, I decided to give Skeleton a whirl by designing a new online portfolio as a favor for my friend Daniel Medina. I'm not kidding – within 10 minutes I'd coded a fully satisfactory responsive portfolio site that looks beautiful on my iPhone and tablet.

So, if you're feeling experimental, try Skeleton out. It's exciting to see how quickly this technology is catching steam, and to see the actual workings of the responsive CSS in action firsthand. My next goal is to integrate a responsive layout into a more-powerful database-driven site, perhaps one designed in Drupal.

digital journalism

geocoding

geospatial

google charts API

INTERACTIVE: Why is the South the most obese part of the country? Five theories

17 minute read

This map displays the obesity rate of each U.S. state in 2010. The darker shade red represents a higher percentage of obese residents, while the green represents states with lower obesity rates. Click on each state to see the exact totals of each state's obesity rate.* 

Southerners need to lay off of the Crisco, cut back on the processed foods and start spending more time on the treadmill to fight the growing epidemic of obesity, experts say.

According to 2010 data released by the Centers for Disease Control and Prevention, the South is the most obese region in the nation, with about one in three of its residents classifying as chronically obese. That's far greater than the entire nation, where the figure is closer to one in four.

Of the 10 states with the highest rates of adult obesity, eight of them are in the South: Mississippi, Alabama, South Carolina, Kentucky, Louisiana, Texas and Tennessee. And that's only assuming you don't count West Virginia as being "southern."

Across the nation, the epidemic has grown worse in recent years. Twelve states now have obesity rates higher than 30 percent, compared to four years ago when only one state, Mississippi, ranked above the 30 percent threshold. The only state in the Deep South without an obesity rate of more than 30 percent today is Georgia, but that appears to be primarily because the more physically fit population of metro Atlanta offsets the rest of the state's obesity.

But what's making the South –– the region CDC Dr. William Dietz has dubbed "the heart disease and stroke belt" –– more chubby than the rest of the nation? Here are five possible explanations:

1. High poverty -– The South may be obese, but it's also poor. With a poverty rate of 14 percent, the South is easily the most impoverished region in the country. And according to data from the USDA, states with a higher poverty rate also tend to have a higher number of obese citizens. Experts say that's because people with a low income are more likely to purchase high-calorie inexpensive processed foods, which contribute to weight gain. "If you overlay a map of obesity onto a map of poverty, the two very clearly correspond," said David A. Davis, a professor of Southern Studies at Mercer University who has conducted extensive research on southern foodways. "The southern diet is a diet of poverty, and it's one based on cheap, fatty processed foods."

2. The "grocery gap" – Because the South is largely rural, many residents don't have quality access to fresh fruits and vegetables, and are forced to drive long distances to find anything healthier than potato chips and sodas at roadside gas stations. All five states with above average of what the USDA calls "food insecurity" levels are located in the South: Texas, Arkansas, Mississippi, Georgia and North Carolina. What's more, it's significantly more expensive to purchase low-fat items in the South than in the rest of the nation. For example, Georgia, Mississippi, North Carolina and Virginia all topped the list of states where it costs the most to buy low-fat milk, USDA data says.

3. The grease-fed "southern" culinary tradition -– One of the easiest explanations for the South's staggering obesity rates is the region's tradition of fried chicken, sweet tea and gravy on top of everything – or what's commonly referred to by non-southerners as the "Paula Deen" effect. "To me, it's simply a cultural habit regarding what we eat, not an issue of poverty," says Andy Breck, director of the Center for a Better South, a nonprofit group based out of the University of South Carolina that seeks to raise awareness about ongoing issues facing the region. "People are fat in Mississippi. People are fat in South Carolina. People are fat in Alabama. There's got to be something going on. And it's not just poor people. It's middle and upper-class folks who grotesquely overeat, because that's all they've ever known to do."

4. Lack of physical activity–– Southerners also tend to be less physically active than the rest of the country, burning off fewer calories and retaining more body fat, USDA data says. All five U.S. states where less than 60 percent of adults met the USDA's recommended physical activity guidelines in 2008 were located below the Mason Dixon Line: Louisiana, Mississippi, Alabama, Tennessee and Kentucky. Some researchers have speculated that the South's lack of physical activity may not be so much sheer laziness as it is a lack of access to places to exercise. Few rural areas have fancy private gyms for southerners to burn off their extra calories, and most of the year it's just too plain hot in the South to exercise outdoors.

5. Lack of quality education – Perhaps at the heart of the southern obesity epidemic, however, is the region's crippling lack of quality public education. "I don't buy the fact that the South is fat because of traditional southern foodways," said Davis, who teaches classes on southern poverty and culture and has written numerous articles on the subject. "To me, it's more of a poverty and an educational problem. If we don't educate people, especially in terms of health education, we're going to keep having obese citizens."

*Source: The Centers for Disease Control and Prevention, April 3, 2012

google maps javascript api v3

Overlaying a bubble chart onto a Google map

25 minute read

Others may hate, but I'm a big fan of using bubbles to display data. When implemented correctly (i.e. scaled in terms of area instead of diameter), bubbles can be an aesthetically appealing and concise way to represent the value of data points in an inherently visual format. Bubbles are even more useful when they include interactivity, with events like mouseover and zoom allowing users to drill down and compare similar-sized bubbles more easily than they can in static graphics. So, when I was recently working on a class project on autism diagnoses in New York City, I decided to use bubbles to represent the percentage of students with individualized education plans at all 1250 or so K-8 New York City schools.

Almost by default, I turned to Google Maps JavaScript API V3, mainly because I'm quasi-familiar with its basic functions and event handlers (as I point out later in  this post, I didn't realize that a nifty new service called CartoDB would have automated most of what I was trying to do, albeit without nearly the level of customization). Nonetheless, based on a tutorial from Karl Agius, as well as some infoWindow help from my data viz professor, Susan McGregor, I created the following interactive bubble map of NYC schools based upon the number of special needs, or IEP, students at each school. The larger the bubble, the greater percentage of special needs students a school has. Click here to see the map full-screen, or here to download a .zip of my source files for your own customization purposes.

Each bubble on this map represents one of New York City's approximately 1,250 K-8 public schools, including charters. The larger the bubble, the higher the percentage of students with individualized education plans (IEP). Click on a bubble to find out more about the school, or click anywhere within a district boundary to see an overall average IEP rate for the district. Zoom and pan to see other parts of the city.

You'll notice the opacity for the bubbles is set to 40 percent. This allows us to get a quick visual of the locations with the highest density of special needs students, given that those areas on the map will naturally be darker because they have multiple semi-opaque layers that overlap. Setting a low opacity also prevents overlapping bubbles from covering up one another. You'll also notice that the map includes polygons for each school district, which you can click on to get an average IEP rate for the entire district. I decided against setting gradated fill colors for the school district shapes so as to avoid implying causation, as well as to lessen the visual clutter.*

Preparing the data

To create the map, I first had to download the underlying data from the New York City Department of Education database as a .csv, then import it into Excel to clean it up and leave only the relevant information. Although the dataset I had only included street addresses split into multiple columns, I was able to use the concatenate function in Excel to merge the street, city, state and zip columns to get a full street address. From there, I used my favorite batch geocoding service to convert the addresses into geographic coordinates that the Google Maps API can read. Check out my resulting .csv file here for an example. Then I imported the .csv into a Google Spreadsheet, and pasted the resulting spreadsheet's URL into dataSourceURL field in the JavaScript of my main index.html file. Here's how that looked in my code:

var dataSourceUrl = "https://docs.google.com/spreadsheet/ccc?key=0Au4D8Alccn4xdHNJdXJmeTdYcEtpRXE1QXRucWtEN3c";
var onlyInfoWindow;

hierarchical data

What we can learn about charts from The WSJ Guide to Information Graphics

12 minute read

Although geared primarily toward the production of static graphics for print publications, Dona M. Wong's The Wall Street Journal Guide to Information Graphics (2010provides a wealth of salient and time-honored tips and guidelines that any student of data visualization would be well-advised to follow. At the heart of Wong's book is the notion that data integrity trumps all else, and no matter how aesthetically pleasing or visually powerful an information graphic may be, if it doesn't communicate clear and accurate data to the reader/user, it doesn't do its job.

In the first two chapters of The WSJ Guide, Wong, a former student of data viz extraordinaire Edward Tufte, addresses the topic of charting. From a theoretical standpoint, Wong lays out four principle steps to the charting process:

  1. Research: Find your data source, and ensure that it's timely, authoritative and free of bias.
  2. Edit:  Figure out what the data says (essentially, determine what your story is), and conceive of how best to boil that data down in a way that's simple enough for your intended audience to understand without skewing its meaning.
  3. Plot: Determine the appropriate chart type for your data (e.g. bar, column, line, pie, stacked bar, etc.), choose the right settings (scale, increments, axes, etc.), labeling the chart (e.g. legends and source lines) and pick the best color and typography combinations to accentuate your key message.
  4. Review: When you're done, ask yourself the following questions: Does the data match up with what external sources say? Are there any outliers? Does the chart make sense? What would the average user/reader think upon first seeing the chart?

Regarding the finer points of charting, Wong does an excellent job at pointing out the various dos and don'ts of the presentation process.  She sets forth clear guidelines about when to use what type of chart. For example, when dealing with change over time, Wong says to always use a line chart instead of a bar chart, as bar charts should ideally be reserved for comparing several different series of data. Also, Wong asserts, pie charts usually aren't as good of a choice for displaying complex data as bar or line charts, primarily because they make it harder to discern discrete differences in size (later, she flat-out dismisses the donut pie chart for the same reason). A few of her other tips I found particularly relevant included: (a) avoiding high-contrast color schemes that draw attention away from the data, (b) shying away from icons with high detail so as to avoid visual overload, (c) never, under any circumstances, add cloying shadow or 3D-effects and (d) never rely on zebra patterns, dotted lines or other fancy methods of labeling. "A chart is not a piece of fine art," Wong says.

Most importantly, Wong sets forth some other general principles to help designers avoid creating misleading charts. For example, when creating a bubble chart, always plot the bubbles by area, not radi. Also, never plot two different data series on noncomparable scales, and when creating bar charts, always start at the zero baseline. Other steps to ensuring data integrity include putting numbers into their appropriate context (comparing apples to apples), holding off on rounding until the end of the data analysis process and avoiding charting predictive numbers alongside actual ones. As Wong so eloquently puts it, "Unlike a misspelled word in a story, one wrong number discredits a whole chart."

Although the new addition of interactivity to chart design adds another layer of complexity to the visualization process that Wong doesn't address here, most of the guidelines she sets forth hold true in both static and dynamic mediums. Yet it would be interesting to hear what she has to say about the vexing question of when and when not to add static labels to interactive charts...

hover

Making the case for hover interactions in maps

19 minute read

In keeping with my recent spate of mapping nerdiness, I decided to take an interactive map I produced last month displaying statewide annual population changes a step further by adding mouseover/hover capabilities. Here's the hover-y, nicely-colored chloropleth map I came up with. But before I get into the nitty-gritty of how I created the map –– which I'll explain step-by-step in a later post –– let me exercise a bit of self-indulgence by defending my growing belief in the need for hover capabilities when visualizing geographic data.

Not too long ago, I was an avid believer in the no-frills, less-interactivity-is-more approach to mapping geographic data, espoused by the brilliant Brian Boyer (@brianboyer), News Applications Editor for NPR and a former member of the News Apps Team at The Chicago Tribune. Boyer's argument for the need to keep maps simple –– like they used to be back in the days of ink and paper –– certainly has its merits. After all, the process of bringing a physical map closer to one's eyes to get a better view is a natural, timeless user interaction, and maps like this one, which Boyer produced during his time at The Tribune, are far more intuitive in communicating information upon first glance than many of the infoWindow-laden Google maps being produced by news organizations these days, many of them simply for the sake of being called 'interactive' (for those of you fortunate enough not to be mapping nerds, infoWindow is just Google-speak for the clickable popup boxes you see in Google maps).

But Boyer's minimalist mapping aesthetic only really works when you have one or two pieces of textual data you want to display for each geographic area. What if you have multiple pieces of information you want to display for each polygon, such as in this snazzy map from The Texas Tribune? Or, less likely but equally problematic, what if you need to bind non-textual data to your geographic polygons, such as images or Google Charts? In cases such as these, you're going to need to provide some sort of interaction that allows the user to expand and collapse the data for each area individually, or you'll just end up with a chronic case of visual overload.

Not to mention, on a more abstract level, studies have repeatedly shown that users tend to spend more time on applications that provide direct feedback based upon their actions, even if that feedback sometimes makes their ability to consume information at first glance less efficient (see Donald Norman's 2005 book Emotional Design: Why We Love (Or Hate) Everyday Things, in which Norman asserts that the feeling of emotional satisfaction and empowerment users receive from triggering an action not only puts them in a clearer state of mind, but also makes them more engaged in the information at hand). So, if we're trying to communicate geographic data to users as effectively as possible, it only makes sense that we'd want to have a certain degree of user interaction –– both for the sake of preventing visual overload and for making users feel more engaged. Such is the logic behind clickable infoWindows.

Still, clickable popups leave us with another problem: Users have to make the conscious and deliberate effort to click a polygon to see the data for that geographic area. Requiring clicks may sound like a trivial task to the designer or journalist-programmer, but for the short-attention span user, it can be an awful lot to ask for. To be fair, however, click-triggered popups may not be much of a problem for maps with only a few dozen polygons. But for maps with hundreds of small polygons –– say, census tracts or zip codes –– it can be very tedious to click the right polygon without first having to zoom into the map so far that you lose sight of the broader context.

That leads me back to a conversation I had a couple of months ago with a friend of mine from Columbia's J-School, Michael Keller (@mhkeller), who's now working as the Senior Data Reporter at The Daily Beast. Michael insisted to me after a Hacks/Hacker event that providing hover interaction for maps is almost always a good thing, because hovers require less work on the part of the user. I'll admit that I was dubious at the time, thinking of hovers as often unwanted, accidental triggers that can be distracting to the map and data at hand. But lately I've come around to his way of seeing things. If implemented correctly (i.e. no flashy interactions that cover up other parts of the map), hovers are almost always a good idea. For example, this recent map of New York Stop-and-Frisk data that Keller produced for The New York World using CartoDB and Leaflet is so detailed that it couldn't possibly have worked without infoWindows, and would have been unwieldy if it relied on click-triggered interaction. By including floating mouseover capabilities, the map allows the user to scan quickly through the chloropleth map to see individual Stop-and-Frisk data from each block, without having to attempt to click through minute geographic areas.*

I'm certainly not advocating interaction for interaction's sake (although such a case could be made, given the dynamics of the Web). But I am saying that hovers give more immediate visual feedback than click-triggered events, especially in maps. Hovers help draw users into the data without requiring them to seek it out consciously–– almost like a catchy lede would in a print narrative. So for the time-being, I'm pro-hover.

*Keller later messaged me letting me know that some examples which better illustrate the power of hovers include this map and this map, both of which use hover functionality to help highlight the effects of proposed redistricting efforts in the New York State Senate. What you'll also notice about Keller's maps is that they include hover states, which I also think is a necessity, especially for maps that include lots of small polygons.

information design

Building a responsive site in less than 20 minutes

6 minute read

An ever-so-sleek responsive portfolio site I designed for a friend in less than 20 minutes using Skeleton as a foundation.[/caption]

With all this talk lately of the new era of responsive design, I realized today that I've yet to create anything that's actually responsive. Given that I've only pondered using it in the implementation of complex, database-driven news sites, the task of tweaking every level of CSS to fit perfectly into a responsive grid system has so far seemed too daunting to tackle.

But I got curious this afternoon, and found my way upon a cool new library called Skeleton, which bills itself as a "a beautiful boilerplate for responsive, mobile-friendly development." Essentially, Skeleton is a collection of CSS and JS files that makes the mystery of responsive design seem a little less illusive. Upon uploading the package to my server, I was pleased to find a neatly-coded, easy-to-understand responsive site that I could play around with and tweak to my own liking. I ended up adjusting the grid size and performing some minor customizations to the underlying Skeleton structural CSS, but other than that, the development kit was, as it promised, "style agnostic."

Since I didn't have any specific projects I needed to be working on today, I decided to give Skeleton a whirl by designing a new online portfolio as a favor for my friend Daniel Medina. I'm not kidding – within 10 minutes I'd coded a fully satisfactory responsive portfolio site that looks beautiful on my iPhone and tablet.

So, if you're feeling experimental, try Skeleton out. It's exciting to see how quickly this technology is catching steam, and to see the actual workings of the responsive CSS in action firsthand. My next goal is to integrate a responsive layout into a more-powerful database-driven site, perhaps one designed in Drupal.

Should data viz be a specialty or a commodity skill in the newsroom?

8 minute read

An interesting question came up at last Wednesday's Doing Data Journalism (#doingdataj) panel hosted by the Tow Center for Digital Journalism here at Columbia's J-School: Should there be data specialists in the newsroom, or can everyone be a data journalist? For New York Times interactive editor Aron Pilholfer, who participated in the panel, the question is not so much should everyone do data as will everyone do data. And for Pilholfer, the answer to that question clearly seems to be no:

I kind of naively thought that at one time you could train everybody to be at least a base level of competency with something like Excel, but I'm not of that belief anymore. I think you do need specialists.

I've always hated the idea of having technology or innovation 'specialists' in a work environment that should ideally be collaborative. So, at first I tended to disagree with Pilholfer's argument. But what won me over was the reasoning behind his claim. For Pilholfer, it's not that the technology, human talent or open source tools aren't there for everyone to scrape, analyze and process data –– in fact, it's now easier than ever to organize messy data with simple and often free desktop applications like Excel and Google Refine. The problem is that there's a cultural lack of interest within newsrooms, often from an editorial level, to produce data-driven stories. As Pilholfer says in what appears to be an indictment of upper-level editors for disregarding the value of data,

The problem is that we continue to reward crap journalism that's based on anecdotal evidence alone . . . But truly if it's not a priority at the top to reward good data-driven journalism, it's going to be impossible to get people into data because they just don't think it's worth it.

I totally agree, but with one lurking suspicion. As with the top-level editors, many traditional users –– or 'readers,' as one might call them –– still at least think they like to read pretty, anecdotal narratives, and tend not to care as much whether the hard data backs them up. In other words, it’s an audience problem just as much as it is a managerial or institutional one. Some legacy news consumers just still aren't data literate. Because they're not accustomed to even having such data freely available to them, they don't even value having it. As the old saying goes, "You can't miss what you never had." Yet as traffic and engagement statistics continually confirm, as soon users have open data readily available to them through news apps and data visualizations, they spend more time accessing the data than they do reading the print narrative.

interactive

interactive design

What makes BostonGlobe.com “the world’s best designed website”

16 minute read

With the Pulitzer Price announcements coming up later this afternoon, you'd think I'd be writing about whose up for the "Best Deadline Reporting" or "Best Public Service Journalism" prizes. But instead I want to talk about a different media award doled out during the past week: BostonGlobe.com's designation as the "world's best designed website" by the Society for News Design. Put simply, I can't say I disagree.

Yet before I divulge in my effusive praise of the folks in Boston, let me point out that I'm still somewhat skeptical of the business-logic behind The Globe's decision to launch its paywall-only site last fall alongside its primary news portal, Boston.com. From a revenue standpoint, I can't see the full-paywall site, BostonGlobe.com, bringing in nearly enough subscription income to compensate for the traffic and ad dollars it will, and to some degree already has, leeched from Boston.com (at last count, in February, BostonGlobe.com had only recruited about 16,000 paid digital subscribers, many of whom had taken advantage of the site's introductory offer of 99 cents for the first four weeks but may not stick it out past the trial period). It might've been wiser for the paper to start out with a metered paywall to warm users to the idea of paying for content before erecting a full-blown ten-foot-tall paywall around its most valuable content under a new, separate domain name. But then again, who really knows – maybe it's a step in the right direction longterm? I'm not the one having to make the tough calls, so I'm in no position to judge.

At any rate, the segregation of the two sites has given the paper's parent company the freedom to make BostonGlobe.com a rich, immersive and interactive user experience that few other news organizations can match. Why, you ask? Because of a little trend gaining steam in the development world called "responsive design." See, BostonGlobe.com isn't only a standout site because of its sophisticated use of white space, its wholehearted embrace of web fonts and its visual-first approach to story art, but also because of its cross-platform capabilities. No matter what device you view BostonGlobe.com on – desktop, tablet or mobile – the site retains its same slick look and feel. The site's adaptive technology allows it to detect the screen size of the user, then adjust its layout to fit the exact resolution at hand. On desktops, that means content stretches to fill the entire browser window, and the grid restructures itself appropriately when you decrease or increase the size of the window. This eliminates the need for time-honored design standards like the 960 grid-system, which is based upon the Desktop-centric idea that all users have at least a 960px resolution. Now, the grid can be as big, or as small, as the user wants it to be.

The site's successful display of a dynamic range of real-time content on any-sized device also essentially eliminates the need for the "app-based" environment that's been the staple of the iOS5 and Android operating systems in recent years. To test it out, I gave the site a whirl on my iPad, and was pleased to see the front page neatly rearrange itself to fit the new orientation, just as any of the best platform-specific news apps out there do, including The New Yorker and Wired apps for iPad. But just because BostonGlobe.com doesn't need a native app to display its content beautifully on tablet and mobile devices doesn't mean that it won't be missing out on the growing app-centric marketplace for publishers. As Apple and Google continue to centralize digital consumption patterns into the app-based model, BostonGlobe.com may be a little too ambitious in thinking it doesn't need to play nice with the big-boy tech companies to reach its audience. On the other hand, however, it could prove to be a brilliant move for the paper, setting an example for other publishers and app-based companies in general who want to break free from the often constricting, not to mention pricey, cost of participating in the app marketplace.

For its design alone, BostonGlobe.com deserves any number of awards. Its bold, minimalist interface allows content to shine above all else, free from clutter and distraction. And with high-res horizontal photos, block-quotes, inline video and stylized headlines that grab your attention without hurting your eyes, that content stands out even more. What's more, the site has the functionality to back up its aesthetics. Its "Save" feature allows users to bookmark articles for reading offline on any device, which, even for sites without responsive design, is a brilliant feature for a news site to implement. Moreover, its "Story Flow" panel at the bottom of each article allows users to click seamlessly through to stories on similar subjects, just as readers would in an old-fashioned newspaper with physical sections. Which leaves me with one nagging question: Is BostonGlobe.com too imitative of physical newspapers to attain success in today's short attention-span digital audience? Is it too skewmorphic to succeed in an SEO-driven, hyperlinked news economy?

I guess we'll have to wait and see.

interactive-data-visualization

interactive-vs-static

mapbox

Making the case for hover interactions in maps

19 minute read

In keeping with my recent spate of mapping nerdiness, I decided to take an interactive map I produced last month displaying statewide annual population changes a step further by adding mouseover/hover capabilities. Here's the hover-y, nicely-colored chloropleth map I came up with. But before I get into the nitty-gritty of how I created the map –– which I'll explain step-by-step in a later post –– let me exercise a bit of self-indulgence by defending my growing belief in the need for hover capabilities when visualizing geographic data.

Not too long ago, I was an avid believer in the no-frills, less-interactivity-is-more approach to mapping geographic data, espoused by the brilliant Brian Boyer (@brianboyer), News Applications Editor for NPR and a former member of the News Apps Team at The Chicago Tribune. Boyer's argument for the need to keep maps simple –– like they used to be back in the days of ink and paper –– certainly has its merits. After all, the process of bringing a physical map closer to one's eyes to get a better view is a natural, timeless user interaction, and maps like this one, which Boyer produced during his time at The Tribune, are far more intuitive in communicating information upon first glance than many of the infoWindow-laden Google maps being produced by news organizations these days, many of them simply for the sake of being called 'interactive' (for those of you fortunate enough not to be mapping nerds, infoWindow is just Google-speak for the clickable popup boxes you see in Google maps).

But Boyer's minimalist mapping aesthetic only really works when you have one or two pieces of textual data you want to display for each geographic area. What if you have multiple pieces of information you want to display for each polygon, such as in this snazzy map from The Texas Tribune? Or, less likely but equally problematic, what if you need to bind non-textual data to your geographic polygons, such as images or Google Charts? In cases such as these, you're going to need to provide some sort of interaction that allows the user to expand and collapse the data for each area individually, or you'll just end up with a chronic case of visual overload.

Not to mention, on a more abstract level, studies have repeatedly shown that users tend to spend more time on applications that provide direct feedback based upon their actions, even if that feedback sometimes makes their ability to consume information at first glance less efficient (see Donald Norman's 2005 book Emotional Design: Why We Love (Or Hate) Everyday Things, in which Norman asserts that the feeling of emotional satisfaction and empowerment users receive from triggering an action not only puts them in a clearer state of mind, but also makes them more engaged in the information at hand). So, if we're trying to communicate geographic data to users as effectively as possible, it only makes sense that we'd want to have a certain degree of user interaction –– both for the sake of preventing visual overload and for making users feel more engaged. Such is the logic behind clickable infoWindows.

Still, clickable popups leave us with another problem: Users have to make the conscious and deliberate effort to click a polygon to see the data for that geographic area. Requiring clicks may sound like a trivial task to the designer or journalist-programmer, but for the short-attention span user, it can be an awful lot to ask for. To be fair, however, click-triggered popups may not be much of a problem for maps with only a few dozen polygons. But for maps with hundreds of small polygons –– say, census tracts or zip codes –– it can be very tedious to click the right polygon without first having to zoom into the map so far that you lose sight of the broader context.

That leads me back to a conversation I had a couple of months ago with a friend of mine from Columbia's J-School, Michael Keller (@mhkeller), who's now working as the Senior Data Reporter at The Daily Beast. Michael insisted to me after a Hacks/Hacker event that providing hover interaction for maps is almost always a good thing, because hovers require less work on the part of the user. I'll admit that I was dubious at the time, thinking of hovers as often unwanted, accidental triggers that can be distracting to the map and data at hand. But lately I've come around to his way of seeing things. If implemented correctly (i.e. no flashy interactions that cover up other parts of the map), hovers are almost always a good idea. For example, this recent map of New York Stop-and-Frisk data that Keller produced for The New York World using CartoDB and Leaflet is so detailed that it couldn't possibly have worked without infoWindows, and would have been unwieldy if it relied on click-triggered interaction. By including floating mouseover capabilities, the map allows the user to scan quickly through the chloropleth map to see individual Stop-and-Frisk data from each block, without having to attempt to click through minute geographic areas.*

I'm certainly not advocating interaction for interaction's sake (although such a case could be made, given the dynamics of the Web). But I am saying that hovers give more immediate visual feedback than click-triggered events, especially in maps. Hovers help draw users into the data without requiring them to seek it out consciously–– almost like a catchy lede would in a print narrative. So for the time-being, I'm pro-hover.

*Keller later messaged me letting me know that some examples which better illustrate the power of hovers include this map and this map, both of which use hover functionality to help highlight the effects of proposed redistricting efforts in the New York State Senate. What you'll also notice about Keller's maps is that they include hover states, which I also think is a necessity, especially for maps that include lots of small polygons.

mapping

Making the case for hover interactions in maps

19 minute read

In keeping with my recent spate of mapping nerdiness, I decided to take an interactive map I produced last month displaying statewide annual population changes a step further by adding mouseover/hover capabilities. Here's the hover-y, nicely-colored chloropleth map I came up with. But before I get into the nitty-gritty of how I created the map –– which I'll explain step-by-step in a later post –– let me exercise a bit of self-indulgence by defending my growing belief in the need for hover capabilities when visualizing geographic data.

Not too long ago, I was an avid believer in the no-frills, less-interactivity-is-more approach to mapping geographic data, espoused by the brilliant Brian Boyer (@brianboyer), News Applications Editor for NPR and a former member of the News Apps Team at The Chicago Tribune. Boyer's argument for the need to keep maps simple –– like they used to be back in the days of ink and paper –– certainly has its merits. After all, the process of bringing a physical map closer to one's eyes to get a better view is a natural, timeless user interaction, and maps like this one, which Boyer produced during his time at The Tribune, are far more intuitive in communicating information upon first glance than many of the infoWindow-laden Google maps being produced by news organizations these days, many of them simply for the sake of being called 'interactive' (for those of you fortunate enough not to be mapping nerds, infoWindow is just Google-speak for the clickable popup boxes you see in Google maps).

But Boyer's minimalist mapping aesthetic only really works when you have one or two pieces of textual data you want to display for each geographic area. What if you have multiple pieces of information you want to display for each polygon, such as in this snazzy map from The Texas Tribune? Or, less likely but equally problematic, what if you need to bind non-textual data to your geographic polygons, such as images or Google Charts? In cases such as these, you're going to need to provide some sort of interaction that allows the user to expand and collapse the data for each area individually, or you'll just end up with a chronic case of visual overload.

Not to mention, on a more abstract level, studies have repeatedly shown that users tend to spend more time on applications that provide direct feedback based upon their actions, even if that feedback sometimes makes their ability to consume information at first glance less efficient (see Donald Norman's 2005 book Emotional Design: Why We Love (Or Hate) Everyday Things, in which Norman asserts that the feeling of emotional satisfaction and empowerment users receive from triggering an action not only puts them in a clearer state of mind, but also makes them more engaged in the information at hand). So, if we're trying to communicate geographic data to users as effectively as possible, it only makes sense that we'd want to have a certain degree of user interaction –– both for the sake of preventing visual overload and for making users feel more engaged. Such is the logic behind clickable infoWindows.

Still, clickable popups leave us with another problem: Users have to make the conscious and deliberate effort to click a polygon to see the data for that geographic area. Requiring clicks may sound like a trivial task to the designer or journalist-programmer, but for the short-attention span user, it can be an awful lot to ask for. To be fair, however, click-triggered popups may not be much of a problem for maps with only a few dozen polygons. But for maps with hundreds of small polygons –– say, census tracts or zip codes –– it can be very tedious to click the right polygon without first having to zoom into the map so far that you lose sight of the broader context.

That leads me back to a conversation I had a couple of months ago with a friend of mine from Columbia's J-School, Michael Keller (@mhkeller), who's now working as the Senior Data Reporter at The Daily Beast. Michael insisted to me after a Hacks/Hacker event that providing hover interaction for maps is almost always a good thing, because hovers require less work on the part of the user. I'll admit that I was dubious at the time, thinking of hovers as often unwanted, accidental triggers that can be distracting to the map and data at hand. But lately I've come around to his way of seeing things. If implemented correctly (i.e. no flashy interactions that cover up other parts of the map), hovers are almost always a good idea. For example, this recent map of New York Stop-and-Frisk data that Keller produced for The New York World using CartoDB and Leaflet is so detailed that it couldn't possibly have worked without infoWindows, and would have been unwieldy if it relied on click-triggered interaction. By including floating mouseover capabilities, the map allows the user to scan quickly through the chloropleth map to see individual Stop-and-Frisk data from each block, without having to attempt to click through minute geographic areas.*

I'm certainly not advocating interaction for interaction's sake (although such a case could be made, given the dynamics of the Web). But I am saying that hovers give more immediate visual feedback than click-triggered events, especially in maps. Hovers help draw users into the data without requiring them to seek it out consciously–– almost like a catchy lede would in a print narrative. So for the time-being, I'm pro-hover.

*Keller later messaged me letting me know that some examples which better illustrate the power of hovers include this map and this map, both of which use hover functionality to help highlight the effects of proposed redistricting efforts in the New York State Senate. What you'll also notice about Keller's maps is that they include hover states, which I also think is a necessity, especially for maps that include lots of small polygons.

Overlaying a bubble chart onto a Google map

25 minute read

Others may hate, but I'm a big fan of using bubbles to display data. When implemented correctly (i.e. scaled in terms of area instead of diameter), bubbles can be an aesthetically appealing and concise way to represent the value of data points in an inherently visual format. Bubbles are even more useful when they include interactivity, with events like mouseover and zoom allowing users to drill down and compare similar-sized bubbles more easily than they can in static graphics. So, when I was recently working on a class project on autism diagnoses in New York City, I decided to use bubbles to represent the percentage of students with individualized education plans at all 1250 or so K-8 New York City schools.

Almost by default, I turned to Google Maps JavaScript API V3, mainly because I'm quasi-familiar with its basic functions and event handlers (as I point out later in  this post, I didn't realize that a nifty new service called CartoDB would have automated most of what I was trying to do, albeit without nearly the level of customization). Nonetheless, based on a tutorial from Karl Agius, as well as some infoWindow help from my data viz professor, Susan McGregor, I created the following interactive bubble map of NYC schools based upon the number of special needs, or IEP, students at each school. The larger the bubble, the greater percentage of special needs students a school has. Click here to see the map full-screen, or here to download a .zip of my source files for your own customization purposes.

Each bubble on this map represents one of New York City's approximately 1,250 K-8 public schools, including charters. The larger the bubble, the higher the percentage of students with individualized education plans (IEP). Click on a bubble to find out more about the school, or click anywhere within a district boundary to see an overall average IEP rate for the district. Zoom and pan to see other parts of the city.

You'll notice the opacity for the bubbles is set to 40 percent. This allows us to get a quick visual of the locations with the highest density of special needs students, given that those areas on the map will naturally be darker because they have multiple semi-opaque layers that overlap. Setting a low opacity also prevents overlapping bubbles from covering up one another. You'll also notice that the map includes polygons for each school district, which you can click on to get an average IEP rate for the entire district. I decided against setting gradated fill colors for the school district shapes so as to avoid implying causation, as well as to lessen the visual clutter.*

Preparing the data

To create the map, I first had to download the underlying data from the New York City Department of Education database as a .csv, then import it into Excel to clean it up and leave only the relevant information. Although the dataset I had only included street addresses split into multiple columns, I was able to use the concatenate function in Excel to merge the street, city, state and zip columns to get a full street address. From there, I used my favorite batch geocoding service to convert the addresses into geographic coordinates that the Google Maps API can read. Check out my resulting .csv file here for an example. Then I imported the .csv into a Google Spreadsheet, and pasted the resulting spreadsheet's URL into dataSourceURL field in the JavaScript of my main index.html file. Here's how that looked in my code:

var dataSourceUrl = "https://docs.google.com/spreadsheet/ccc?key=0Au4D8Alccn4xdHNJdXJmeTdYcEtpRXE1QXRucWtEN3c";
var onlyInfoWindow;

media

“Newspapers are the new startups”

1 minute read

Newspapers are the new startups . . . we’re starting to see a lot of great changes as technologies improve and cultures change."
-John Levitt, Director of Sales and Marketing,  Parse.ly
</p>

Levitt's is one of the most insightful takes on the publishing industry I've heard in a while. It's going to take a lot of restructuring and a ground-up approach, but I'm excited to be a part of it as we embrace the start-up culture in Savannah.

media business

mobile

Building a responsive site in less than 20 minutes

6 minute read

An ever-so-sleek responsive portfolio site I designed for a friend in less than 20 minutes using Skeleton as a foundation.[/caption]

With all this talk lately of the new era of responsive design, I realized today that I've yet to create anything that's actually responsive. Given that I've only pondered using it in the implementation of complex, database-driven news sites, the task of tweaking every level of CSS to fit perfectly into a responsive grid system has so far seemed too daunting to tackle.

But I got curious this afternoon, and found my way upon a cool new library called Skeleton, which bills itself as a "a beautiful boilerplate for responsive, mobile-friendly development." Essentially, Skeleton is a collection of CSS and JS files that makes the mystery of responsive design seem a little less illusive. Upon uploading the package to my server, I was pleased to find a neatly-coded, easy-to-understand responsive site that I could play around with and tweak to my own liking. I ended up adjusting the grid size and performing some minor customizations to the underlying Skeleton structural CSS, but other than that, the development kit was, as it promised, "style agnostic."

Since I didn't have any specific projects I needed to be working on today, I decided to give Skeleton a whirl by designing a new online portfolio as a favor for my friend Daniel Medina. I'm not kidding – within 10 minutes I'd coded a fully satisfactory responsive portfolio site that looks beautiful on my iPhone and tablet.

So, if you're feeling experimental, try Skeleton out. It's exciting to see how quickly this technology is catching steam, and to see the actual workings of the responsive CSS in action firsthand. My next goal is to integrate a responsive layout into a more-powerful database-driven site, perhaps one designed in Drupal.

mouseover

Making the case for hover interactions in maps

19 minute read

In keeping with my recent spate of mapping nerdiness, I decided to take an interactive map I produced last month displaying statewide annual population changes a step further by adding mouseover/hover capabilities. Here's the hover-y, nicely-colored chloropleth map I came up with. But before I get into the nitty-gritty of how I created the map –– which I'll explain step-by-step in a later post –– let me exercise a bit of self-indulgence by defending my growing belief in the need for hover capabilities when visualizing geographic data.

Not too long ago, I was an avid believer in the no-frills, less-interactivity-is-more approach to mapping geographic data, espoused by the brilliant Brian Boyer (@brianboyer), News Applications Editor for NPR and a former member of the News Apps Team at The Chicago Tribune. Boyer's argument for the need to keep maps simple –– like they used to be back in the days of ink and paper –– certainly has its merits. After all, the process of bringing a physical map closer to one's eyes to get a better view is a natural, timeless user interaction, and maps like this one, which Boyer produced during his time at The Tribune, are far more intuitive in communicating information upon first glance than many of the infoWindow-laden Google maps being produced by news organizations these days, many of them simply for the sake of being called 'interactive' (for those of you fortunate enough not to be mapping nerds, infoWindow is just Google-speak for the clickable popup boxes you see in Google maps).

But Boyer's minimalist mapping aesthetic only really works when you have one or two pieces of textual data you want to display for each geographic area. What if you have multiple pieces of information you want to display for each polygon, such as in this snazzy map from The Texas Tribune? Or, less likely but equally problematic, what if you need to bind non-textual data to your geographic polygons, such as images or Google Charts? In cases such as these, you're going to need to provide some sort of interaction that allows the user to expand and collapse the data for each area individually, or you'll just end up with a chronic case of visual overload.

Not to mention, on a more abstract level, studies have repeatedly shown that users tend to spend more time on applications that provide direct feedback based upon their actions, even if that feedback sometimes makes their ability to consume information at first glance less efficient (see Donald Norman's 2005 book Emotional Design: Why We Love (Or Hate) Everyday Things, in which Norman asserts that the feeling of emotional satisfaction and empowerment users receive from triggering an action not only puts them in a clearer state of mind, but also makes them more engaged in the information at hand). So, if we're trying to communicate geographic data to users as effectively as possible, it only makes sense that we'd want to have a certain degree of user interaction –– both for the sake of preventing visual overload and for making users feel more engaged. Such is the logic behind clickable infoWindows.

Still, clickable popups leave us with another problem: Users have to make the conscious and deliberate effort to click a polygon to see the data for that geographic area. Requiring clicks may sound like a trivial task to the designer or journalist-programmer, but for the short-attention span user, it can be an awful lot to ask for. To be fair, however, click-triggered popups may not be much of a problem for maps with only a few dozen polygons. But for maps with hundreds of small polygons –– say, census tracts or zip codes –– it can be very tedious to click the right polygon without first having to zoom into the map so far that you lose sight of the broader context.

That leads me back to a conversation I had a couple of months ago with a friend of mine from Columbia's J-School, Michael Keller (@mhkeller), who's now working as the Senior Data Reporter at The Daily Beast. Michael insisted to me after a Hacks/Hacker event that providing hover interaction for maps is almost always a good thing, because hovers require less work on the part of the user. I'll admit that I was dubious at the time, thinking of hovers as often unwanted, accidental triggers that can be distracting to the map and data at hand. But lately I've come around to his way of seeing things. If implemented correctly (i.e. no flashy interactions that cover up other parts of the map), hovers are almost always a good idea. For example, this recent map of New York Stop-and-Frisk data that Keller produced for The New York World using CartoDB and Leaflet is so detailed that it couldn't possibly have worked without infoWindows, and would have been unwieldy if it relied on click-triggered interaction. By including floating mouseover capabilities, the map allows the user to scan quickly through the chloropleth map to see individual Stop-and-Frisk data from each block, without having to attempt to click through minute geographic areas.*

I'm certainly not advocating interaction for interaction's sake (although such a case could be made, given the dynamics of the Web). But I am saying that hovers give more immediate visual feedback than click-triggered events, especially in maps. Hovers help draw users into the data without requiring them to seek it out consciously–– almost like a catchy lede would in a print narrative. So for the time-being, I'm pro-hover.

*Keller later messaged me letting me know that some examples which better illustrate the power of hovers include this map and this map, both of which use hover functionality to help highlight the effects of proposed redistricting efforts in the New York State Senate. What you'll also notice about Keller's maps is that they include hover states, which I also think is a necessity, especially for maps that include lots of small polygons.

news organizations

Why news organizations should stop differeniating blogs from articles

13 minute read

Andy Boyle (@andymboyle) of The Boston Globe made an┬áimpassioned┬áplea to news organizations earlier this week that they stop differentiating between blogs and articles because they’re both equally forms of content. Someone’s been needing to put this into writing for a while now, and I’m glad Andy said it so eloquently.

news strategies

opinion pages

politics

private-prisons

quotations

“Newspapers are the new startups”

1 minute read

Newspapers are the new startups . . . we’re starting to see a lot of great changes as technologies improve and cultures change."
-John Levitt, Director of Sales and Marketing,  Parse.ly
</p>

Levitt's is one of the most insightful takes on the publishing industry I've heard in a while. It's going to take a lot of restructuring and a ground-up approach, but I'm excited to be a part of it as we embrace the start-up culture in Savannah.

re-escaping-the-flatlands

responsive design

Building a responsive site in less than 20 minutes

6 minute read

An ever-so-sleek responsive portfolio site I designed for a friend in less than 20 minutes using Skeleton as a foundation.[/caption]

With all this talk lately of the new era of responsive design, I realized today that I've yet to create anything that's actually responsive. Given that I've only pondered using it in the implementation of complex, database-driven news sites, the task of tweaking every level of CSS to fit perfectly into a responsive grid system has so far seemed too daunting to tackle.

But I got curious this afternoon, and found my way upon a cool new library called Skeleton, which bills itself as a "a beautiful boilerplate for responsive, mobile-friendly development." Essentially, Skeleton is a collection of CSS and JS files that makes the mystery of responsive design seem a little less illusive. Upon uploading the package to my server, I was pleased to find a neatly-coded, easy-to-understand responsive site that I could play around with and tweak to my own liking. I ended up adjusting the grid size and performing some minor customizations to the underlying Skeleton structural CSS, but other than that, the development kit was, as it promised, "style agnostic."

Since I didn't have any specific projects I needed to be working on today, I decided to give Skeleton a whirl by designing a new online portfolio as a favor for my friend Daniel Medina. I'm not kidding – within 10 minutes I'd coded a fully satisfactory responsive portfolio site that looks beautiful on my iPhone and tablet.

So, if you're feeling experimental, try Skeleton out. It's exciting to see how quickly this technology is catching steam, and to see the actual workings of the responsive CSS in action firsthand. My next goal is to integrate a responsive layout into a more-powerful database-driven site, perhaps one designed in Drupal.

What makes BostonGlobe.com “the world’s best designed website”

16 minute read

With the Pulitzer Price announcements coming up later this afternoon, you'd think I'd be writing about whose up for the "Best Deadline Reporting" or "Best Public Service Journalism" prizes. But instead I want to talk about a different media award doled out during the past week: BostonGlobe.com's designation as the "world's best designed website" by the Society for News Design. Put simply, I can't say I disagree.

Yet before I divulge in my effusive praise of the folks in Boston, let me point out that I'm still somewhat skeptical of the business-logic behind The Globe's decision to launch its paywall-only site last fall alongside its primary news portal, Boston.com. From a revenue standpoint, I can't see the full-paywall site, BostonGlobe.com, bringing in nearly enough subscription income to compensate for the traffic and ad dollars it will, and to some degree already has, leeched from Boston.com (at last count, in February, BostonGlobe.com had only recruited about 16,000 paid digital subscribers, many of whom had taken advantage of the site's introductory offer of 99 cents for the first four weeks but may not stick it out past the trial period). It might've been wiser for the paper to start out with a metered paywall to warm users to the idea of paying for content before erecting a full-blown ten-foot-tall paywall around its most valuable content under a new, separate domain name. But then again, who really knows – maybe it's a step in the right direction longterm? I'm not the one having to make the tough calls, so I'm in no position to judge.

At any rate, the segregation of the two sites has given the paper's parent company the freedom to make BostonGlobe.com a rich, immersive and interactive user experience that few other news organizations can match. Why, you ask? Because of a little trend gaining steam in the development world called "responsive design." See, BostonGlobe.com isn't only a standout site because of its sophisticated use of white space, its wholehearted embrace of web fonts and its visual-first approach to story art, but also because of its cross-platform capabilities. No matter what device you view BostonGlobe.com on – desktop, tablet or mobile – the site retains its same slick look and feel. The site's adaptive technology allows it to detect the screen size of the user, then adjust its layout to fit the exact resolution at hand. On desktops, that means content stretches to fill the entire browser window, and the grid restructures itself appropriately when you decrease or increase the size of the window. This eliminates the need for time-honored design standards like the 960 grid-system, which is based upon the Desktop-centric idea that all users have at least a 960px resolution. Now, the grid can be as big, or as small, as the user wants it to be.

The site's successful display of a dynamic range of real-time content on any-sized device also essentially eliminates the need for the "app-based" environment that's been the staple of the iOS5 and Android operating systems in recent years. To test it out, I gave the site a whirl on my iPad, and was pleased to see the front page neatly rearrange itself to fit the new orientation, just as any of the best platform-specific news apps out there do, including The New Yorker and Wired apps for iPad. But just because BostonGlobe.com doesn't need a native app to display its content beautifully on tablet and mobile devices doesn't mean that it won't be missing out on the growing app-centric marketplace for publishers. As Apple and Google continue to centralize digital consumption patterns into the app-based model, BostonGlobe.com may be a little too ambitious in thinking it doesn't need to play nice with the big-boy tech companies to reach its audience. On the other hand, however, it could prove to be a brilliant move for the paper, setting an example for other publishers and app-based companies in general who want to break free from the often constricting, not to mention pricey, cost of participating in the app marketplace.

For its design alone, BostonGlobe.com deserves any number of awards. Its bold, minimalist interface allows content to shine above all else, free from clutter and distraction. And with high-res horizontal photos, block-quotes, inline video and stylized headlines that grab your attention without hurting your eyes, that content stands out even more. What's more, the site has the functionality to back up its aesthetics. Its "Save" feature allows users to bookmark articles for reading offline on any device, which, even for sites without responsive design, is a brilliant feature for a news site to implement. Moreover, its "Story Flow" panel at the bottom of each article allows users to click seamlessly through to stories on similar subjects, just as readers would in an old-fashioned newspaper with physical sections. Which leaves me with one nagging question: Is BostonGlobe.com too imitative of physical newspapers to attain success in today's short attention-span digital audience? Is it too skewmorphic to succeed in an SEO-driven, hyperlinked news economy?

I guess we'll have to wait and see.

standard deviation

Why calculus matters when it comes to data-driven stories

7 minute read

A quick refresher from my data visualization professor here at Columbia a couple of weeks ago reminded me why I was forced to spend all those grueling hours calculating standard deviation back in high school.

See, when you're using a data set to tell a story, the first step is to understand what that data says. And to do that, you've got to have a good idea of the range and variation of the values at hand. Not only can figuring that information out help you determine whether there's any statistical significance to your data set, but it can also pinpoint outliers and possible errors that may exist within the data before you begin the work of visualizing it.

Thanks to powerful processing programs like Excel, we can figure out the variability of data sets pretty easily using the program's built-in standard deviation function (remember this intimidating-looking equation from calculus class?). Still, it always helps to know how to calculate the information out by hand, if only to get a conceptual idea of why numbers such as the standard deviation (the variability of a data-set) and the z-value (the number of standard deviations a given value is away from the mean) even matter in the first place when it comes to data visualization.

So, to brush up on my formulas and also better understand the numbers behind an actual story assignment for one of my classes, I recently hand-calculated the standard deviation and z-values for a set of data on state-by-state obesity rates. From my calculations, I was able to use the standard deviation (3.24) to determine that, on average, most states fell within the middle of the bell-curve for the average national obesity rate (27.1 percent) . In addition, the z-values helped me understand which states stood out from the pack as possible outliers (Mississippi is by far the most obese with a 2.13 z-value, Colorado the least obese with a -1.9 z-value). To get an idea of how those formulas look hand-calculated in Excel, check out my spreadsheet here. And keep these formulas in mind while working on your next data story. They can potentially save you time and effort by helping you figure out what your data set says before you have to go through the often-lengthy process of visualizing it.

the south

INTERACTIVE: Why is the South the most obese part of the country? Five theories

17 minute read

This map displays the obesity rate of each U.S. state in 2010. The darker shade red represents a higher percentage of obese residents, while the green represents states with lower obesity rates. Click on each state to see the exact totals of each state's obesity rate.* 

Southerners need to lay off of the Crisco, cut back on the processed foods and start spending more time on the treadmill to fight the growing epidemic of obesity, experts say.

According to 2010 data released by the Centers for Disease Control and Prevention, the South is the most obese region in the nation, with about one in three of its residents classifying as chronically obese. That's far greater than the entire nation, where the figure is closer to one in four.

Of the 10 states with the highest rates of adult obesity, eight of them are in the South: Mississippi, Alabama, South Carolina, Kentucky, Louisiana, Texas and Tennessee. And that's only assuming you don't count West Virginia as being "southern."

Across the nation, the epidemic has grown worse in recent years. Twelve states now have obesity rates higher than 30 percent, compared to four years ago when only one state, Mississippi, ranked above the 30 percent threshold. The only state in the Deep South without an obesity rate of more than 30 percent today is Georgia, but that appears to be primarily because the more physically fit population of metro Atlanta offsets the rest of the state's obesity.

But what's making the South –– the region CDC Dr. William Dietz has dubbed "the heart disease and stroke belt" –– more chubby than the rest of the nation? Here are five possible explanations:

1. High poverty -– The South may be obese, but it's also poor. With a poverty rate of 14 percent, the South is easily the most impoverished region in the country. And according to data from the USDA, states with a higher poverty rate also tend to have a higher number of obese citizens. Experts say that's because people with a low income are more likely to purchase high-calorie inexpensive processed foods, which contribute to weight gain. "If you overlay a map of obesity onto a map of poverty, the two very clearly correspond," said David A. Davis, a professor of Southern Studies at Mercer University who has conducted extensive research on southern foodways. "The southern diet is a diet of poverty, and it's one based on cheap, fatty processed foods."

2. The "grocery gap" – Because the South is largely rural, many residents don't have quality access to fresh fruits and vegetables, and are forced to drive long distances to find anything healthier than potato chips and sodas at roadside gas stations. All five states with above average of what the USDA calls "food insecurity" levels are located in the South: Texas, Arkansas, Mississippi, Georgia and North Carolina. What's more, it's significantly more expensive to purchase low-fat items in the South than in the rest of the nation. For example, Georgia, Mississippi, North Carolina and Virginia all topped the list of states where it costs the most to buy low-fat milk, USDA data says.

3. The grease-fed "southern" culinary tradition -– One of the easiest explanations for the South's staggering obesity rates is the region's tradition of fried chicken, sweet tea and gravy on top of everything – or what's commonly referred to by non-southerners as the "Paula Deen" effect. "To me, it's simply a cultural habit regarding what we eat, not an issue of poverty," says Andy Breck, director of the Center for a Better South, a nonprofit group based out of the University of South Carolina that seeks to raise awareness about ongoing issues facing the region. "People are fat in Mississippi. People are fat in South Carolina. People are fat in Alabama. There's got to be something going on. And it's not just poor people. It's middle and upper-class folks who grotesquely overeat, because that's all they've ever known to do."

4. Lack of physical activity–– Southerners also tend to be less physically active than the rest of the country, burning off fewer calories and retaining more body fat, USDA data says. All five U.S. states where less than 60 percent of adults met the USDA's recommended physical activity guidelines in 2008 were located below the Mason Dixon Line: Louisiana, Mississippi, Alabama, Tennessee and Kentucky. Some researchers have speculated that the South's lack of physical activity may not be so much sheer laziness as it is a lack of access to places to exercise. Few rural areas have fancy private gyms for southerners to burn off their extra calories, and most of the year it's just too plain hot in the South to exercise outdoors.

5. Lack of quality education – Perhaps at the heart of the southern obesity epidemic, however, is the region's crippling lack of quality public education. "I don't buy the fact that the South is fat because of traditional southern foodways," said Davis, who teaches classes on southern poverty and culture and has written numerous articles on the subject. "To me, it's more of a poverty and an educational problem. If we don't educate people, especially in terms of health education, we're going to keep having obese citizens."

*Source: The Centers for Disease Control and Prevention, April 3, 2012

tilemill

Making the case for hover interactions in maps

19 minute read

In keeping with my recent spate of mapping nerdiness, I decided to take an interactive map I produced last month displaying statewide annual population changes a step further by adding mouseover/hover capabilities. Here's the hover-y, nicely-colored chloropleth map I came up with. But before I get into the nitty-gritty of how I created the map –– which I'll explain step-by-step in a later post –– let me exercise a bit of self-indulgence by defending my growing belief in the need for hover capabilities when visualizing geographic data.

Not too long ago, I was an avid believer in the no-frills, less-interactivity-is-more approach to mapping geographic data, espoused by the brilliant Brian Boyer (@brianboyer), News Applications Editor for NPR and a former member of the News Apps Team at The Chicago Tribune. Boyer's argument for the need to keep maps simple –– like they used to be back in the days of ink and paper –– certainly has its merits. After all, the process of bringing a physical map closer to one's eyes to get a better view is a natural, timeless user interaction, and maps like this one, which Boyer produced during his time at The Tribune, are far more intuitive in communicating information upon first glance than many of the infoWindow-laden Google maps being produced by news organizations these days, many of them simply for the sake of being called 'interactive' (for those of you fortunate enough not to be mapping nerds, infoWindow is just Google-speak for the clickable popup boxes you see in Google maps).

But Boyer's minimalist mapping aesthetic only really works when you have one or two pieces of textual data you want to display for each geographic area. What if you have multiple pieces of information you want to display for each polygon, such as in this snazzy map from The Texas Tribune? Or, less likely but equally problematic, what if you need to bind non-textual data to your geographic polygons, such as images or Google Charts? In cases such as these, you're going to need to provide some sort of interaction that allows the user to expand and collapse the data for each area individually, or you'll just end up with a chronic case of visual overload.

Not to mention, on a more abstract level, studies have repeatedly shown that users tend to spend more time on applications that provide direct feedback based upon their actions, even if that feedback sometimes makes their ability to consume information at first glance less efficient (see Donald Norman's 2005 book Emotional Design: Why We Love (Or Hate) Everyday Things, in which Norman asserts that the feeling of emotional satisfaction and empowerment users receive from triggering an action not only puts them in a clearer state of mind, but also makes them more engaged in the information at hand). So, if we're trying to communicate geographic data to users as effectively as possible, it only makes sense that we'd want to have a certain degree of user interaction –– both for the sake of preventing visual overload and for making users feel more engaged. Such is the logic behind clickable infoWindows.

Still, clickable popups leave us with another problem: Users have to make the conscious and deliberate effort to click a polygon to see the data for that geographic area. Requiring clicks may sound like a trivial task to the designer or journalist-programmer, but for the short-attention span user, it can be an awful lot to ask for. To be fair, however, click-triggered popups may not be much of a problem for maps with only a few dozen polygons. But for maps with hundreds of small polygons –– say, census tracts or zip codes –– it can be very tedious to click the right polygon without first having to zoom into the map so far that you lose sight of the broader context.

That leads me back to a conversation I had a couple of months ago with a friend of mine from Columbia's J-School, Michael Keller (@mhkeller), who's now working as the Senior Data Reporter at The Daily Beast. Michael insisted to me after a Hacks/Hacker event that providing hover interaction for maps is almost always a good thing, because hovers require less work on the part of the user. I'll admit that I was dubious at the time, thinking of hovers as often unwanted, accidental triggers that can be distracting to the map and data at hand. But lately I've come around to his way of seeing things. If implemented correctly (i.e. no flashy interactions that cover up other parts of the map), hovers are almost always a good idea. For example, this recent map of New York Stop-and-Frisk data that Keller produced for The New York World using CartoDB and Leaflet is so detailed that it couldn't possibly have worked without infoWindows, and would have been unwieldy if it relied on click-triggered interaction. By including floating mouseover capabilities, the map allows the user to scan quickly through the chloropleth map to see individual Stop-and-Frisk data from each block, without having to attempt to click through minute geographic areas.*

I'm certainly not advocating interaction for interaction's sake (although such a case could be made, given the dynamics of the Web). But I am saying that hovers give more immediate visual feedback than click-triggered events, especially in maps. Hovers help draw users into the data without requiring them to seek it out consciously–– almost like a catchy lede would in a print narrative. So for the time-being, I'm pro-hover.

*Keller later messaged me letting me know that some examples which better illustrate the power of hovers include this map and this map, both of which use hover functionality to help highlight the effects of proposed redistricting efforts in the New York State Senate. What you'll also notice about Keller's maps is that they include hover states, which I also think is a necessity, especially for maps that include lots of small polygons.

unfeatured

visualization

Should data viz be a specialty or a commodity skill in the newsroom?

8 minute read

An interesting question came up at last Wednesday's Doing Data Journalism (#doingdataj) panel hosted by the Tow Center for Digital Journalism here at Columbia's J-School: Should there be data specialists in the newsroom, or can everyone be a data journalist? For New York Times interactive editor Aron Pilholfer, who participated in the panel, the question is not so much should everyone do data as will everyone do data. And for Pilholfer, the answer to that question clearly seems to be no:

I kind of naively thought that at one time you could train everybody to be at least a base level of competency with something like Excel, but I'm not of that belief anymore. I think you do need specialists.

I've always hated the idea of having technology or innovation 'specialists' in a work environment that should ideally be collaborative. So, at first I tended to disagree with Pilholfer's argument. But what won me over was the reasoning behind his claim. For Pilholfer, it's not that the technology, human talent or open source tools aren't there for everyone to scrape, analyze and process data –– in fact, it's now easier than ever to organize messy data with simple and often free desktop applications like Excel and Google Refine. The problem is that there's a cultural lack of interest within newsrooms, often from an editorial level, to produce data-driven stories. As Pilholfer says in what appears to be an indictment of upper-level editors for disregarding the value of data,

The problem is that we continue to reward crap journalism that's based on anecdotal evidence alone . . . But truly if it's not a priority at the top to reward good data-driven journalism, it's going to be impossible to get people into data because they just don't think it's worth it.

I totally agree, but with one lurking suspicion. As with the top-level editors, many traditional users –– or 'readers,' as one might call them –– still at least think they like to read pretty, anecdotal narratives, and tend not to care as much whether the hard data backs them up. In other words, it’s an audience problem just as much as it is a managerial or institutional one. Some legacy news consumers just still aren't data literate. Because they're not accustomed to even having such data freely available to them, they don't even value having it. As the old saying goes, "You can't miss what you never had." Yet as traffic and engagement statistics continually confirm, as soon users have open data readily available to them through news apps and data visualizations, they spend more time accessing the data than they do reading the print narrative.

z-value

Why calculus matters when it comes to data-driven stories

7 minute read

A quick refresher from my data visualization professor here at Columbia a couple of weeks ago reminded me why I was forced to spend all those grueling hours calculating standard deviation back in high school.

See, when you're using a data set to tell a story, the first step is to understand what that data says. And to do that, you've got to have a good idea of the range and variation of the values at hand. Not only can figuring that information out help you determine whether there's any statistical significance to your data set, but it can also pinpoint outliers and possible errors that may exist within the data before you begin the work of visualizing it.

Thanks to powerful processing programs like Excel, we can figure out the variability of data sets pretty easily using the program's built-in standard deviation function (remember this intimidating-looking equation from calculus class?). Still, it always helps to know how to calculate the information out by hand, if only to get a conceptual idea of why numbers such as the standard deviation (the variability of a data-set) and the z-value (the number of standard deviations a given value is away from the mean) even matter in the first place when it comes to data visualization.

So, to brush up on my formulas and also better understand the numbers behind an actual story assignment for one of my classes, I recently hand-calculated the standard deviation and z-values for a set of data on state-by-state obesity rates. From my calculations, I was able to use the standard deviation (3.24) to determine that, on average, most states fell within the middle of the bell-curve for the average national obesity rate (27.1 percent) . In addition, the z-values helped me understand which states stood out from the pack as possible outliers (Mississippi is by far the most obese with a 2.13 z-value, Colorado the least obese with a -1.9 z-value). To get an idea of how those formulas look hand-calculated in Excel, check out my spreadsheet here. And keep these formulas in mind while working on your next data story. They can potentially save you time and effort by helping you figure out what your data set says before you have to go through the often-lengthy process of visualizing it.