Building a Better Table of Contents

As I near the completion of my thesis, I find it challenging to navigate through 250 pages of prose and remember where certain arguments or certain pieces of evidence are located; or challenging to get a broad overview of the narrative sequence.

Search is only effective if I can recall the name of reference or if the search term is suitably unique. Unfortunately technical terms are frequently repeated – for example – preattentive occurs 46 times. Even using keyword in context (KWIC) in the search panel still provides an excessively long list and perhaps the associated words of interest aren’t within a few words of the search term. Furthermore, search doesn’t work for the broad overview.

A Table of Contents should help, but not as much as I had hoped. A few problems:

  • On a large document, the table of contents is spread out across many pages, requiring paging and requiring the use of short term memory to hold onto relevant headings.
  • Headings tend to be terse, so it may not be obvious what the content is associated with a heading.
  • Furthermore, headings reduce information down to a few words – the narrative stringing the pieces of the argument together are lost.
  • While there are different levels of headings, overall, the representation of a table of contents is fairly uniform and undifferentiated, thereby not providing landmarks to facilitate navigation back and forth across the pages.
TextInVizTOC

Table of Contents: an ordered list of headings without connective narrative and minimal landmarks spread across many pages. 

It may be suggested that the gist of document should instead be conveyed by the abstract. An abstract is more terse and provides the narrative sequence of the document – however – the abstract does not facilitate navigation of the document as it does not provide any links or page references. Furthermore, the length of abstracts tend to be quite limited: meaning that only the broadest overview can be described in the abstract.

Instead, I decided to use a very old technique. Chambers’ Cyclopaedia from 1728 creates a View of Knowledge – a unique narrative description of contents of his encyclopedia, all neatly organized in a hierarchical word-tree. Here’s a subset – you can read it left to right as a continuous narrative:

Cyclopaedia

So, instead of a Table of Contents, perhaps Chambers’ approach is more useful. Here is a a narrative Description of Contents  for my thesis:

TextInVizDOC

In one page, this description provides a brief readable narrative, outlining all the major arguments of the thesis, using FULL CAPS (and whitespace) to indicate major parts of the thesis. SmallCaps indicate each major heading. Superscripts (in blue) are links to page numbers.

[Aside: Note that word trees tend to use word size to indicate data. Unfortunately, word size uses alot of space; Full caps, small caps, superscripts and color can all be used to indicate data while offering higher data density and improved readability.]

The approach can be applied recursively: each part of the thesis has its own description of contents. For example, here’s the description of Part II:
TextInVizDOCpartII

In this example, the description contains many thumbnail images providing cues to the contents associated with each section (e.g. what’s the difference between text semantics and typographic semantics? – hint: comic book text is the image associated with typographic semantics). The large image at the bottom is a diagrammatic representation summarizing the design space derived in from the preceding sections.  Other cues (e.g. lines) can be added too to connect relations between items further apart.  Images also act as landmarks and commonality between different types of images help visually separate out various sections as well.

Does it work? I created the Descriptions to help me clarify how the pieces fit together. A couple of reviewers have commented how the descriptions have helped them understand the document – acting as a preview to each part and as a quick reference to navigate around the document. Obviously, more effort is required to create the description than a table of contents or abstract – but I’ve found it much more useful than either the abstract or table of contents to both navigate the document and to help move portions of the thesis into a better topics and flows.

 

Advertisements
Posted in Data Visualization, Design Space, Search | Leave a comment

Color in Text Visualization

Highlighting text via bold or italic is less noticeable than color. There is an incredibly long history of the use of color in a continuum from medieval times to present. From a visualization perspective, color is scoped to the level of words, or Proper Nouns, such as keywords in context (KWIC).  Color ranks highly in a study by Strobelt et al. Color is already among the most highly used attribute for encoding data into text (as of Jan 2016: 71 of 249 text visualizations at textvis.lnu.se used color, only size was slightly higher at 76, and the next closest was orientation down at 10). And color has a strong visceral appeal – note all the tag clouds that vary word color randomly.

So, what more can be said about color in text?

The scope of text in visualization doesn’t need to be constrained to words: color can be applied to entire phrases, or down to individual glyphs. Certainly medieval authors were fond of illuminating initial letters of paragraphs and rubrication for the leading letter in sentences:

IlluminatedInitials

Colorful leading letters: Illuminated initials leading paragraphs, rubrication on lead initials of sentences. (bonus, note the sparkline at top right)

Note also that color does not need be constrained to a single color. In the above manuscript, the illuminated initials are blue set on an ornate red background. The idea of varying both foreground and background color of text has certainly occurred to some visualization designers as well:

SpatioTemporalTags

Spatiotemporal tags vary both foreground and background color.

Mixing different foreground and background colors can result in illegible type: the words architecture and alemania are hard to read in the above example, as are some of the foreground/background combinations in this stem&leaf timetable:

 

Timetable

Red text on an orange bar isn’t particularly readable.

Typographers have different ways of combining multiple colors into letters. Chromatic fonts contain complementary shapes, each printed in a separate color, each layer carefully aligned on a press. Consider this historic poster:

TheatreRoyal
Chromatic fonts on historic poster: colored outlines and drop shadows. Image courtesy of  Gerry Leonidas, Department of Typography & Graphic Communication, University of Reading. 

Chromatic fonts emerged in the mid-1800’s, along with large sizes, many weights, wide variety of serifs and decorations. For the first 40 years of computer screens 72-96 DPI probably wasn’t sufficient for chromatic fonts and likely would have resulted in visual artifacts (i.e. jagged bits of high contrast colors interfering with legibility).  Support for chromatic fonts on computers is very recent and currently not consistent across browsers.
But, computer-based chromatic fonts open up interesting design possibilities. Wood letters with ink-based printing are limited: the approach scales to only a few discrete ink colors.  Instead, with software-based chromatic fonts, many hues and gradients are feasible and can be applied to portions of letters. The implication for visualization is that color can be applied to any subset of letters or parts of glyphs; potentially used to indicate quantities; still maintain legibility; and have a much more dynamic aesthetic than plain text. Some of the examples of computer-based chromatic fonts hint at very interesting future visualization possibilities:

 

chromaticfont

Sample chromatic fonts via colorfonts.wtf

 

It would be very interesting to see what new ideas some visualization research or graphic design programs might come up with!
(Title image conveniently cropped from Specimens of Chromatic Wood Type… 1874)

Posted in Data Visualization, Font Visualization, Text Visualization | Tagged , | Leave a comment

Text Visualization and Code Editors

Long before I started investigating typographic attributes for data visualization, Aaron Marcus and Ron Baecker were doing the same thing for software code. Back in the mid 1980s there weren’t integrated development environments, no integrated debugging tools and so on. My first coding environment in the mid 80s consisted of green text on a black background: 24 x 80 characters at maybe 72 dpi (*).

IBM3277_smaller

Old school: low resolution green-screen display.

If you wanted to better understand what was happening in a few blocks of code, there was no windowing system: you had to rely on flipping back and forth between code sections and carrying a lot of detail in short term memory – which is difficult.

So, instead, we printed it out. High speed line printers printed out large amounts of text on fan fold paper. You can cross reference between a few hundred of lines of code on paper much more easily than 24 lines on screen. And, on paper, you can use a highlighter to indicate variable declarations, underline conditionals, add squiggles or boxes around code and so on.

Baecker and Marcus realized that the then new Apple Laserwriter offered great resolution (300dpi) and excellent typographic capabilities. It could automate the tasks of differentiating types of statements using typographic attributes, instead of manually marking up the print out after the fact. They itemized the typographic attributes available, wrote routines to format and print code, did numerous studies and created recommendations through the 1980’s (e.g. CHI paper) – all before the academic research field of data visualization got started. Here’s some code from their book Human Factors and Typography for More Readable Programs formatted following their recommendations.

Baecker Marcus Code Formatting

1980’s laser printer-based code formatting using font weight, size, italics, shading, even smallcaps.

It’s a great example of visualization using typography. Italics, bold, background shading, slightly larger size for the function name, even smallcaps on the constant.

Over time, the display resolutions got better, code writing software got better, typography on desktops got better. Leading code editors nowadays do syntax highlighting reminiscent of Baecker and Marcus’s work – notice the many typographic attributes at work:

WebStorm

Snapshot of WebStorm – syntax highlighting varies underline, italics, bold, colors aid understanding. 

While the rich typographic formatting of in-line contextual highlights is the norm in software editors; typographic formatting to convey data is not normal in data visualization. Two years ago I went through all the text visualizations then listed at the Text Visualization Browser and cataloged which visual attributes were used to add data to text. Out of 249 visualizations then listed:

  • 40 didn’t have any text in the main visualization at all – they were visualizations such as point clouds, graphs, etc.
  • 103 used plain text without indicating any additional information, not even color coding – essentially plain labels.
  • Only 106 out of 249 (40%) used any kind of additional formatting(!) And what kinds of formatting did they use? Almost entirely size and color were used (often both). Only very rarely were typographic attributes – such as bold, italics, caps or underlines  – used, as indicated in this table:

TextVisualizationBrowserStats

Notice, for example, that varying font intensity is more common (occurred 9 times) than varying font weight (occurred 6 times). This is a bad design choice, because fonts come in a wide variety of weights which are more legible than varying font intensity:

TextVaryingIntensity

Changing brightness reduces text legibility…

TextVaryingWeight

while, changing weight maintains text legibility.

What makes this particularly interesting is that the researchers designing and coding these visualizations were spending long hours staring at code editors full of typographic cues like the WebStorm snapshot — but then didn’t use any typographic cues in their visualizations! Possibly they had become accustomed to the typographic manipulation and no longer consciously aware of it, or, perhaps because it was situated in a different (non-visualization) context, the mental connection to visualization was never made (because visualization research rarely talks about typography).

This is just one example of a kind of “design blindness” (like change blindness, but without the change – i.e. missing something that is clearly visible). What other cues are UI designers seeing but completely missing?

Posted in Data Visualization, Font Visualization, Legibility, Text Visualization | Leave a comment

Visualizations of Many Variables

Underlying many of the posts on typographic visualization in this blog is the notion of visualization of many variables. Many modern visualizations only tackle a few variables beyond x,y layout. Treemaps use color and size; choropleth maps use color; tag clouds use size. Even Hans Rosling’s engaging TED talk is a bubble plot with only size, color and animation. Yes, parallel coordinate plots scale up to potentially a dozen or more variables but interactive techniques are often required to reveal the crooked path that corresponds to one particular item — it’s not feasible to do so through a mass of cluttered lines. And furthermore, most visualization techniques work with homogeneous data: not messy heterogeneous data at different scales, different entities, differing qualities and not necessarily easy ways to computationally join the disparate data together.

Tufte’s  Envisioning Data has quite a few examples of the latter; and many more variants are available on the Internet. Maps with many layers, exploded or cutaway drawings, dance notation, graphic timetables and many other kinds of detailed charts:

Bertin considered these types of visualizations graphical inventories – not for visual perception of patterns but rather a means to graphically organize a lot of related information. And so, inventory-style visualizations are not commonly used because most visualization tools are focused on the easy perception of patterns without the clutter.

But can inventory-visualizations be also used for rapid preattentive perception of patterns? At the outset they may seem cluttered, but they can still use attributes such as color to draw fast attention, and then still provide all the detailed context. Presumably this use of visualization as inventory plus preattentive highlights is already used today in real-time operational settings such as factories, pipelines and electric grids: the blinking box requires attention, but all the rich graphical detail around the blinking box indicates other assets that can be manipulated to rectify the situation. Something to consider for 2018.

Posted in Data Visualization | Leave a comment

Stem & Leaf Bigrams

Stem & leaf plots are quirky alphanumeric data visualizations. In typical usage they show distributions of numeric values. Here’s a simple dataset of heights of my family on the left, transformed into a stem and leaf plot on the right:

bigram-stemLeafIntro

The resulting stem & leaf plot, at a macro level, shows a distribution. At a micro-level, you can easily read off the minimum value (152), the maximum value (187), and probably make a reasonable guess to the medium value – or in this case – the median is explicitly indicated with an added underline (172).

Character-based Stem & Leaf Plots

So can stem & leaf plots be extended to plot text-based data?

Consider a simple example based on letter pairs. Bigrams are pairs of letters. They can be analyzed across a large body of text and certain letter pairs will be common in certain languages. This can be useful for applications such as auto-detection of language and for cryptography.

Since bigrams are pairs of letters, one could split a bigram into a first letter (for the stem) and the second letter (for the leaf). The resulting stem & leaf bigram plot for all the letter pairs that occur more than 0.5% in the English language are shown here:

bigram-Leading

The stem (left side) is the leading character of the bigram; the leaves (right side) are all the trailing letters, with font weight indicating frequency. You can see that many of the most common English bigrams start with E. And you can see that ER, IN and TH are among the most frequent bigrams.

Unlike a stem & leaf plot based on numeric values, the trailing characters may be of interest. Instead we shift the stem to the trailing character when we add a second stem&leaf plot on the right:

Bigrams-Both

In the second stem&leaf plot, the stem (now on the far right) indicates the trailing character; and all the leaves indicate the leading character. E is also the most common trailing letter. However, it is now a bit more visible that a trailing N is fairly high frequency.

Stem & Leaf Trigrams

And the approach can be extended to trigrams. The plot below is centered on the second letter of high frequency English language trigrams:

Bigram-Trigram

The trigrams are ordered from the center out – so in the top row, the top trigram is HAT, followed by EAR, followed by WAS. The most common trigram is THE, and there appear to be many trigrams with T at the center (note, dataset is from Practical Cryptography: http://bit.ly/1wf1Lgc, which didn’t consider word spaces).

Word Bigrams as Stem & Leaf Plots

Text-based stem & leaf plots can go beyond characters as units and extend to words and phrases. I did an previous blog post regarding characters and adjectives from Grimms Fairy Tales, which essentially was a stem & leaf plot.

Another variant is to use people’s names as bigrams: the typical forename-surname is a word bigram. Forename-surname is commonly used in the west can be used to construct textual stem & leaf plots. Here’s some of the families that were passengers on the Titanic. Surnames form the central stems – first class on the left half of this diagram (in a fancy serif font) – third class on the right half (in a plain sans serif font):

bigrams-Titanic

In this example, leaves on the left side of the stems are women (italic), leaves on the right are men. Children are indicated in ALLCAPS. Those who perished are bold. For example, top left is Ethel Fortune, a first class adult woman who survived. In the same family are Mark and Charles Fortune,  first class adult men who died. Among the first class, one can see that the women (along the far left) are almost entirely non-bold, meaning that they survived. Among the first class men, there are many more bold entries (died) although many of the survivors are allcaps (children). Thus, we see among the first class, that many women and children survived. However, in the right half of this plot we see the third class almost entirely bold: Most of the third class perished – regardless if they were women or children.

Phrases as Stem & Leaf Plots

As a final example, we expand the approach out to phrases. This example is based on the book of Psalms from the Bible, which has a number of repetitious structures. Here, we identify some of the most common phrases, indicated along the horizontal grey line. Above the grey line are the leaves that precede the common phrase, below the grey line are the leaves that follow the common phrase.

bigram-PsalmPhrases

Interestingly, the phrase “I will praise thee” is common, but there is no commonality among the preceding and following phrases. However “O give thanks unto the Lord” is typically preceded by “Praise ye the Lord” and followed with “for he is good”.

 

 

 

 

Posted in Alphanumeric Chart, Data Visualization, Font Visualization, Text Visualization | Leave a comment

Lost Works of Jacques Bertin on Typography

It’s been 50 years since Jacques Bertin’s Sémiologie Graphique was published. Bertin looms large in history of both data visualization and cartography. Before we had textbooks on data visualization by Munzner (Visualization Analysis and Design), Ware (Information Visualization: Perception for Design) or even Spence (Information Visualization: Design for Interaction), Bertin provided the theoretical foundation that much of visualization relies on today. Sémiologie Graphique structures the design space of visualization with the now familiar concepts of marks (point, line, area); visual attributes (only six in his version); type of perception (quantitative, ordered, categoric and associative); and layout. Beyond these, Bertin also considered many other aspects such as spatial separation (to form small multiples) ordifferent use cases of visualization including communication, analysis, and inventory.

However, only one aspect of Bertin’s work never made the translation from the French original to English: typography! Strangely, out of 450 pages+ only 4 pages on typography were not translated. In these 4 pages Bertin discusses the importance of the literal information represented by text. He notes that text is often the only encoding commonly accessible to both the textual/verbal system of encoding information; and the visualization system of encoding data.

Furthermore, Bertin points out that text is not selective. In other words, text is not preattentive, meaning that patterns do not automatically pop-out. If I ask you to find the word “six” occurring in the first paragraph, you need to linearly scan through the text – the benefits of visualization do not occur. Typographers would agree: they put significant effort in making text appear uniform with no visual anomalies to standout, carefully tweaking letterforms and kerning pairs to achieve this effect.

However, typographers also understand the need to make some words visually pop-out from surrounding text and therefore provide forms of emphasis such as italics and bold. Bertin also points out that these forms of emphasis are available and discusses them in the context of the technology of his time: pencil, pen, professional printers (which would have used phototypesetting in the late 60’s), and dry-transfer lettering (e.g. Letraset). And he nicely itemizes attributes of typography, available on page 415 of the French edition of Sémiologie Graphique:

BertinType2

Bertin’s font attributes included letter forms, font family, font width, spacing, size, weight, case, and slope (italic).

So, why was Bertin wildly successful, but his commentary on typography so minimal that it was dropped from the English translation? Good question!

One answer is that even though Bertin indicates the potential of text to indicate data beyond literal text; he says the incremental addition of text only helps low-level elementary reading, not the higher level of visual perception of patterns. However, Bertin is writing in the late 1960’s: 15-20 years before Edward Tufte popularizes the notion that a visualization can be read at many different levels depending on the task, which Tufte calls micro/macro reading (Tufte: Envisioning Information).

 

Another answer is that even though Bertin acknowledges typographic attributes such as individual letter forms, typeface, width, spacing, size, weight, case and italic – he doesn’t provide any examples of the use of these attributes. On the otherhand, he provides hundreds if not thousands of examples of the other six visual attributes (size, orientation, hue, brightness, texture, shape), making sure that his core concepts are well explained and illustrated.  A parallel can be seen in open-source visualization libraries: there were many different open-source visualization alternatives in the early 2010’s.  However, Mike Bostock not only provided a well organized library with D3.js, he also provided a lot of compelling examples of visualizations implemented in D3 with source code. Mike made it far easier to adapt and extend D3’s model by starting with examples rather than requiring the extra effort to learn some other library and then figuring out how to create those examples.

There are other possible reasons, but the unfortunately reality is that Bertin’s typographic insights were side-stepped and never exposed to the English language research community. Bertin also wrote a follow-on article (Classification typographique : Voulez-vous jouer avec mon A doi : 10.3406/colan.1980.1369) specifically on the visual attributes of type in 1980 – but again, no examples and no translation (it does provide a better organization of the typographic attributes).

In Sémiologie Graphique, Bertin made 100 different visualizations of a dataset indicating  three major occupations across 90 departments in France. None use typographic attributes (although a few use simple plain labels). I decided to make one typographic example – here’s Bertin’s ternary plot (p. 115) where the bubbles have been replaced with text, sized in proportion to population, colored based on occupation proportions. You can choose to focus on the macro patterns (e.g. most districts have an agricultural bias, or most of the agricultural districts tend to have smaller populations); or you can choose to focus on the micro details, e.g. district P (Paris) has the largest population and no agriculture; or, district 32 is the district with the highest proportion of people employed in agriculture.  (If you want more non-Bertin examples, see many of my recent postings regarding typographic visualizations).

BertinTernaryPlot

Ternary plot, based on Bertin, using alphanumeric codes instead of dots.

According to Google Scholar, there are 6004 citations for Bertin’s Sémiologie Graphique (across the French, German and English editions); while there is only ONE citation for Classification typographique : Voulez-vous jouer avec mon A. Perhaps this short article and references will help Bertin’s ideas of typography get more recognition and citations in future visualization and cartography research work.

 

Posted in Alphanumeric Chart, Data Visualization, Font Visualization, Text Visualization | Tagged , | Leave a comment

Patent Visualization and Litigation Ratios

Earlier this week, Scott Langevin and I were fortunate to speak at the Strata Big Data Conference in NYC. The topic was Text Analytics and New Visualization Techniques. It discussed some of the examples on this blog and my research; and additionally showed these techniques applied as a front-end to big data and text analytics in some large-scale real-world applications from Uncharted.

One example was an extension to a visualization of patents. Understanding patent activity is of interest, as patent activity is a leading indicator of new commercial opportunity and areas where new skills and expertise are required. Also, patent litigation is an indicator of areas with problems where people need to be more diligent in research and more careful in crafting patents.

At Uncharted , we created a visualization of all the patents granted since 1982 as a massive graph. All patent applications refer to earlier patents. From these references, we can build all the connections between patents into a massive graph. Then, we use a hierarchical graph layout technique so that patents that are highly interconnected are drawn close to each other (described here). The result is a visualization where each patent is a small transparent orange dot and links between them are thin transparent blue lines (Images courtesy Uncharted Software, used with permission).

PatentGraph

Graph of all patents since 1982.

The graph layout nicely clusters patents together into visible communities. The graph is labeled, by using two or three unique terms from the most heavily cited patent in each community.

As an interactive application, the viewer can zoom in to successively lower and lower levels to see sub-communities and sub-sub-communities. There are also additional features such as search, color-coding, trend analysis and so on. All these features are used to aid the viewer in the deep analysis of IP topics, growth areas, problem areas and so on. In this post, we’ll just look at one feature regarding litigation. In this next image, patents with litigation are colored with purple dots (labels turned off, so you can see all the dots).

PatentGraph_wLitigation.PNG

Communities of patents, with purple dots indicating patents with litigation.

Clearly, there are various communities that have significant patent litigation. But the ratio of litigated patents to uncontested patents in each community is not clearly distinguishable. While each individual patent is visible as a dot, what’s needed is some way to indicate summary metrics for each community.

Rather than adding extra visual elements that clutter the screen, we can re-use a scene element that already exists at the aggregate community level — in this case, the labels. Following the techniques discussed in this blog, we use the oblique angle of the text to indicate the litigation ratio: text with steep italics indicates communities that have high litigation, text with no italics have normal litigation, text with reverse italics have no or very low litigation.

PatentGraph_wLitigationRatio.PNG

Label oblique angle indicates ratio of patents under litigation in each community.

This is useful to know in advance if crafting a new patent related to a particular community: more care is likely required to create a new patent in a community that already has many disputes.

There are a half dozen other examples of text analytics and visualization in the full set of Strata slides, available here or at Strata.

 

 

 

Posted in Data Visualization, Font Visualization, Graph Visualization | Tagged | Leave a comment