Drawing Connections between Entity Columns

Correlation is a statistical function to measure how two features are related. When feature columns have discrete values, we can measure co-occurrence by plotting joint distributions. But co-occurrence is sometimes insufficient for discovering the semantics of a relationship.

The concept of co-occurrence can be generalized if we think of it as one type of interaction. Many such types of interactions can be defined between discrete entities. Multiple interaction variables give us a better shot at being more effective when encapsulating some form of relatedness.

Interactions can be directed or undirected. Datasets containing entities can be visualized as directed (or undirected) flow graphs. The edges between entities are the interaction variables that we can analyze. Our goal in this chapter is to make charts that help us to visualize...

Datasets

Data on bilateral trade flowing between nations fits right into the definition of a dataset with entities and relationships to explore. Katherine Barbieri, University of South Carolina, and Omar Keshk, Ohio State University, maintain a dataset that tracks the flow of trade between nations between 1870 and 2017. Amounts are converted to their equivalent US dollar values as of 2014.

The dataset is available publicly as part of The Correlates of War Project. Here is the link to the project: http://www.correlatesofwar.org/data-sets/bilateral-trade. We will be using version 4.0 of this dataset in this chapter.

The IMF's Direction of Trade Statistics (DOTS) quarterly release shares trade data between nations and is a source from which the project aggregates data. It is important to specify that DOTS includes only the trade value of merchandise (or ready-to-sell goods)...

Directed force networks

By using a directed force network, we can analyze the value of the imports nations exchange with each other. The time granularity of this dataset is in years. The time filter box available in Superset is easy to interpret and use for time columns that are more granular than yearly. Just select the clear option on it; we will not use it:

The time filter box

In the Query box, select the exporter column as the Source and importer column as the Target. The Metric function we choose will be irrelevant. Because this is time series data we are dealing with, we will inspect the trade flow between the nation entities for a specific year only. Therefore, there will be only one record for each exporter, importer directed relationship. You can use AVG or any other Metric function; the result will be the same:

Setting parameters for trade imports that spiked after...

Chord diagrams

Using a directed network diagram, we were able to identify the nations that experienced a spike or dip in their imports from other nations in 2010. We also noticed the nations that imported the most in that visualization. How about we now ask the question: which country exported more than they imported in 2014? How much did they import or export? Let's say we already have a list of nations in which we are interested, and they happen to be China, Germany, India, Japan, the United Kingdom, and the United States.

We will use a chord diagram to do this. The filter conditions on the year 2014 and the list of trading nations in which we are interested can be specified in the Custom WHERE clause text field:

(exporter="United Kingdom" OR exporter="Japan" OR exporter="China" OR exporter="Germany" OR exporter="India"...

Sunburst chart

Sometimes, it is useful to measure relationships between entities relative to a common denominator. In the case of trade flow data, let's try to visualize the fraction of the global export that each nation services. To do this, we will use a Sunburst chart.

We will take a snapshot of the time-series data and select the trade flow that took place in the year 2013. For this, we will use the Custom Where clause. In the Hierarchy field, set exporter and importer as the parent-child relationship. In the Primary Metric, select SUM(import_in_us_billion) and set its name as total_trade_volume:

Setting parameters for the Sunburst diagram to display trade flow in 2013

Perfect! Now just click on Run Query to render your Sunburst chart Global Trade Flow Between Nations in 2013:

The Sunburst diagram displaying trade flow in 2013

Selecting a parent entity from the inner...

Sankey's diagram

Sankey diagrams are flow diagrams where the width of the arrows is shown as proportional to the flow quantity. We will use one to understand from where merchandise goods imported into Western European nations originate. Because the number of nations that export to Western European nations is too large to visualize, we will group trade flow by the geographical regions of the exporting nations:

Setting parameters for the Sankey's diagram for showing imports to the Western Europe in 2014

The Since and Until fields in the Time section can be cleared because we will use 2014 as the value in the year column to filter rows in the dataset. Groups of nations can be represented using the value in their exporter_region as the Source field and importer column to represent the Western European nations. In order to only select a Western European country as the Target...

Partitioning

We can visualize how much nations in a particular geographical region import from other regions in a different way, using partition diagrams called TreeMaps. We will create a filter to select records for Asian (excluding Near East) nations. The first partitions will be proportional to the total import trade volume of each Asian nation. Then, each nation's partition will be further partitioned to show how much and in what proportions the nation imports from different geographical regions.

The nations controlling the largest area in the first partition will be the nations dominating the import market in Asia (excluding the Near East). The larger partitions inside each nation will represent the export market that supplies most of the merchandise goods to the corresponding Asian nation:

Setting parameters for the partition graph to display Asian nations imports...

Summary

We made a lot of charts in this chapter! Those were just some approaches to visualizing and analyzing a dataset with entities and a value quantifying a type of relationship. The Superset chart, called a partition diagram, is similar to a TreeMap, but it only generates a single level of partitioning. So, I chose to use a TreeMap instead of that, because it provided a more efficient and powerful way to visualize data.

Hopefully, you are now comfortable with using these chart examples for inspiration to upload your own entity-relationship dataset and analyze it in new and different ways.

In the next chapter, we will continue the trend of analyzing geographical regions by working with location data.