• Home
  • Readings
  • Github
  • MIES
  • TmVal
  • About
Gene Dan's Blog

Monthly Archives: November 2014

You are browsing the site archives by month.

No. 113: International Flight Paths

30 November, 2014 7:48 PM / 1 Comment / Gene Dan

So as  continuation from my last post, I’ve decided to map out all the major air traffic routes worldwide. Data were obtained from openflights, with the individual airport data here and the route data here. You can see the processed gexf file here. I’ve started work on creating an automated script to convert raw data into a .gexf file. It’s not quite done yet, but the script used to do so can be found below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
setwd("./openflights-code-769/openflights/data")
 
### create node XML
airports <- read.csv("airports.dat",header=FALSE)
head(airports)
airports$nodes <- paste('<node id="', as.character(airports$V1), '.0" label="',airports$V5, '"/>',sep="")
 
### create edge XML
routes <- read.csv("routes.dat",header=FALSE)
routes$edgenum <- 1:nrow(routes)
head(routes)
routes$edges <- paste('<edge id="', as.character(routes$edgenum),'.0" source="', routes$V3, '" target="',routes$V5, '"/>',sep="")
 
### build metadata
gexfstr <- '<?xml version="1.0" encoding="UTF-8"?>
<gexf xmlns:viz="http:///www.gexf.net/1.1draft/viz" version="1.1" xmlns="http://www.gexf.net/1.1draft">
<meta lastmodifieddate="2010-03-03+23:44">
<creator>Gephi 0.7</creator>
</meta>
<graph defaultedgetype="undirected" idtype="string" type="static">'
 
### append nodes
gexfstr <- paste(gexfstr,'\n','<nodes count="',as.character(nrow(airports)),'">\n',sep="")
fileConn<-file("output.gexf")
for(i in 1:nrow(airports)){
gexfstr <- paste(gexfstr,airports$nodes[i],"\n",sep="")}
gexfstr <- paste(gexfstr,'</nodes>\n','<edges count="',as.character(nrow(routes)),'">\n',sep="")
 
 
### append edges and print to file
for(i in 1:nrow(routes)){
  gexfstr <- paste(gexfstr,routes$edges[i],"\n",sep="")}
gexfstr <- paste(gexfstr,'</edges>\n</graph>\n</gexf>',sep="")
writeLines(gexfstr, fileConn)
close(fileConn)

I’ve created three separate graphs using different measures of influence – Betweenness Centrality, Eigenvector Centrality, and Degree. Click on an image to expand the graph, although be aware that it may take a while to load.

Selection_001

Node and text size based on betweenness centrality

You’ll notice that some airports such as LAX, which weren’t noteworthy in the graphs depicting domestic flight routes have gained new prominence from an international perspective since they serve as important traffic hubs for overseas flights. Other important airports include Dubai International Airport (DXB), Beijing Capital International Airport (PEK), Charles De Gaulle International Airport (CDG), and interestingly, Anchorage International Airport (ANC).

Selection_002

Node and text size based on eigenvector centrality

Selection_003

Node and text size based on degree

Posted in: Mathematics / Tagged: international flight network graph, international flight routes

No. 112: Flight Routes in the United States

22 November, 2014 4:30 PM / Leave a Comment / Gene Dan

Check out these network graphs I made that show domestic flight routes in the United States. You can take a look at the data set here. The following network diagram displays airports as nodes (the circles in the graph) and routes between airports as edges (the lines between the circles). The size of the circles is based on a measure called betweenness centrality, which is a measure of an airport’s influence within the network.

air2

Node size based on betweenness centrality

So, there are a couple of things that stand out – first, even without any information dealing with the number of passengers, betweenness centrality has identified some of the busiest airports in the United States. Second, when I created the input dataset, I stripped the file of all geographic information. Even so, if you invert the graph and rotate it a little you might be able to get an image that closely matches the geographic locations of the airports.

The two tails represent airports from remote parts of the United States – Alaska and Guam.

Below is another diagram, with the difference that node size is calculated via eigenvector centrality rather than betweenness centrality. With respect to eigenvector centrality, nodes are given a higher score if they are connected to other nodes that have a high score, and less otherwise.

Node size based on eigenvector centrality

Node size based on eigenvector centrality

Posted in: Mathematics, Uncategorized / Tagged: domestic air traffic, domestic flight paths, graph theory, network diagram, US flight paths

No. 111: Visualizing Erdos Collaborators

20 November, 2014 8:31 PM / Leave a Comment / Gene Dan

So a while back I wrote about visualizing networks – well, I’m back at it again and I found a sweet dataset containing the Enron emails. However, that dataset might take a few day’s worth of computing power to process, so in the meantime I’ll be showing a much more manageable dataset containing research partnerships of mathematicians who collaborated with Paul Erdos.

In short, Paul Erdos was a great 20th century mathematician who was famous for his eccentric behavior, prolific publishing, and his copious consumption of amphetamines and caffeine. Today, mathematicians often refer to something called an Erdos number, which indicates one’s closeness to the late mathematician. For example, those who published papers directly collaborating with Erdos himself receive an Erdos number of 1, those who collaborate with mathematicians whose Erdos number is 1, but not Erdos himself are given an Erdos number of 2, and so on.

 

Downloading the Dataset

You can get the dataset here. It is a simple text file, although it needs to be processed into .gexf xml format in order to be used with gephi.

Erdos02 (~-Desktop) - gedit_003

The text file can be divided into two main sections, one containing the node labels, or the names of the mathematicians who collaborated with Erdos. The second section contains the edges, which link one mathematician to the other.

Erdos02 (~-Desktop) - gedit_004

I had to use a script to transform the text file into XML format. I was impatient, so I used Excel. If I were doing a more important project, I would have chosen otherwise.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Option Explicit
Sub create_edges()
 
Dim x, y, z As Long
Dim currform, currend, curredge As Long
 
For x = 1 To 507
    currform = Range("A1").Offset(x-1, 0).Value
    y = WorksheetFunction.CountA(Range(x &amp; ":" &amp; x)) - 1
    For z = 1 to y
        currend = Range("A1").Offset(x-1,z).Value
        Sheets("edges").Range("A1").Offset(curredge-1,0).Value = curredge
        Sheets("edges").Range("A1").Offset(curredge-1,1).Value = currfrom
        Sheets("edges").Range("A1").Offset(curredge-1,2).Value = currend
    Next z
Next x
 
End Sub

erdos (copy).gexf (~-Desktop) - gedit_006

The above image shows the processed data in XML form.

Visualizing the Collaborators

The next step is to import the dataset into gephi. At first, the visual form of the data looks like a meaningless blob:

Gephi 0.8.2 - Project 0_002

To remedy this, I used the Force Atlas algorithm to spread the nodes out to visualize the network structure.

Midway through the Force Atlas Algorithm

Midway through the Force Atlas Algorithm

Here’s what it looks like after running for half an hour As you can see, the network structure is becoming more apparent:

Gephi 0.8.2 - Project 0_008

After stopping the Force Atlas algorithm, the network is a little easier to interpret, but the nodes are all the same size. To emphasize the most important nodes, I adjusted the size of the nodes by eigenvector centrality. As you can see, Erdos is the most influential member.

Gephi 0.8.2 - Project 0_010

After running a community detection algorithm, we can see several distinct communities of mathematicians amongst the collaborators, indicated by color:

Gephi 0.8.2 - Project 0_011

The final output is below. Click to see it in full resolution (it might take a while to render if you try to click on it):

erdos2

 

Posted in: Uncategorized / Tagged: erdos, erdos collaboration graph, erdos network graph, erdos number

No. 110: The Black Market

17 November, 2014 8:10 PM / Leave a Comment / Gene Dan

2013 was an eventful year for the black market, with  Silk Road (and now Silk Road 2.0) having been seized by the federal government, substantially impacting the value of bitcoins relative to the U.S. Dollar (although the vast majority of bitcoins are used in legal transactions). That, in addition to other developments in cyberspace including the government’s involvement in surveilling and controlling it, has gotten me interested in the underground economy, how it works, along with the involvement of the government and seemingly legitimate private organizations. I’ve started to educate myself with Andrew Feinstein’s book on the global arms trade, but before that I would like state some of my naive assumptions about the way the black market works, and then maybe come back to this in a few months’ time once I’ve learned a little bit more.

Selection_006

The Black Market Exists Because Demand is Left Unsatisfied in Conventional Markets

Simply put, people want goods that aren’t available from legally authorized markets. People want drugs. They want pirated media and counterfeit goods of name-brand products. Vengeful lovers want murders carried out on their behalf and curious individuals might want to try some products from countries currently under economic sanctions.

The Absence of Property Rights Leads to Organized Crime

The threat of punishment by the authorities, in tandem with government control over conventional routes of transportation deters everyone but the most die-hard suppliers from reaching their customers. This government-induced scarcity causes prices of some illicit good or services to be thousands of times more than what they would otherwise be in the absence of government restriction. The high price drives those who are the most willing to risk punishment to take extreme measures to make markets.

Unlike suppliers of legal goods, black market suppliers do not receive government protection via property rights. This creates the need for black market businessmen to raise their own private armies to protect their own inventory along with their supply chain affiliates – this leads to the formation of quasi-governmental entities that are commonly known as gangs, or drug cartels. If you think about it, protection money is kind of like taxes – in exchange for a payment, you will be protected by the gang. If you fail to pay up, you’ll get your property sacked and your kneecaps broken. Likewise with taxes, in exchange for a payment you will be protected by an army. If you fail to pay up, you’ll either go to jail, face garnishment of wages, or both. So a gang is, in a way, a de-facto government overseeing the market for whatever illegal good or service it sells. The lack of property rights creates a huge incentive for rival gangs to fight over control of the market, as consolidation would allow them to further control prices via monopoly power. Now, you might ask, why then aren’t Dell and Lenovo raising their own private armies and blasting each other to smithereens to gain control over the computer industry? The reason is that it’s too expensive – computer prices simply aren’t high enough to justify the cost of raising an army and going to war, because the production, transport, and sale of computers aren’t restricted by the government. In other words, there’s no artificial scarcity with respect to computers.

Violence as a Symptom

You might have heard that buying drugs and counterfeit goods supports violence. Well, in this steady-state of affairs it does – your money would likely end up in some drug-lord’s hands which could fuel his next turf war – but it’s a poor justification for discouraging someone from buying something on the black market. One should ask why the market for these goods are controlled by violent drug lords in the first place. The high price of restricted goods, along with the lack of property rights encourages the formation of privately-armed businesses which in turn increases the demand for weapons which in turn fuels the illegal arms market – not a pretty picture. Legalization of drugs would lead to a result similar to legalized alcohol, the lower price would decrease the presence of organized crime and street violence, but at the cost of public health and safety which would also put a strain on our public services. So sometimes the question of legalization is not so easy to answer.

 

 

Posted in: Uncategorized

Archives

  • September 2023
  • February 2023
  • January 2023
  • October 2022
  • March 2022
  • February 2022
  • December 2021
  • July 2020
  • June 2020
  • May 2020
  • May 2019
  • April 2019
  • November 2018
  • September 2018
  • August 2018
  • December 2017
  • July 2017
  • March 2017
  • November 2016
  • December 2014
  • November 2014
  • October 2014
  • August 2014
  • July 2014
  • June 2014
  • February 2014
  • December 2013
  • October 2013
  • August 2013
  • July 2013
  • June 2013
  • March 2013
  • January 2013
  • November 2012
  • October 2012
  • September 2012
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • February 2012
  • January 2012
  • December 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • January 2011
  • December 2010
  • October 2010
  • September 2010
  • August 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • September 2009
  • August 2009
  • May 2009
  • December 2008

Categories

  • Actuarial
  • Cycling
  • Logs
  • Mathematics
  • MIES
  • Music
  • Uncategorized

Links

Cyclingnews
Jason Lee
Knitted Together
Megan Turley
Shama Cycles
Shama Cycles Blog
South Central Collegiate Cycling Conference
Texas Bicycle Racing Association
Texbiker.net
Tiffany Chan
USA Cycling
VeloNews

Texas Cycling

Cameron Lindsay
Jacob Dodson
Ken Day
Texas Cycling
Texas Cycling Blog
Whitney Schultz
© Copyright 2025 - Gene Dan's Blog
Infinity Theme by DesignCoral / WordPress