NSA – Silverton Consulting

We have written about this in the past but it’s getting more interesting every day. Just this week we read another article in MIT Technology Review about How a New Science of Cities is Emerging from Mobile Phone Data Analysis. (For more see my post on Cheap phones + big data = smarter world).

This is the same phone metadata that the NSA has been having so much trouble with. However, when identities are properly protected and mobile phone metadata are provided to non- intelligence data analysts, real science can be performed. New urban scientists can now get a peak into activities they could never see before.

The MIT article talks about mapping commuting activity, supplying population density maps over time, the distribution of wealth in Africa and additional science being discovered about cities and only revealed through phone metadata.

Urban science started decades ago

When I was in college we were just beginning the quantification of geography leading to a prototype of urban science. We used to use newspaper subscriptions, traffic statistics, travel statistics, etc. to determine which cities were more connected to which other cities. Of course, newspaper subscriptions would no longer suffice in the Internet world we live in today. Nowadays, mobile phone metadata have taken over the role of all this data.

We need more and better data

There are two problems with the current level of mobile phone metadata today.

Mobile metadata aren’t that freely available. Yes there are a few datasets which are widely available but these aren’t even the top 50 cities of the world and are only for a relatively short period of time. Mostly these datasets are available on an ad hoc basis at best. What we need is ongoing data, for the 50 largest cities world wide to have a chance at really understanding urban science.
Mobile metadata must be both anonymized AND uniquely identifiable. We don’t want to know who owns the mobile phones but we would need to know something about them, such as zip/postal code, sex, age and perhaps other information. (Although scientists have already figured out a way to identify a mobile phone owner’s home cell tower which is probably even more precise than their postal code).

For the later problem, it shouldn’t be too hard to come up with a reasonable way to protect phone owner identities but still provide some way of uniquely identifying a phone. I would guess that the current mobile phone datasets used in the studies above have already solved this. However, if they haven’t I should think a random ID number assigned to uniquely identify each phone in the dataset would suffice, with an associated database table that mapped these random IDs to some set of phone owner demographics.

Supplying more metadata

But for the earlier problem, the lack or freely available, continuous mobile phone metadata, I would suggest some ongoing type of dataset, periodically published by telecom providers in each city, say once per quarter for the prior year (or even two years back) phone activity. Such metadata would be freely available to anyone who wanted it.

Perhaps in the US this mobile phone metadata collection could be mandated and maintained by the FCC or some other federal entity or maybe the telecom providers would host the database for themselves by location. It doesn’t much matter where the data is as long as it’s Internet accessible, provides anonymity, and is supplied in an ongoing basis for a representative set of large cities in a country.

Given the troubles the NSA has been having, it has been suggested that some entity host mobile phone metadata outside the government which could process “approved” queries by intelligence and other agencies. It seems to me that if such an entity were to exist this would be the perfect place to provide an anonymized version of this data on a periodically delayed basis to the world’s scientist.

Once the data becomes more freely and more periodically available, there’s probably an awful lot that scientists and even commercial endeavors could make of it.

A finer grained urban physics could emerge from such data if it were more freely available. Even commercial interests would be interested in the phone metadata as well.

Comments?

Photo Credits: Jakarta Skyline, Part 2 by The Diary of a Hotel Addict

What’s a hash?

Cryptographic hash functions like SHA-1 are designed such that, when a string of characters is “hash”ed they generate a binary value which has a couple of great properties:

Irreversibility – given a text string and a “hash_value” generated by hashing “text_string”, there is no way to determine what the “text_string” was from its hash_value.

Uniqueness – given two or more text strings, “text_string1” and “text_string2” they should generate two unique hash values, “hash_value1” and “hash_value2”.

Although hash functions are designed to be irreversible that doesn’t mean that they couldn’t be broken via a brute force attack. For example, if one were to try every known text string, sooner or later one would come up with a “text_string1” that hashes to “hash_value1”.

But perhaps even more serious, the SHA-1 algorithm is prone to hash collisions which makes fails the uniqueness property above. That is, there are a few “text_string1″s that hash to the same “hash_value1”.

All this wouldn’t be much of a problem except that with Moore’s law in force and continuing for the next 6 years or so we will have processing power in chips capable of doing a brute force attack against SHA-1 to find text_strings that match any specific hash value.

So what’s the big deal?

Well it turns out that SHA-1 algorithms underpin almost all secure data transmissions today. That is, most Public-key infrastructure (PKI) depend on SHA-1 to sign digital certificates. And although that’s pretty bad, what’s even worse is that Secure Socket Layer/Transport Layer Security (SSL/TLS) used by “https://” websites the world over also depend on SHA-1 to send key information used to encrypt/decrypt secure Internet transactions.

On top of all that, many of today’s secure systems with passwords, use SHA-1 to hash passwords and instead of storing actual passwords in plain-text on their password files, they only store the SHA-1 hash of the passwords. As such, by 2021, anyone that can read the hashed password file can retrieve any password in plain text.

What all this means is that by 2018 for some and 2021 or thereabouts for just about anybody else, todays secure internet traffic, PKI and most system passwords will no longer be secure.

What needs to be done

It turns out that NSA knew about the failings of SHA-1 quite awhile ago and as such, NIST released SHA-2 as a new hash algorithm and its functional replacement. Probably just in time, this month, NIST announced a winner for a new SHA-3 algorithm as a functional replacement for SHA-2.

This may take awhile, what needs to be done is to have all digital certificates that use SHA-1, be invalidated with new ones generated using SHA-2 or SHA-3. And of course, TLS and SSL Internet functionality all have to be re-coded to recognize and use SHA-2 or SHA-3, instead of SHA-1.

Finally, for most of those password systems, users will need to re-login and have their password hashes changed over from SHA-1 to SHA-2 or SHA-3.

Naturally, in order to use SHA-2 or SHA-3 many systems may need to be upgraded to later levels of code. Seems like Y2K all over again, only this time it’s security that’s going to crash. It’s good to be in the consulting business, again.

~~~~

But the real problem IMHO, is Moore’s law. If it continues to double processing power/transistor density every two years or so, how long before SHA-2 or SHA-3 succumb to same sorts of brute force attacks? Given that, we appear destined to change hashing, encryption and other security algorithms every decade or so until Moore’s law slows down or god forbid, stops altogether.