What really is the truth now?

[Pholus]

9. Well, we can look at the evidence of course...

LIST OF PRIMARY EVIDENCE (from US government claims, http://www.guardian.co.uk/world/2013/jun/10/nsa-spying-scandal-what-we-have-learned)

a) VERIZON METADATA

I see estimates of a trillion phone calls per year in the US. If each call has 80 characters of "metadata" (number called, duration, only a couple other things as we've been told), 80 trillion bytes would be 80 Terabytes/year. This "database" would easily sit 4-5 blade servers in someone's office. Furthermore if we are to trust what we were told just this week, only 300 numbers were targeted. My personal research data takes 20 TB so I can say with certainty that the project officially described is easily capable of being run out of a single office room on a single server by a team of 4-6 people.

This number is completely consistent with other estimates (the guy who runs the Internet Archive, for example) who says even content is not that expensive to store.

http://blog.archive.org/2013/06/15/cost-to-store-all-us-phonecalls-made-in-a-year-in-cloud-storage-so-it-could-be-datamined/

b) BOUNDLESS INFORMANT

Another metadata project, but basically traffic analysis from computer networks... By these numbers, it is similar in scope to the VERIZON estimate I made:

"A fact sheet leaked to the Guardian explains that almost 3bn pieces of intelligence had been collected from US computer networks in the 30-day period ending in March this year, as well as indexing almost 100bn pieces worldwide. "

So 36 billion US bits of metadata and 1.2 Trillion worldwide metadata entries. Scales pretty much like VERIZON, so add another office, another server (which fits in the same rack as the first one) and another 4-6 people.

c) PRISM

All foreign communication (emails, chat logs, other data from nine internet companies). From http://www.washingtontimes.com/news/2013/jun/7/prism-used-collect-personal-web-data-clapper-says/ we find a mathematician, William Binney who estimates (yeah, guesses) PRISM has 20 trillion emails/phone calls. At 100k/email (http://searchengineland.com/google-web-report-average-page-size-320-kb-46316) and 3 MB/phone call (3 min average per http://adraughtofvintage.com/2012/05/07/average-length-of-local-cell-phone-call-in-2003-was-3-min-in-2010-its-1-min-47-sec/) and estimating about a 10-1 ratio (going by http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper_c11-481360_ns827_Networking_Solutions_White_Paper.html and comparing to a phone traffic average above) we come up with an estimated size of about 10 exabytes.

Old estimates (2008) indicate that 2 exabytes could be done in 40,000 square feet of server space per http://storagemojo.com/2008/10/12/building-a-18-exabyte-data-center/ and we need five times that, but racks and disk capacity have improved by about the same proportion -- So PRISM sits (at Fort Meade?) in a single storage facility. http://singularityhub.com/2009/11/03/enter-the-yottabyte-one-billion-petabytes/ estimates that Google stores on the order of exabytes in a single data center. So PRISM is a single data center.

SO -- IF IT STOPPED WITH WHAT WE WERE OFFICIALLY TOLD THAT WOULD BE IT IT WOULD FIT IN A SINGLE COMPLEX

1) Why do we need a bunch of new multi-billion buildings in Utah capable of holding AT A MINIMUM 5 zettabytes (10 years of ALL TRAFFIC by a quick pencil and paper calculation) up through "yottabytes" if the program truly is this small?

http://en.wikipedia.org/wiki/Utah_Data_Center

This is an expansion of the capacity at San Antonio, Fort Meade, Fort Gordon in Georgia, NSA Hawaii. http://en.wikipedia.org/wiki/National_Security_Agency

2) Why are there stories about the U.S. government being interested in all kinds of commercial databases?

http://www.bloomberg.com/news/2013-06-14/u-s-agencies-said-to-swap-data-with-thousands-of-firms.html

3) Why does a quick perusal of the DARPA website talk about opportunities to do "anomaly detection" in massive databases.

http://www.darpa.mil/Our_Work/I2O/Programs/Anomaly_Detection_at_Multiple_Scales_%28ADAMS%29.aspx

CONCLUSION

Someone, somewhere made the successful sales pitch to these guys that you could build a system like the ones we see on CSI on teevee. When you know the name of the bad guy, just press a button and those massive floor to ceiling monitors on the "command center" wall pops up their picture and every significant bit of data ever collected on that person like you'd had a gumshoe on their tails for a decade. Worse, there are probably promises that you can take all that data, put it in a pot, wave a magic wand and out pops all the bad guys (people who don't fit the mean behaviors of the population). Once again, we are all suspects who will need to prove our innocence rather than them proving our guilt.

At the very least there is far more capacity available than publically disclosed uses.