Welcome to DU!
The truly grassroots left-of-center political community where regular people, not algorithms, drive the discussions and set the standards.
Join the community:
Create a free account
Support DU (and get rid of ads!):
Become a Star Member
Latest Breaking News
Editorials & Other Articles
General Discussion
The DU Lounge
All Forums
Issue Forums
Culture Forums
Alliance Forums
Region Forums
Support Forums
Help & Search
General Discussion
In reply to the discussion: More crackpot thinking from Glenn Greenwald [View all]leveymg
(36,418 posts)93. Large core components of TIA were shifted to NSA and DNI, and never ended.
http://en.wikipedia.org/wiki/Information_Awareness_Office
Going backwards, related programs included Trailblazer, an NSA program that focused on interception and analysis of data carried on web communications networks, cell phones, VOIP, and e-mail. After receiving adverse publicity Trailblazer was shutdown but reportedly morphed into the NSA Turbulance Program. Thin Thread was a rival NSA program that went operational, resulting in massive domestic surveillance. This is described by Jane Mayer in a 2011 New Yorker article: http://www.newyorker.com/reporting/2011/05/23/110523fa_fact_mayer?currentPage=all
Binney described Thin Thread to Mayer, who describes The Program this way:
In addition to NSA's Thinthread and Trailblazer programs, DIA operated its own domestic-focused pre-9/11 surveillance program. At least one of these monitored AQ cells operating inside the US. The story of the Able Danger has been well documented and fairly widely known. Able Danger was shut down by Undersecretary of Defense for Intelligence Steve Cambone in late 2000-early 2001. That operation built up a social network analysis focusing electronic surveillance on members of the so-called Brooklyn Cell that had remained in place after its establishment by CIA as part of Operation Cyclone, the Agency's operation that recruited and trained Jihadis for war against the Soviets in Afghanistan. Much of bin Laden's organization grew out of the US and Saudi organized covert operations against the Russia and its allies in Central Asia and the oil-rich region of the Transcaucasus that flared up again as wars in Bosnia, Kosovo, Dagestan, and Chechnya.
As early as 1997, as the US and Saudi paramilitary organized by bin Laden were cooperating in Kosovo, US intelligence officials were bragging that they had "mapped out" bin Laden's financial and donor network by human and technical means. Al Qaeda was already the focus of multiple surveillance operations inside the US, but this was gravely complicated by intelligence agencies from several countries stumbling over each other and the fatal duplicity of their network of interwoven double-agents that created the opportunity for the 9/11 attacks.
NSA and DIA technical collection and analysis, elsewhere referred as "The Program" survived the reorganization of intelligence that followed 9/11, and the closing down some legacy programs that followed a series of disclosures and scandals involving "The Program" in the middle of the decade. Part of this was described by Shane Harris in The National Journal, "TIA LIVES ON":
February 23, 2006, http://nationaljournal.com/about/njweekly/stories/2006/0223nj1.htm or http://mediachannel.org/blog/node/3509
Components of TIA projects that continue to be developed
Despite the withdrawal of funding for the TIA and the closing of the IAO, the core of the project survived.[3][4][25] Legislators included a classified annex to the Defense Appropriations Act that preserved funding for TIA's component technologies, if they were transferred to other government agencies. TIA projects continued to be funded under classified annexes to Defense and Intelligence appropriation bills. However, the act also stipulated that the technologies only be used for military or foreign intelligence purposes against foreigners.[26]
TIA's two core projects are now operated by Advanced Research and Development Activity (ARDA) located among the 60-odd buildings of "Crypto City" at NSA headquarters in Fort Meade, MD. ARDA itself has been shifted from the NSA to the Disruptive Technology Office (run by to the Director of National Intelligence). They are funded by National Foreign Intelligence Program for foreign counterterrorism intelligence purposes.
One technology, codenamed "Basketball" is the Information Awareness Prototype System, the core architecture to integrate all the TIA's information extraction, analysis, and dissemination tools. Work on this project is conducted by SAIC through its former Hicks & Associates consulting arm run by former Defense and military officials and which had originally been awarded US$19 million IAO contract to build the prototype system in late 2002.[27]
Despite the withdrawal of funding for the TIA and the closing of the IAO, the core of the project survived.[3][4][25] Legislators included a classified annex to the Defense Appropriations Act that preserved funding for TIA's component technologies, if they were transferred to other government agencies. TIA projects continued to be funded under classified annexes to Defense and Intelligence appropriation bills. However, the act also stipulated that the technologies only be used for military or foreign intelligence purposes against foreigners.[26]
TIA's two core projects are now operated by Advanced Research and Development Activity (ARDA) located among the 60-odd buildings of "Crypto City" at NSA headquarters in Fort Meade, MD. ARDA itself has been shifted from the NSA to the Disruptive Technology Office (run by to the Director of National Intelligence). They are funded by National Foreign Intelligence Program for foreign counterterrorism intelligence purposes.
One technology, codenamed "Basketball" is the Information Awareness Prototype System, the core architecture to integrate all the TIA's information extraction, analysis, and dissemination tools. Work on this project is conducted by SAIC through its former Hicks & Associates consulting arm run by former Defense and military officials and which had originally been awarded US$19 million IAO contract to build the prototype system in late 2002.[27]
Going backwards, related programs included Trailblazer, an NSA program that focused on interception and analysis of data carried on web communications networks, cell phones, VOIP, and e-mail. After receiving adverse publicity Trailblazer was shutdown but reportedly morphed into the NSA Turbulance Program. Thin Thread was a rival NSA program that went operational, resulting in massive domestic surveillance. This is described by Jane Mayer in a 2011 New Yorker article: http://www.newyorker.com/reporting/2011/05/23/110523fa_fact_mayer?currentPage=all
Code-named ThinThread, it had been developed by technological wizards in a kind of Skunk Works on the N.S.A. campus. Formally, the project was supervised by the agencys Signals Intelligence Automation Research Center, or SARC.
While most of the N.S.A. was reeling on September 11th, inside SARC the horror unfolded almost like an I-told-you-so moment, according to J. Kirk Wiebe, an intelligence analyst who worked there. We knew we werent keeping up. SARC was led by a crypto-mathematician named Bill Binney, whom Wiebe describes as one of the best analysts in history. Binney and a team of some twenty others believed that they had pinpointed the N.S.A.s biggest problemdata overloadand then solved it. But the agencys management hadnt agreed.
Binney, who is six feet three, is a bespectacled sixty-seven-year-old man with wisps of dark hair; he has the quiet, tense air of a preoccupied intellectual. Now retired and suffering gravely from diabetes, which has already claimed his left leg, he agreed recently to speak publicly for the first time about the Drake case. When we met, at a restaurant near N.S.A. headquarters, he leaned crutches against an extra chair. This is too serious not to talk about, he said.
Binney expressed terrible remorse over the way some of his algorithms were used after 9/11. ThinThread, the little program that he invented to track enemies outside the U.S., got twisted, and was used for both foreign and domestic spying: I should apologize to the American people. Its violated everyones rights. It can be used to eavesdrop on the whole world. According to Binney, Drake took his side against the N.S.A.s management and, as a result, became a political target within the agency.
While most of the N.S.A. was reeling on September 11th, inside SARC the horror unfolded almost like an I-told-you-so moment, according to J. Kirk Wiebe, an intelligence analyst who worked there. We knew we werent keeping up. SARC was led by a crypto-mathematician named Bill Binney, whom Wiebe describes as one of the best analysts in history. Binney and a team of some twenty others believed that they had pinpointed the N.S.A.s biggest problemdata overloadand then solved it. But the agencys management hadnt agreed.
Binney, who is six feet three, is a bespectacled sixty-seven-year-old man with wisps of dark hair; he has the quiet, tense air of a preoccupied intellectual. Now retired and suffering gravely from diabetes, which has already claimed his left leg, he agreed recently to speak publicly for the first time about the Drake case. When we met, at a restaurant near N.S.A. headquarters, he leaned crutches against an extra chair. This is too serious not to talk about, he said.
Binney expressed terrible remorse over the way some of his algorithms were used after 9/11. ThinThread, the little program that he invented to track enemies outside the U.S., got twisted, and was used for both foreign and domestic spying: I should apologize to the American people. Its violated everyones rights. It can be used to eavesdrop on the whole world. According to Binney, Drake took his side against the N.S.A.s management and, as a result, became a political target within the agency.
Binney described Thin Thread to Mayer, who describes The Program this way:
ThinThread would correlate data from financial transactions, travel records, Web searches, G.P.S. equipment, and any other attributes that an analyst might find useful in pinpointing the bad guys. By 2000, Binney, using fibre optics, had set up a computer network that could chart relationships among people in real time. It also turned the N.S.A.s data-collection paradigm upside down. Instead of vacuuming up information around the world and then sending it all back to headquarters for analysis, ThinThread processed information as it was collecteddiscarding useless information on the spot and avoiding the overload problem that plagued centralized systems. Binney says, The beauty of it is that it was open-ended, so it could keep expanding.
Pilot tests of ThinThread proved almost too successful, according to a former intelligence expert who analyzed it. It was nearly perfect, the official says. But it processed such a large amount of data that it picked up more Americans than the other systems. Though ThinThread was intended to intercept foreign communications, it continued documenting signals when a trail crossed into the U.S.
< . . .>
Binney, for his part, believes that the agency now stores copies of all e-mails transmitted in America, in case the government wants to retrieve the details later. In the past few years, the N.S.A. has built enormous electronic-storage facilities in Texas and Utah. Binney says that an N.S.A. e-mail database can be searched with dictionary selection, in the manner of Google. After 9/11, he says, General Hayden reassured everyone that the N.S.A. didnt put out dragnets, and that was true. It had no needit was getting every fish in the sea.
Pilot tests of ThinThread proved almost too successful, according to a former intelligence expert who analyzed it. It was nearly perfect, the official says. But it processed such a large amount of data that it picked up more Americans than the other systems. Though ThinThread was intended to intercept foreign communications, it continued documenting signals when a trail crossed into the U.S.
< . . .>
Binney, for his part, believes that the agency now stores copies of all e-mails transmitted in America, in case the government wants to retrieve the details later. In the past few years, the N.S.A. has built enormous electronic-storage facilities in Texas and Utah. Binney says that an N.S.A. e-mail database can be searched with dictionary selection, in the manner of Google. After 9/11, he says, General Hayden reassured everyone that the N.S.A. didnt put out dragnets, and that was true. It had no needit was getting every fish in the sea.
In addition to NSA's Thinthread and Trailblazer programs, DIA operated its own domestic-focused pre-9/11 surveillance program. At least one of these monitored AQ cells operating inside the US. The story of the Able Danger has been well documented and fairly widely known. Able Danger was shut down by Undersecretary of Defense for Intelligence Steve Cambone in late 2000-early 2001. That operation built up a social network analysis focusing electronic surveillance on members of the so-called Brooklyn Cell that had remained in place after its establishment by CIA as part of Operation Cyclone, the Agency's operation that recruited and trained Jihadis for war against the Soviets in Afghanistan. Much of bin Laden's organization grew out of the US and Saudi organized covert operations against the Russia and its allies in Central Asia and the oil-rich region of the Transcaucasus that flared up again as wars in Bosnia, Kosovo, Dagestan, and Chechnya.
As early as 1997, as the US and Saudi paramilitary organized by bin Laden were cooperating in Kosovo, US intelligence officials were bragging that they had "mapped out" bin Laden's financial and donor network by human and technical means. Al Qaeda was already the focus of multiple surveillance operations inside the US, but this was gravely complicated by intelligence agencies from several countries stumbling over each other and the fatal duplicity of their network of interwoven double-agents that created the opportunity for the 9/11 attacks.
NSA and DIA technical collection and analysis, elsewhere referred as "The Program" survived the reorganization of intelligence that followed 9/11, and the closing down some legacy programs that followed a series of disclosures and scandals involving "The Program" in the middle of the decade. Part of this was described by Shane Harris in The National Journal, "TIA LIVES ON":
February 23, 2006, http://nationaljournal.com/about/njweekly/stories/2006/0223nj1.htm or http://mediachannel.org/blog/node/3509
As early as February 2003, the Pentagon planned to use Genoa II technologies at the Army's Information Awareness Center at Fort Belvoir, Va., according to an unclassified Defense budget document. The awareness center was an early tester of various TIA tools, according to former employees. A 2003 Pentagon report to Congress shows that the Army center was part of an expansive network of intelligence agencies, including the NSA, that experimented with the tools. The center was also home to the Army's Able Danger program, which has come under scrutiny after some of its members said they used data-analysis tools to discover the name and photograph of 9/11 ringleader Mohamed Atta more than a year before the attacks.
The other project has been re-designated "TopSail" (formerly Genoa II) and would provide IT tools to help anticipate and preempt terrorist attacks. SAIC has also been contracted to work on Topsail, including a US$3.7 million contract in 2005.
The other project has been re-designated "TopSail" (formerly Genoa II) and would provide IT tools to help anticipate and preempt terrorist attacks. SAIC has also been contracted to work on Topsail, including a US$3.7 million contract in 2005.
Edit history
Please sign in to view edit histories.
Recommendations
0 members have recommended this reply (displayed in chronological order):
99 replies
= new reply since forum marked as read
Highlight:
NoneDon't highlight anything
5 newestHighlight 5 most recent replies
RecommendedHighlight replies with 5 or more recommendations
You're overlooking the technical problems of constantly streaming 4 million conversations
struggle4progress
May 2013
#8
Google precesses 24 petabytes/day. That's one company. Apparently you can't grasp that reality.
leveymg
May 2013
#11
Google does process about a billion queries daily. But they all arrive at the company
struggle4progress
May 2013
#24
The scale of data is roughly the same: Google 24 petabytes/day; ATT network 19 petabytes/day
leveymg
May 2013
#26
Correction, Google does store web pages. check the "cache" link on most search results.
bobduca
May 2013
#91
Yeah, you're right. My mistake. But google doesn't save the whole history of that page:
struggle4progress
May 2013
#95
I think your point basically stands, but the numbers (while still large) are a bit off...
Silent3
May 2013
#2
The OP is WAY, WAY off. At 60KB/minute storage for voice audio, 2.2 trillion min/yr requires
leveymg
May 2013
#7
It depends on the use you plan to make of the recording. It may be that by reducing quality
struggle4progress
May 2013
#12
One shouldn't confuse the radio channel signal with the conversation. When cell phone companies
struggle4progress
May 2013
#82
Nothing but very compressed data ever gets out of your phone into the air, however
Silent3
May 2013
#90
Greenwald seems to be repeating a claim made by a former FBI counter-terrorism agent.
Jim__
May 2013
#4
He indeed reports the claim, then he repeats it in manner indicating that he accepts it:
struggle4progress
May 2013
#14
Google's servers process 24 petabytes per day. That would leave a lot of spare capacity to process
leveymg
May 2013
#10
If you're recording conversations for intelligence analysis purposes, you want the best quality
struggle4progress
May 2013
#20
I wrote a voice recording app for the Commodore 128 back when it was the latest thing
Fumesucker
May 2013
#21
If you've got 4 million conversations coming in at every minute, and you're gonna listen to
struggle4progress
May 2013
#16
No, his claim was that all calls were recorded and stored. I'm simply doing the math here on what
struggle4progress
May 2013
#22
I was responding to a comment made in #6: please do read subthreads when commenting
struggle4progress
May 2013
#25
I didn't argue any such thing: I've actually pointed out that there are too many to listen to
struggle4progress
May 2013
#36
It is of course true, and always has been true, that there are unprincipled people
struggle4progress
May 2013
#43
So where is the proof? All you offer is speculation and with it you are smearing a good name.
cui bono
May 2013
#56
GG's PoV seems to be that if a "former FBI agent" suggests the Administration
struggle4progress
May 2013
#86
You'll never find "most Americans" loudly and vocally oppose anything political. Most Americans
cui bono
May 2013
#88
Well, Tim Clemente is getting lots of attention for making this claim, but actually
struggle4progress
May 2013
#40
OMG. And DU posters are making claims that it isn't true and there's still no reason to believe
cui bono
May 2013
#53
There are currently something like 14K FBI employees. The FBI has no statutory authority
struggle4progress
May 2013
#84
And you're refuting what the FBI agent says based on your ideas of what is possible digitally which
cui bono
May 2013
#87
We've been hearing about this for years. Not sure why the OP is freaking out about the reporting now
cui bono
May 2013
#58
Your theory requires the view that ISPs, landline companies, and wireless communication companies
struggle4progress
May 2013
#39
Installation of the intercept equipment by telcos has been mandatory under CALEA since 1994
leveymg
May 2013
#44
Do you know how a "driftnet" warrant of the type legalized by the '08 Amendment works?
leveymg
May 2013
#69
The EFF suit against Bush Admin officials is still alive, the ATT case was dismissed.
leveymg
May 2013
#74
I did not locate any further FR notices establishing capacity requirements. Following links from one
struggle4progress
May 2013
#83
Oh, the "national security" establishment is certainly out of control: it's been oversized
struggle4progress
May 2013
#42
Redundancy and waste are only two of the three threats: There is institutional jealousy and lust for
byeya
May 2013
#45
Institutional competitiveness can be a good thing for democracy: it means none of them
struggle4progress
May 2013
#48
I think having 2, or more, agencies with the same mission is a waste of money and Cabinent
byeya
May 2013
#57
They tell us they are spying on us to "protect" us from tyranny. Of course, they miss the irony.
Tierra_y_Libertad
May 2013
#32
You hit it on the head. This post is propaganda that works with other similar smears on this board
leveymg
May 2013
#63
The old left activists I knew decades ago would never discuss any concrete plans over the phone
struggle4progress
May 2013
#79
I used to know a guy whose job it was to censor the Western press in Poland.
JDPriestly
May 2013
#81