Welcome to DU! The truly grassroots left-of-center political community where regular people, not algorithms, drive the discussions and set the standards. Join the community: Create a free account Support DU (and get rid of ads!): Become a Star Member Latest Breaking News Editorials & Other Articles General Discussion The DU Lounge All Forums Issue Forums Culture Forums Alliance Forums Region Forums Support Forums Help & Search

kurt_cagle

(534 posts)
1. Cool post and very true
Sat Mar 17, 2012, 04:46 PM
Mar 2012

I work a lot with Semantic systems - technologies that can make inferences based upon mountains of assertions, and that have featured extensively in TIA-type applications. Some of the technology is quite cool - there's been a fairly major advance in AI systems because of this, and such systems are also very useful for everything from Siri on the iPhone to advanced inferential search systems for Google and Microsoft.

However, what also emerged out of this was that the principle of Garbage In Garbage Out holds even more true for Semantics than it does elsewhere. For such a system to work, you need to make a lot of assertions about a lot of things, and to a certain extent you need to build those assertions as models with a number of underlying assumptions. The more accurate (and more timely) those assumptions, the higher the degree of probability that your inferences are correct, but getting accurate and timely assumptions in the first place is the problem. In other words, you can do TIA type analysis to determine the possible next moves of someone that you believe to be a terrorist in the first place, but if your initial assumptions are incorrect (a terrorist will be part of Al Quaida, therefore they will be Islamic and come from an Islamic country) then no amount of analysis is going to help. What's more, what TIA is looking for in general is a pattern, but anyone who's committed enough to be a terrorist will go out of their way not to establish a pattern in the first place. Being able to read "non-patterns" is an incredibly difficult task, because your false positives will be far, far higher than your hit rate.

I take away from this that the long term investment has certainly improved the overall semantic field, but that was largely an accidental byproduct of attempting to solve a problem which never really needed a resolution, especially because the use of it successfully would also have raised some major ethical issues.

Recommendations

0 members have recommended this reply (displayed in chronological order):

Latest Discussions»Editorials & Other Articles»Why NSA's Total Informati...»Reply #1