General Discussion
In reply to the discussion: How does "data may be queried only when there is a reasonable suspicion" = 20 million/month? [View all]bemildred
(90,061 posts)Although, once again, I find that idea questionable, it means they are naive. The right data will improve your queries, more useless data will slow them down.
They are using the database system heuristically, profiling with it, looking for things, which is a very clumsy way to do things, and almost certainly implies failure with their human intelligence assets. Which sounds about right, now I think of it.
Edit: what I am saying is this is in some ways a very Rube Goldberg operation, they have gone to a good deal of trouble to patch these technologies together in a most hurried and inefficient way so they could search all of this data NOW. That's the point, to search it all, NOW, there is no other reason to grab it, or to build storage for it. I imagine the point of the Utah project is to fix some of those things and grab even more to search even faster looking for things, because it would take way-the-hell too long to implement something better.