Twitter Says Its API Can't Be Used For Surveillance, But What Does It Think The FBI's Going To Do With It?

from the spies-in-sports-coats-or-just-LEOs? dept

Dataminr, the company whose Twitter firehose access has become somewhat of cause celebre on both sides of the privacy fence, is back in the news. After being told it couldn't sell this access to government agencies for surveillance purposes, Dataminr had to disconnect the CIA from its 500 million tweets-per-day faucet.

Twitter was pretty specific about what this buffed-up API could and could not be used for. The CIA's surveillance efforts were on the "Don't" list. This rejection of the CIA's access was linked to existing Twitter policies -- policies often enforced inconsistently or belatedly. What the CIA had access to was public tweets from public accounts -- something accessible to anyone on the web, albeit with a better front-end for managing the flow and an API roughly 100x more robust than those made available to the general public.

The question now is how Twitter defines surveillance.

The FBI will soon be able to search a vast repository of public tweets in real time for hints about potential terrorist attacks and other public-safety crises.

The bureau awarded a sole-source contract to Dataminr, a company that allows customers to churn through Twitter's "firehose," which includes more than 500 million 140-character messages posted daily. Twitter's public API only gives users access to about 1 percent of tweets, according to a FedBizOpps posting.

Now, the question is not whether or not the FBI should have access to publicly-available Tweets. It always will have that access, with or without Dataminr's assistance. The question is whether Twitter believes the FBI is not engaged in the sort of surveillance it disagrees with.

In the context of its Dataminr access, I'm sure the FBI would have preferred to be thought of as a law enforcement agency. Divorced from the API-access context, it has done much in recent years to place itself on the same level as the CIA. It honestly feels it should be given more foreign intelligence gathering powers -- more so than the CIA, which has traditionally handled only foreign-facing operations.

Likewise with the NSA. The NSA's bulk collection orders under Section 215 were obtained in the FBI's name, with the data going directly to the NSA and the intelligence agency "tipping" an unspecified amount of the haul back to the FBI for further examination.

What the FBI is going to engage in with this access will be a form of surveillance, albeit one with very few privacy implications. Twitter has yet to speak up about the recently-awarded contract. It may never do so. It may believe the FBI is primarily engaged in law enforcement, even though the agency rebranded in the midst of the Snowden leaks, emerging as the "national security" agency it apparently felt it always should have been.

The statement issued by Twitter suggests it's only the "surveillance" that bothers them, not so much what each government agency seeking access feels its core mission is. The policy says "government or intelligence agenc[ies]" will be forbidden from purchasing access for surveillance purposes and the FBI certainly can't deny it's a government agency.

It also shouldn't matter which hat the FBI wears when attaching the hose. Twitter yanked Geofeedia's API access after discovering it was selling access to law enforcement agencies all over the US for the purposes of tracking First Amendment-protected activity. Its policies also list "track" and "investigate" as problematic uses of its API -- two things the FBI does often.

Given the agency's long history of engaging in surveillance of protected political activity, it's not much of a stretch to believe the FBI will use Dataminr's tools for the same ends. Then again, Dataminr or no Dataminr, the tweets it's seeking to analyze are already out there where anyone can see them. All the agency is really buying is a hose and a funnel.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: cia, data, fbi, fire hose, social media, surveillance
Companies: dataminr, twitter

Reader Comments

Subscribe: RSS

View by: Thread

  • identicon
    WeHearYouTim, 22 Nov 2016 @ 3:35pm

    Interesting problem

    What do honest developers do to prevent unauthorized use of their APIs?

    Sue the government?
    First they would need to prove the government(s) used it, standing would be tough as IP addresses don't always resolve accordingly.

    Monitor then block IP ranges using the API?
    Then players would decry censorship.

    This is an interesting problem Tim.

    Might this also apply to Google, Facebook, Yahoo, Reddit, et. and any other site?

    Perhaps the approach to use today, is education.
    Twitter, educating users that any post they make will end up in multiple government archives, that there is no erase button on the internet and users should be very cautious about providing any information to online sites...

    reply to this | link to this | view in chronology ]

  • identicon
    Christenson, 22 Nov 2016 @ 3:43pm

    Hoses and Funnels

    Twitter's position seems to boil down to:
    You can't use dynamite if you want to hurt people!

    But dynamite doesn't know about just goes "Boom!"

    And I don't see twitter getting out of the "dynamite" business anytime soon, since one reasonable way to measure effectiveness of advertising is to ask how many tweet about the product. And I don't think twitter supervising everyone using their API can be cost effective either.

    Not that the FBI shouldn't *need* a warrant to use dynamite.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 22 Nov 2016 @ 3:52pm

    FBI -> CIA?

    I thought in recent years, things like the DHS and various bits of legislation were designed to provide more data sharing between government organizations? So if the FBI gets this feed, doesn't the CIA get it by extension?

    reply to this | link to this | view in chronology ]

  • identicon
    Capt ICE Enforcer, 22 Nov 2016 @ 4:01pm

    Trump FBI

    Donald Trump's first order as president is all government agencies must send 27 tweets per day. Why else would the FBI want access to Twitter

    reply to this | link to this | view in chronology ]

  • icon
    Coyne Tibbets (profile), 22 Nov 2016 @ 4:26pm

    The full statement

    "Trust us, our API can't be used for surveillance. The NSA made us say so."

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 22 Nov 2016 @ 5:17pm

    Wait a sec ... the government is in violation of a Terms Of Service? What about other computer crime laws they are probably in violation of?

    Oh yeah .. laws are only for the little people.

    reply to this | link to this | view in chronology ]

  • identicon
    Le Bizz, 22 Nov 2016 @ 9:06pm

    Just Thinkin out loud

    I was under the impression such agencies with such resources could easily build a distributed firehose all by themselves if Twitter decides not to play along...

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 23 Nov 2016 @ 1:29am

    What will the TLAs do when they discover that the bad guys have used bot nets to send them chasing red herrings? This gather it all mentality opens up such opportunities to divert their attention away from the real activities.

    reply to this | link to this | view in chronology ]

  • icon
    Ninja (profile), 23 Nov 2016 @ 1:36am

    Considering the messages are public anyway and that the soup letter acronym surveillance agencies will eventually build a solution that does what the API Dataminr offer do even if it means going through the hard way I don't think it really matters. The issue here is more complex and deep than that.

    The fact that law enforcement and intel agencies are actively monitoring the platform (and others) for Constitutionally protected activity is the real issue here. And the fact that they can do so and even harass those engaged in such protected activity just by labeling them terrorists should be the real concern, not the ability to screen all the messages conveniently and quickly.

    reply to this | link to this | view in chronology ]

  • icon
    DannyB (profile), 23 Nov 2016 @ 6:00am


    I'm sure Comey would assure us that the FBI would never use the data to undermine people's rights. The data would only be used for purely statistical purposes. In order to better understand what eyeballs to put warrants in front of. The FBI wants to protect everyone's first amendment right to agree with the FBI.

    reply to this | link to this | view in chronology ]

  • identicon
    wiserabbit, 23 Nov 2016 @ 7:06am

    I'm confused.

    Twitter says "surveillance", tracking and investigation are no-no's.

    a. What else does the FBI do?
    b. What else do all of the marketing companies do?

    c. Boil it down, why else would anyone want to use the API?

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 23 Nov 2016 @ 10:50am

    Dear Tim,

    "All the agency is really buying is a hose and a funnel."

    Uhm... Beside that Mrs. Lincoln, how was the play?


    Although I appreciate the overall tone of this piece, perhaps you might consider not so playing down the significance of the implications of the FBI having a much more powerful Twitter UI than is available to the general public. Because in doing so, you're flirting with the same reasoning used by law enforcement to justify ALPR (i.e., because anyone in public can see your license plate, we are justified in the mass/automated collection, indefinite storage, and incredibly powerful search UI of license plate location data - two scenarios which couldn't be more different).

    The same is true with providing the FBI with these types of very, very powerful collection and search UI's. Because the general public has little to no reference point for understanding how their tweets can be accumulated over time and aggregated with all their other digital data, they are put at a significant disadvantage in understanding the implications of how such information can be used against their best interests. And that is the crux of the the public good issue. I'd appreciate it if going forward, you wouldn't minimize its significance with such dismissive statements.



    reply to this | link to this | view in chronology ]

  • identicon
    John Cressman, 23 Nov 2016 @ 11:46am

    I wish...

    I wish I lived in a fantasy world with ice cream mountains, lollipop forests, purple gummy unicorns and APIs that the government wouldn't abuse.

    Sadly... we live in the real world and if the governments don't use Twitter's API, they'll just find some other way.

    Or like China, the government will simply tell them what the company will provide if they want to do business in their country.

    reply to this | link to this | view in chronology ]

  • icon
    Groaker (profile), 23 Nov 2016 @ 1:08pm

    There is a portion of the population which believes that the FBI will behave with absolute morality. In full alignment with all legal requirements.

    At least a portion of the FBI has a belief it will find top level agents and terrorists communicating through twitter and the like.

    The two groups are intellectually congruent.

    reply to this | link to this | view in chronology ]

Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here

Subscribe to the Techdirt Daily newsletter

Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)


Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here

Subscribe to the Techdirt Daily newsletter

Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Special Affiliate Offer

Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it

Email This

This feature is only available to registered users. Register or sign in to use it.