We had the opportunity to collect data using the Twitter API.
The Twitter API has a Search API for searching tweets, just like Google search, which I used this time.
I usually use Twitter mainly as a reader, so I was not really aware of its functions, However, when I tried to search for tweets using the Search API by specifying various conditions, I was confused by Twitter's terminology, functions, and structure.
Here is a summary of how to use Twitter's Search API to search for a desired tweet.
How to use the Twitter API is described in a separate article.
Twitter Composition
The Search API is for tweets that are publicly available externally.
A tweet that is published externally is composed of the following
There are "user_A" and "user_B" timelines, and user_A is "normal tweeting," "retweeting user_B," "quoting user_B," and "replying to user_B."
Tweet Type
There are four types of tweets: tweets, retweets, quote retweets, and replies.
tweet
- So-called normal tweets
- Display on the timeline of the person who tweeted it
retweet
- Display other people's tweets as tweets on your timeline
- On the tweet you want to retweet, press the retweet button and do it with "Retweet".
Quote Retweet
- Add your own comments to other people's tweets and have them appear as tweets on your timeline.
- In the tweet you want to quote retweet, press the retweet button and do it with "retweet with comment".
return letter
- Tweet in response to a tweet. It will appear in the thread of the tweet and in the timeline of the person who replied.
Twitter Data Structure
With Twitter's structure in mind, let's look at Twitter's data structure.
The "text" is the body of the tweet that will be searched for when searching by keywords and hashtags.
The other elements are "retweet," "quote retweet," and "reply," data added by the type of tweet, which allows us to identify the type of tweet from the data received.
Example of search conditions by purpose
Basic Writing
Basically, you write the string or condition you want to search for as a query string and pass it to the API, just as you would search on Google.
If there are multiple conditions, write them side by side, opened with a space, and they will be searched AND.
The NOT condition is prefixed by a "-" before the condition.
For example, to search for tweets containing "keyword 1" and "#hashtag1" but not "user 1", the query would be
キーワード１ #ハッシュタグ１ -from:ユーザー１
Below is a list of frequently performed searches, grouped by purpose.
Search by Keyword
キーワード_1 キーワード_2 ...
Search by hashtag
#ハッシュタグ_1 #ハッシュタグ_2 ...
Specific user's tutu
from:user_A
The user uses the display name under "@".
Reply to a specific user
to:user_A
Retweets and replies to specific users
@user_A
Note that "@" includes both retweets and replies.
Retweet only
filter:retweets
Retweets to specific users
@user_A filter:retweets
The "@" includes both retweets and replies, so if you want to exclude replies and search only for retweets, combine the "retweets only" filter condition.
Retweet Exclusion
-filter:retweet
Retweet is a copy of the original tweet, so to speak, so it is used when you want to exclude retweets and search purely for posts.
Cannot search for quoted retweets.
This is where we had the most trouble this time.
The "quote retweet" isretweetAlthough it is named as,This is not a retweet.
Trouble is, searching for retweets does not include quote retweets.
Even more troubling, there are no search criteria to narrow down quoted retweets like there are for retweets.
And as you can see from the data structure, the body of a retweet is the content of the retweeter's tweet, but for a quoted retweet, the comment is the body of the quoted retweet, and the quoted source tweet text is not included in the body.
What is troubling about this is that quoted retweets are not caught when searching by the content of the original tweet's text.
In summary...
If you're trying to search with the Search API and you're retweeting,
- Narrow your search to retweets
- Search excluding retweets
- Search by keyword or hashtag for retweet text
can be done, but in the case of a quote retweet
- Narrow your search to quote retweets
- Search excluding quoted retweets
- Search by keyword or hashtag for quoted text
is not possible.
In other words,
- Quoted retweets for specific tweets
- Quote retweet for a specific user
- Citation retweet by a specific user
I can't search for the
Search for quotes and retweets
However, only "quoted retweets to a specific tweet" can be searched by devising search criteria.
The Search API has a search condition that searches for URLs that are included in the search. By specifying a part of the URL of a quoted tweet here, you can search for quoted retweets of a specific tweet.
The URL of the tweet will appear in the browser address when you click on the tweet to open it, and will look something like the following
https://twitter.com/<@以下のユーザー名>/status/<id_str(数字のTweetID)>
The number "id_str" is the ID of the tweet and can be specified in the URL to search for quoted retweets of that tweet.
However, that would include retweets, so the quoted retweet would be the one that excludes retweets.
url:<id_str> -filter:retweets
Searches are ordered by most recent
There are free and paid versions of the Search API, but the free version can only search tweets from the last 7 days.
If there are many search results, paging is done, but since old tweets are not caught, the pages are ordered from the latest result or the last date specified.retroactive formThe page is sent to the next page.
Impressions, etc.
When we launched a large promotion, we could ask Twitter to tally the tweets, but it was difficult to say for a trivial tally, so we had no choice but to crawl the site on a regular basis.
It would have been nice if quoted retweets could be searched in the same way as retweets, but quoted retweets can only be searched in specific tweet units, which is troublesome.
I wish they would include a search criteria for quoted retweets....
