[TBD]Understanding Twitter API

Fetching Tweets for a user with anonymous (TODO: Need to try)

Ref: https://github.com/twintproject/twint

twython: Wrapper python library

http://2017.compciv.org/guide/topics/python-nonstandard-libraries/twython-guide/twitter-twython-api-basics.html

Tweet projects

Twint

https://github.com/twintproject/twint

TWARC

http://digitalcollecting.lib.virginia.edu/toolkit/docs/social-media/twarc-commands/#retweets ,

https://github.com/DocNow/twarc/blob/master/README.md

https://github.com/docnow/twarc

Twecol

https://github.com/jdevoo/twecoll

https://raw.githubusercontent.com/lucahammer/twecoll3/master/twecoll3.py

Comparison among tools

https://docs.ropensci.org/rtweet/

Twitter response codes

https://developer.twitter.com/en/docs/basics/response-codes

Fetching Tweet for given ID

Ref: https://stackoverflow.com/questions/897107/can-i-fetch-the-tweet-from-twitter-if-i-know-the-tweets-id

For getting list of users(IDs) who has retweeted the given Tweet ID

https://developer.twitter.com/en/docs/tweets/post-and-engage/api-reference/get-statuses-retweeters-ids

https://developer.twitter.com/en/docs/tweets/post-and-engage/api-reference/post-statuses-retweet-id

For getting actual retweet ID, use Twlet chrome extension

http://twlets.com/

Python code to fetch retweet IDs

https://github.com/ideoforms/python-twitter-examples/blob/master/twitter-list-retweets.py

API to fetch tweets for a given user

https://stackoverflow.com/questions/9218730/what-is-the-max-number-of-tweets-you-can-get-from-the-twitter-statuses-user-tim

API to check if X can dm to Y or not (can_dm field)

I didn't see any rate limit when I used single thread

Authentication can be any

https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-friendships-show

https://docs.rs/egg-mode/0.8.0/egg_mode/user/struct.RelationSource.html

Example for any two user relationship

Request:

'https://api.twitter.com/1.1/friendships/show.json?source_screen_name=emileifrem&target_screen_name=deepadoc&count=200&lang=en&cursor=-1'

Response:

{'relationship': {'source': {'id': 14684110, 'id_str': '14684110', 'screen_name': 'emileifrem', 'following': False, 'followed_by': False, 'live_following': False, 'following_received': None, 'following_requested': None, 'notifications_enabled': None, 'can_dm': True, 'blocking': None, 'blocked_by': None, 'muting': None, 'want_retweets': None, 'all_replies': None, 'marked_spam': None}, 'target': {'id': 135102232, 'id_str': '135102232', 'screen_name': 'deepadoc', 'following': False, 'followed_by': False, 'following_received': None, 'following_requested': None}}}

Python code to check if I can DM (direct message) to a user (include_can_dm)

https://gist.github.com/Beomi/9d263bf9d1128180e1c17c1e94b0409b

Response will have can_dm field with example ("can_dm" : false ,)

Example:

Example Request URL for Can DM

Request URL:

https://api.twitter.com/1.1/dm/conversation/135102232-163170036.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_composer_source=true&include_ext_alt_text=true&include_reply_count=1&tweet_mode=extended&dm_users=false&include_groups=true&include_inbox_timelines=true&include_ext_media_color=true&supports_reactions=true&include_conversation_info=true&context=FETCH_DM_CONVERSATION&ext=mediaColor%2CaltText%2CmediaStats%2ChighlightedLabel%2CcameraMoment

To know current rate limit

https://developer.twitter.com/en/docs/basics/rate-limiting

https://api.twitter.com/1.1/application/rate_limit_status.json

Reference: https://blog.ailand.date/2020/02/27/howToCrawlTwitter/ (translate it)

To know about operators in search query

https://developer.twitter.com/en/docs/tweets/rules-and-filtering/overview/standard-operators

https://developer.twitter.com/en/docs/tweets/rules-and-filtering/guides/build-standard-queries

To know about csvql for CSV file processing

https://inkdroid.org/2017/08/31/retweets/

For appauth to improve the ratelimit

https://bhaskarvk.github.io/2015/01/how-to-use-twitters-search-rest-api-most-effectively./

Reference

https://developer.twitter.com/en

https://developer.twitter.com/en/docs/api-reference-index

https://github.com/twintproject/twint

https://raw.githubusercontent.com/lucahammer/twecoll3/master/twecoll3.py