-
-
Notifications
You must be signed in to change notification settings - Fork 525
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace tokens with guest accounts, swap endpoints #985
base: master
Are you sure you want to change the base?
Conversation
RSS returns forbidden, was this disabled? |
|
hi @zedeus, this is amazing, just a question, how is guest_accounts.json formatted? |
It's a JSON array of the raw object you get from guest account flow, looks like this (you can use this one for testing if you want): [{"user":{"id":1693553468835831800,"id_str":"1693553468835831808","name":"Open App User","screen_name":"_LO_082193WA35n","user_type":"Soft"},"next_link":{"link_type":"subtask","link_id":"next_link","subtask_id":"OpenAppFlowStartAccountSetupOpenLink"},"oauth_token":"1693553468835831808-TBis5qLp17jenKVyu9jeRjnTcTKjpL","oauth_token_secret":"BvbmNeJucYaUAWoEzrSUzTjkWxifoYaqAnjXTevL6ICxU","attribution_event":"signup"}] |
Great! thanks a lot, it's the format from BANKA's script in an array. |
I'm getting an error on compilation with this branch though, do you know anything about it? |
This happens with Nim versions older than 1.6.14, I'll fix it soon, for now you can work around it by updating to 1.6.14 or preferably 2.0 |
For the record: I successfully built a Docker image for aarch64 from this branch and it works like a charm :) I changed the base image to use alpine 3.18 which has Nim 1.6.14 |
Sorry for the long post, couple of questions just for my own learning:
|
I used a virtual machine to listen to network requests, but most of the reverse engineering work was done by other folks.
I'm not sure what you mean since that's how it already works, storing guest accounts in a file. No endpoints are being used beyond the initial creation of the account, which expires after 30 days. When you say blocked, do you mean you get an error code 88 (rate limit exceeded)? That block goes away 24 hours later. |
Would you mind sharing what the kind of vm you use for this? something like wireshark or mitmproxy? |
I used the free trial from Corellium to run Twitter and inspect network logs. I tried running BlissOS via qemu to no avail first. I just noticed the thing you posted is a normal account, didn't look close enough. What I said still stands for guest accounts, they don't get blocked though just limited for 24 hours. The problem with burner accounts is you need a lot for any meaningfully big instance, so it's not really feasible especially since they just get locked. It's extremely easy to get guest accounts, just takes a lot of proxies which are easy to get. |
Great thank you! |
Indeed, but that's the case with every solution. With accounts they can start banning or further restricting, there is no perfect solution. |
How to upgrade to this version?
|
Correct, but you also need to get one or more guest accounts, otherwise nothing will work |
I have this running on my instance and so far it's running great, haven't seen any issues so far. Getting gust tokens definitely the trickiest part, but being as Im just running a private/relay instance one guest user should cover it. Thanks for getting this put together! |
Is it possible to run this branch with an actual account? What are the tokens I need to extract from the real account to create the JSON file? |
It might be possible but I haven't tried, there's no code in this branch for regular accounts as they use cookies and bearer tokens etc instead of oauth. |
What's the status of this? Can nitter itself at least run the guest script itself once a day to replenish the tokens? |
Also improve icon rendering on Firefox
I hacked myself a small script to auto generate guest accounts with the #!/bin/bash
# Grab Twitter guest account tokens for use with Nitter.
guest_token=$(curl -s -XPOST https://api.twitter.com/1.1/guest/activate.json -H 'Authorization: Bearer AAAAAAAAAAAAAAAAAAAAAFXzAwAAAAAAMHCxpeSDG1gLNLghVe8d74hl6k4%3DRUMF4xAQLsbeBhTSRrCiQpJtxoGWeyHrDb5te2jpGskWDFW82F' | jq -r '.guest_token')
flow_token=$(curl -s -XPOST 'https://api.twitter.com/1.1/onboarding/task.json?flow_name=welcome&api_version=1&known_device_token=&sim_country_code=us' \
-H 'Authorization: Bearer AAAAAAAAAAAAAAAAAAAAAFXzAwAAAAAAMHCxpeSDG1gLNLghVe8d74hl6k4%3DRUMF4xAQLsbeBhTSRrCiQpJtxoGWeyHrDb5te2jpGskWDFW82F' \
-H 'Content-Type: application/json' \
-H 'User-Agent: TwitterAndroid/9.95.0-release.0 (29950000-r-0) ONEPLUS+A3010/9 (OnePlus;ONEPLUS+A3010;OnePlus;OnePlus3;0;;1;2016)' \
-H 'X-Twitter-API-Version: 5' \
-H 'X-Twitter-Client: TwitterAndroid' \
-H 'X-Twitter-Client-Version: 9.95.0-release.0' \
-H 'OS-Version: 28' \
-H 'System-User-Agent: Dalvik/2.1.0 (Linux; U; Android 9; ONEPLUS A3010 Build/PKQ1.181203.001)' \
-H 'X-Twitter-Active-User: yes' \
-H "X-Guest-Token: ${guest_token}" \
-d '{"flow_token":null,"input_flow_data":{"country_code":null,"flow_context":{"start_location":{"location":"splash_screen"}},"requested_variant":null,"target_user_id":0},"subtask_versions":{"generic_urt":3,"standard":1,"open_home_timeline":1,"app_locale_update":1,"enter_date":1,"email_verification":3,"enter_password":5,"enter_text":5,"one_tap":2,"cta":7,"single_sign_on":1,"fetch_persisted_data":1,"enter_username":3,"web_modal":2,"fetch_temporary_password":1,"menu_dialog":1,"sign_up_review":5,"interest_picker":4,"user_recommendations_urt":3,"in_app_notification":1,"sign_up":2,"typeahead_search":1,"user_recommendations_list":4,"cta_inline":1,"contacts_live_sync_permission_prompt":3,"choice_selection":5,"js_instrumentation":1,"alert_dialog_suppress_client_events":1,"privacy_options":1,"topics_selector":1,"wait_spinner":3,"tweet_selection_urt":1,"end_flow":1,"settings_list":7,"open_external_link":1,"phone_verification":5,"security_key":3,"select_banner":2,"upload_media":1,"web":2,"alert_dialog":1,"open_account":2,"action_list":2,"enter_phone":2,"open_link":1,"show_code":1,"update_users":1,"check_logged_in_account":1,"enter_email":2,"select_avatar":4,"location_permission_prompt":2,"notifications_permission_prompt":4}}' | jq -r .flow_token)
curl -s -XPOST 'https://api.twitter.com/1.1/onboarding/task.json' \
-H 'Authorization: Bearer AAAAAAAAAAAAAAAAAAAAAFXzAwAAAAAAMHCxpeSDG1gLNLghVe8d74hl6k4%3DRUMF4xAQLsbeBhTSRrCiQpJtxoGWeyHrDb5te2jpGskWDFW82F' \
-H 'Content-Type: application/json' \
-H 'User-Agent: TwitterAndroid/9.95.0-release.0 (29950000-r-0) ONEPLUS+A3010/9 (OnePlus;ONEPLUS+A3010;OnePlus;OnePlus3;0;;1;2016)' \
-H 'X-Twitter-API-Version 5' \
-H 'X-Twitter-Client: TwitterAndroid' \
-H 'X-Twitter-Client-Version: 9.95.0-release.0' \
-H 'OS-Version: 28' \
-H 'System-User-Agent: Dalvik/2.1.0 (Linux; U; Android 9; ONEPLUS A3010 Build/PKQ1.181203.001)' \
-H 'X-Twitter-Active-User: yes' \
-H "X-Guest-Token: ${guest_token}" \
-d "{\"flow_token\":\"${flow_token}\",\"subtask_inputs\":[{\"open_link\":{\"link\":\"next_link\"},\"subtask_id\":\"NextTaskOpenLink\"}],\"subtask_versions\":{\"generic_urt\":3,\"standard\":1,\"open_home_timeline\":1,\"app_locale_update\":1,\"enter_date\":1,\"email_verification\":3,\"enter_password\":5,\"enter_text\":5,\"one_tap\":2,\"cta\":7,\"single_sign_on\":1,\"fetch_persisted_data\":1,\"enter_username\":3,\"web_modal\":2,\"fetch_temporary_password\":1,\"menu_dialog\":1,\"sign_up_review\":5,\"interest_picker\":4,\"user_recommendations_urt\":3,\"in_app_notification\":1,\"sign_up\":2,\"typeahead_search\":1,\"user_recommendations_list\":4,\"cta_inline\":1,\"contacts_live_sync_permission_prompt\":3,\"choice_selection\":5,\"js_instrumentation\":1,\"alert_dialog_suppress_client_events\":1,\"privacy_options\":1,\"topics_selector\":1,\"wait_spinner\":3,\"tweet_selection_urt\":1,\"end_flow\":1,\"settings_list\":7,\"open_external_link\":1,\"phone_verification\":5,\"security_key\":3,\"select_banner\":2,\"upload_media\":1,\"web\":2,\"alert_dialog\":1,\"open_account\":2,\"action_list\":2,\"enter_phone\":2,\"open_link\":1,\"show_code\":1,\"update_users\":1,\"check_logged_in_account\":1,\"enter_email\":2,\"select_avatar\":4,\"location_permission_prompt\":2,\"notifications_permission_prompt\":4}}" | jq -r '.subtasks[0]|if(.open_account) then .open_account else empty end'
#!/bin/bash
# Run script and capture its output
output=$(./guest_tokens.sh)
# Check if the output is not empty
if [ -n "$output" ]; then
# Check if the JSON file exists
if [ ! -f guest_accounts.json ]; then
# If the file does not exist, create it with an opening square bracket
echo "[" > guest_accounts.json
else
# If the file already exists, remove the trailing square bracket
sed -i '$ d' guest_accounts.json
# Add a comma between entries
echo "," >> guest_accounts.json
fi
# Append the output to the JSON file and add closing square bracket
echo $output >> guest_accounts.json
echo "]" >> guest_accounts.json
# Format the JSON file using jq
jq . guest_accounts.json > output-formatted.json
mv output-formatted.json guest_accounts.json
# Print a success message
echo "Entry appended to and formatted in json file."
else
echo "No output from script. Nothing appended to json file."
fi Now generated entries should be appended while maintaining proper JSON syntax, I also format the whole file with Finally you can use # Every day at 12AM
0 0 * * * cd $HOME/Docker/nitter && ./generate_guest_accounts.sh Here is a sample script used for testing in order to simulate what #!/bin/bash
# Generate a JSON block with a random user ID
generate_json_entry() {
user_id=$((RANDOM % 1000000))
cat <<EOF
{
"user": {
"id": $user_id
}
}
EOF
}
generate_json_entry |
@y0nei JSONL is supported, you don't need to do all that. Just append each json object on a new line and name the file |
It seems the script for getting guest accounts it broken now. And maybe they expired all those credentials, I only get what's already cached in my instance. |
Same issue for me, I'm unable to generate any new guest accounts, all of my current ones got "expired" and I can only view what is cached on my instance. |
Oddly I can view posts in Nitter again with my old tokens, but the guest token generation script still returns Even during the outage, I could view user profiles and old tweets, but expanding any tweet resulted in an error. |
Can confirm, the token script fails, but I can read Nitter again, mostly. Seems hit and miss, the RSS feeds show the tweet, but I can't see the tweet responses for example. |
Avoid expiring the tokens for now. See: - #983 (comment) - #1155 (comment) Thanks @cmj
This replaces the old method of using guest tokens with using "guest accounts".
For more information see this comment #983 (comment)
Live now on nitter.net for testing. If you'd like to test and don't have any guest accounts, feel free to reach out to me on Matrix. A way to acquire guest accounts will be build and shared very soon.