Twitter scraping is detected - 429 (Too Many Requests)

[fashionprivate]

Hello everybody,

I'm writing a bot to continuously scrape a Twitter account to retrieve the most recent tweets.
After 25 refresh requests, Twitter's response is as follows: 429 (Too Many Requests).

Below is the code and the response header by Twitter:

import traceback
from seleniumbase import SB
import random


def manageException(sb, profile_url):
    print("Something went wrong in Chromedriver library.")
    print(traceback.format_exc())
    sb.get_new_driver(undetectable=True)
    sb.driver.uc_open_with_reconnect(profile_url, reconnect_time=3)
    sb.sleep(1.2)
    return sb.driver.get_text("div[data-testid='tweetText']")

print("Logging to Twitter...")
    
with SB(uc=True) as sb:
    sb.driver.uc_open("https://www.twitter.com")
    
    # do the login in Twitter
    
    account = "daniesilvestri" # account with no blue tick
    profile_url = f'https://twitter.com/{account}'
    
    sb.driver.uc_open(profile_url)
    last_tweet = ""
    try:
        last_tweet = sb.driver.get_text("div[data-testid='tweetText']")
    except Exception as err:
        last_tweet = manageException(sb, profile_url)

    print(last_tweet)
    
    # do some stuff with the tweet
            
    sb.sleep(random.randint(800, 2100) / 1000.0)
            
    while 1:
        sb.driver.refresh()
        new_tweet = ""
        try:
            new_tweet = sb.driver.get_text("div[data-testid='tweetText']")
        except Exception as err:
            new_tweet = manageException(sb, profile_url)
        print(new_tweet)
        
        # do some stuff
                    
        sb.sleep(random.randint(800, 2100) / 1000.0)

The response header:

As you can see the parameter "X-Rate-Limit-Remaining" is 0. That means that it is no longer possible to make requests like this one.
The bot is limited for approximately 12 minutes (the exact time at which it will be possible to refresh the page again is represented by the "X-Rate-Limit-Reset" header).
Documentation on the parameters can be found here: https://developer.twitter.com/en/docs/twitter-api/rate-limits
Is there a way to bypass the limits imposed by Twitter? For example, I tried logging out and logging in again but it doesn't work.
Thanks in advance

[mdmintz]

Hello. It sounds like you're asking if SeleniumBase has the power to increase your Twitter API limits. I'm flattered that you think SeleniumBase has that kind of power. Although SeleniumBase has many talents and can do many things, increasing your Twitter API limits is simply not one of those abilities.

However, (long pause), if those API limits are tied to an IP Address, then you can get around that API limit by using SeleniumBase to change your proxy settings via the proxy parameter. Two formats are accepted for that:

• host:port
• user:pass@host:port