Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parsing peers #728

Closed
cryptid11 opened this issue Dec 31, 2016 · 3 comments
Closed

Parsing peers #728

cryptid11 opened this issue Dec 31, 2016 · 3 comments

Comments

@cryptid11
Copy link

I'm keeping some stats on peers using a function


def ParsePeers(site):
    stats_content = requests.get('http:https://127.0.0.1:43110/Stats').text
    soup = BeautifulSoup(stats_content)

    def contains_site(tag):
        if tag.name != 'tr':
            return False
        link = tag.find('a')
        if not link:
            return False
        return site in link.string

    tr = soup.find(contains_site)

    if not tr:
        return None

    pattern = re.compile("[0-9]+/[0-9]+/([0-9]+)")

    def contains_peers(tag):
        return tag.name == 'td' and pattern.search(unicode(tag.string))

    peer_td = tr.find(contains_peers)
    peer_count = int( pattern.findall(unicode(peer_td.string))[0] )

    return (tr.find('a').string, peer_count)

that I remember as founded in ZeroBlog, but it seems I have confused memory or somebody delete article, anyway.

The stats page doesn't allow anymore to be parsed in new version, it seems it checks user-agent, it produces this horrible result:

<style>
                * { font-family: Consolas, Monospace; color: #333 }
                pre { padding: 10px; background-color: #EEE }
                </style>
<h1>Forbidden</h1>
<h2>Invalid Accept header to load wrapper
<h3>Please <a href="https://github.com/HelloZeroNet/ZeroNet/issues" target="_blank">report it</a> if you think this an error.</h3>
<h4>Details:</h4>
<pre>{
    "GATEWAY_INTERFACE": "CGI/1.1", 
    "HTTP_ACCEPT": "*/*", 
    "HTTP_ACCEPT_ENCODING": "gzip, deflate", 
    "HTTP_CONNECTION": "keep-alive", 
    "HTTP_HOST": "127.0.0.1:43110", 
    "HTTP_USER_AGENT": "python-requests/2.10.0", 
    "PATH_INFO": "/Stats", 
    "QUERY_STRING": "", 
    "REMOTE_ADDR": "127.0.0.1", 
    "REMOTE_PORT": "38987", 
    "REQUEST_METHOD": "GET", 
    "SCRIPT_NAME": "", 
    "SERVER_NAME": "localhost", 
    "SERVER_PORT": "43110", 
    "SERVER_PROTOCOL": "HTTP/1.1", 
    "SERVER_SOFTWARE": "gevent/1.0 Python/2.7", 
    "arguments": {
        "action": "main", 
        "batch": false, 
        "bit_resolver": "1Name2NXVi1RDPDgf5617UoW7xA6YrhM9F", 
        "coffeescript_compiler": null, 
        "config_file": "zeronet.conf", 
        "connected_limit": 10, 
        "data_dir": "data", 
        "debug": false, 
        "debug_gevent": false, 
        "debug_socket": false, 
        "disable_db": false, 
        "disable_encryption": false, 
        "disable_sslcompression": true, 
        "disable_udp": false, 
        "fileserver_ip": "*", 
        "fileserver_port": 15441, 
        "homepage": "1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D", 
        "ip_external": null, 
        "keep_ssl_cert": false, 
        "language": "en", 
        "log_dir": "log", 
        "max_files_opened": 2048, 
        "msgpack_purepython": true, 
        "open_browser": null, 
        "optional_limit": "30%", 
        "proxy": null, 
        "size_limit": 10, 
        "stream_downloads": false, 
        "tor": "enable", 
        "tor_controller": "127.0.0.1:9051", 
        "tor_proxy": "127.0.0.1:9050", 
        "trackers": [
            "zero:https://boot3rdez4rzn36x.onion:15441", 
            "zero:https://boot.zeronet.io#f36ca555bee6ba216b14d10f38c16f7769ff064e0e37d887603548cc2e64191d:15441", 
            "udp:https://tracker.coppersurfer.tk:6969", 
            "udp:https://tracker.leechers-paradise.org:6969", 
            "udp:https://9.rarbg.com:2710", 
            "http:https://tracker.tordb.ml:6881/announce", 
            "http:https://explodie.org:6969/announce", 
            "http:https://tracker1.wasabii.com.tw:6969/announce"
        ], 
        "trackers_file": false, 
        "ui_ip": "127.0.0.1", 
        "ui_port": 43110, 
        "ui_restrict": false, 
        "updatesite": "1UPDatEDxnvHDo7TXvq6AEBARfNkyfxsp", 
        "use_openssl": true, 
        "use_tempfiles": false, 
        "verbose": false
    }, 
    "plugins": [
        "AnnounceZero", 
        "CryptMessage", 
        "MergerSite", 
        "Newsfeed", 
        "OptionalManager", 
        "PeerDb", 
        "Sidebar", 
        "Stats", 
        "TranslateSite", 
        "Trayicon", 
        "Zeroname"
    ], 
    "version_gevent": "1.0.1", 
    "version_python": "2.7.9 (default, Aug 13 2016, 16:41:35) \n[GCC 4.9.2]", 
    "version_zeronet": "0.5.1 r1791", 
    "wsgi.url_scheme": "http"
}</pre>
</h2>

How can I stop this and get back the results? (Also, if somebody know the source of that function let me know)

@HelloZeroNet
Copy link
Owner

try stats_content = requests.get('http:https://127.0.0.1:43110/Stats', headers = {"Accept": "text/html"}).text

@cryptid11
Copy link
Author

Wow thanks, just adding that it works, what a noob! Great, so you don't know where is that code from? I will continue my quest.

@HelloZeroNet
Copy link
Owner

I searched for ParsePeers on ZeroHello and dropped this result:
http:https://127.0.0.1:43110/1CiDoBP8RiWziqiBGEd8tQMy66A6fmnw2V/?Post:7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants