Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty download #14

Open
MordicusEtCubitus opened this issue May 23, 2024 · 1 comment
Open

Empty download #14

MordicusEtCubitus opened this issue May 23, 2024 · 1 comment

Comments

@MordicusEtCubitus
Copy link

Hi,

I'm trying to use your nice tool from command line but always got an empty dataset:

  • On my browser this query (https://xeno-canto.org/api/2/recordings?query=pinson ) returns 8083 records
    {"numRecordings":"8083","numSpecies":"4","page":1,...}
  • With xeno-canto -m pinso I got an empty dataset
    {'numRecordings': '0', 'numSpecies': '0', 'page': 1, 'numPages': 1, 'recordings': []}
  • Trying directly using your code or another one with requests (see below):
    {'numRecordings': '0', 'numSpecies': '0', 'page': 1, 'numPages': 1, 'recordings': []}
import json
from urllib import request, error
url = 'https://www.xeno-canto.org/api/2/recordings?query=pinson'
try:
    r = request.urlopen(url)
except error.HTTPError as e:
    print("An error has occurred: " + str(e))
    exit()

data = json.loads(r.read().decode('UTF-8'))
print(data)

got

  {'numRecordings': '0', 'numSpecies': '0', 'page': 1, 'numPages': 1, 'recordings': []}

Same with

import requests
url = 'https://xeno-canto.org/api/2/recordings'
headers = {"User-Agent": "Mozilla/5.0" }
params = {"query" : "pinson", "page" : 1 }
r = requests.get(url, params=params)
r.json()

I think there is another security on the server preventing use outside browser, but I do not find what it is

If you have an idea ?

Thanks for helping

@MordicusEtCubitus
Copy link
Author

Ok, I think I got it, this may help to update your code:

I've started to add my browser headers one by one:

headers = {"User-Agent": "Mozilla/5.0"
          , "Accept-Encoding": "gzip, deflate, br, zstd"
           , "Accept-Language": "fr,en;q=0.5"
          }

When adding "Accept-Language" I did got data !

Maybe the 2 others are required too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant