This library was initially to search for bibliographic data of books, and it was expanded for DVDs and CDs. We can search with keywords, with the isbn (so than we can use barcode scanners), with some advanced search, and we have pagination.

We get the data from existing websites. We scrape:

• for French books:
  • Dilicom's profesional provider (new in v0.4, jan. 2020),
  • http:https://www.librairie-de-paris.fr See its doc
• for Switzerland: lelivre.ch (new in 0.5, feb. 2020)
• for Spain: http:https://www.casadellibro.com
• for Germany: http:https://www.buchlentner.de
• for DVDs: https://www.momox-shop.fr
• and for CDs: https://www.discogs.com (may need more testing)

We retrieve: the title and authors, the price, the isbn, the publisher(s), the cover,...

This library forms the heart of Abelujo, a free software for bookshops.

Install from pypi:

pip install bookshops

You can try this lib on the command line with the following commands:

• livres: french books
• lelivre: swiss books
• dilicom: Dilicom search (only searches ISBNs, no free search)
• libros: spanish books
• bucher: german books
• discogs: CDs
• movies: DVDs
• come and ask for more :)

For example:

livres antigone

or

livres 9782918059363

and you get the above screenshot.

Options: (this may vary according to the scrapers, check them with -h)

• -i or --isbn to ensure to get all the isbn. The command line tool won't get them by default if they need to be fetched with another http request for each book. That depends on the websites.

The Dilicom interface requires you to set two environment variables:

export DILICOM_USER="300xxx"
export DILICOM_PASSWORD="xyz"

In addition, you can set this third one, that allows you to view a book's product page on Dilicom's website (within your account). You find it on the url of your account. For example, when I am visiting this book page:

https://dilicom-prod.centprod.com/catalogue/detail_article_consultation.html?ean=9782840550877&emet=3010xxxxx0100

I set the environment variable like so:

export DILICOM_EMET="3010xxxxx0100"

In Abelujo, this sets the "details_url" Card slot accordingly and you can click on the "source" link when viewing a book's page.

But most of all, from within your program:

from bookshops.frFR.librairiedeparis.librairiedeparisScraper import Scraper as frenchScraper

scraper = frenchScraper("search keywords")
cards = scraper.search()
# we get a list of dictionnaries with the title, the authors, etc.

Results are cached in memory for about 1 day (except Dilicom results, in purpose). It allows long-running software based on this library (e.g., Abelujo) to feel more dynamic in certain cases.

Work in progress.

You can search ed:agone to search for a specific publisher.

We do pagination:

scraper = frenchScraper("search keywords", page=2)

Amazon kills the book industry and its employees. But moreover, we can add value to our results. We can link to a good and independent bookshop from within our application, we could command books from it, we could say if it has exemplaries in stock or not, etc.

Technically speaking, the Amazon API web service can be too limiting and not appropriate. One must register to Amazon Product Advertising and to AWS, making it more difficult for deployment or independant users, and it changes way more often than our resailers' websites.

It has very few data.

Because, for bookshops, we need recent books (they enter the BNF database after a few months), and the price.

See http:https://dev.abelujo.cc/webscraping.html

Development mode:

pip install -e .

Now you can edit the project and run the development version like the lib is meant to be run, i.e. with the entry_points: livres, libros, etc.

doc: https://python-packaging-user-guide.readthedocs.org/en/latest/distributing/#working-in-development-mode

Note: the Dilicom interface is not concerned by these limitations.

This is webscraping, so it doesn't go without pitfalls:

• the site can go down. It happened already.
• the site can change, it which case we would have to change our sraper too. To catch this early we run automatic tests every week. The actual website didn't change in 3 years.

• multiple ISBN search for Dilicom (by batches of one hundred).

• results are cached again. Simply in memory for about 1 day.

• added a Swiss interface.
• added support to fetch and print prices in another currency.

• added Dilicom interface. It only provides search of ISBN(s), it doesn't provide free and advanced search.

• added search of DVDs
• updated french scrapers (first time needed in four years).

• remove deprecated import from ods/csv feature. Might do a simpler one in the future.

• german scraper: search by isbn

• German scraper
• multiprocessing for the german scraper (from 15 to 9s) (see issue #1)
• --isbn option for it

• french, spanish scrapers
• command line tool

LGPLv3