A small AI-based conversational agent capable of providing accurate informations about U.S located restaurants, and integrated with a content-based recommendation engine.
Made with โค๏ธ by Alexandre ZAJAC, Nicolas CAILLIEUX and Marc-Etienne Dartus.
- Loa ๐งโโ๏ธ
We use wit.ai to convert user-text to intents using Natural Language Processing. We have trained a model with Python to recommend restaurants with respect to user queries and desires. If they do not ask for recommendation, we use the yelp API to provide information based on user-location and his query.
In this project and in like many nowadays, we chose to use React for the client interface.
Description
React is a JavaScript library for building user interfaces. It is maintained by Facebook and a community of individual developers and companies. React can be used as a base in the development of single-page or mobile applications.React is only concerned with rendering data to the DOM.
How we use it
We use React for all the frontend part. React only powers the user interface, interprets what the user wants to perform and calls our NodeJS API.
For the back-end, we decided to stay with NodeJS as a central service, as it's a technology we are familiar and realtively easy to integrate with other services. We also have a Python Flask API, which powers the recommendation of restaurants.
Description
Node.js is an open-source, cross-platform, JavaScript runtime environment that executes JavaScript code outside of a web browser. Node.js lets developers use JavaScript to write command line tools and for server-side scriptingโrunning scripts server-side to produce dynamic web page content before the page is sent to the user's web browser.
How we use it
We use it to centralize the variety of the user's requests. It almost serves as a middleware between the client, and the three various API's (wit.ai, yelp and our own recommender API). For every message that the user sends, we send the payload to wit from there, and then handle the logic of calling either YELP or our own recommender system.
Description
Flask is a lightweight WSGI web application framework. It is designed to make getting started quick and easy, with the ability to scale up to complex applications. It began as a simple wrapper around Werkzeug and Jinja and has become one of the most popular Python web application frameworks.
How we use it
We use Flask to build a basic API that will handle only one route for recommending restaurants. This API is agnostic of all the intent extracting and analysis logic, it's only made to provide an abstraction to return recommended restaurants to the NodeJS API.
You can test the project with a live demo by going there!
Or, you can test this project locally with Docker using the following steps :
- Then in the project folder run:
docker-compose up
The project will then be available on http:https://DOCKER_IP_HOST url, which in most cases, will be http:https://localhost
Types of Question:
- How many restaurants are in New York ?
- What is the number of restaurants around me ?
- What is the number of restaurants in Colorado ?
1 - Request :
Requesting the Node API :
Endpoint :
POST /message
We call our Node API we this structure :
{
"message": "What is the number of restaurants in Colorado ?",
"user": {
"coordinates": {
"latitude": 34.052234,
"longitude": -118.243685
}
}
}
2 - Wit.ai Intent extraction :
Wit.ai will exctract intent and entities from the user question.
The question :
What is the number of restaurants in Colorado ?
Will return :
Variable | Value |
---|---|
Intent |
Number |
Object |
Restaurants |
Location |
Colorado |
3 - Yelp API :
With the information, we call the Yelp API and use the result to build our API response :
Yelp Graphql API request :
{
search(term: "restaurant", location:"Colorado") {
total
}
}
Yelp Graphql API response :
{
"data": {
"search": {
"total": 2952
}
}
}
4 - User reponse generation
After getting all the information in order to answer the user demand, we use Natural Language Generation to display a response.
additionally, we display the concenred result(s) on the right side of our web application.
In our example we want to display a map of the result and tell it to the user.
5 - Node.js API response
Gathering all of these steps, our API will return a sample response :
{
"intent": "Number",
"type": "Restaurants",
"location": {
"name": "Colorado",
"coordinates": {
"latitude": 39.55051,
"longitude": -105.782067
}
},
"message": [
{
"type": "text",
"content": "There are 2952 restaurants in Colorado. โฒ๏ธ"
}
],
"results": []
}
Types of Question :
- Can you show me the restaurants around me ?
- Can you show me the restaurants in Los Angeles ?
1 - Node.js API request :
Requesting the Node API :
Endpoint :
POST /message
We call our Node API we this structure :
{
"message": "Can you show me the restaurants in Los Angeles ?",
"user": {
"coordinates": {
"latitude": 34.052234,
"longitude": -118.243685
}
}
}
2 - Wit.ai Intent extraction :
Wit.ai will exctract intent and entities from the user question.
The question :
Can you show me the restaurants in Los Angeles ?
Will return :
Variable | Value |
---|---|
Intent |
Search |
Object |
Restaurants |
Location |
Los Angeles |
3 - Yelp API :
With the information, we call the Yelp API and use the result to build our API response :
Yelp Graphql API request :
{
search(term: "restaurant", location:"Los Angeles") {
business {
name
id
url
display_phone
rating
price
location {
address1
city
postal_code
}
coordinates {
latitude
longitude
}
photos
}
}
}
Yelp Graphql API response :
{
"data": {
"search": {
"business": [
{
"name": "Howlin' Ray's",
"id": "7O1ORGY36A-2aIENyaJWPg",
"url": "https://www.yelp.com/biz/howlin-rays-los-angeles-3?adjust_creative=94DePyCeUwdjASSwoI0YbA&utm_campaign=yelp_api_v3&utm_medium=api_v3_graphql&utm_source=94DePyCeUwdjASSwoI0YbA",
"display_phone": "(213) 935-8399",
"rating": 4.5,
"price": "$$",
"location": {
"address1": "727 N Broadway",
"city": "Los Angeles",
"postal_code": "90012"
},
"coordinates": {
"latitude": 34.061517,
"longitude": -118.239716
},
"photos": [
"https://s3-media2.fl.yelpcdn.com/bphoto/9hGoyECcrewigEKYEnrYTw/o.jpg"
]
}
]
}
}
}
4 - User reponse generation
After getting all the information in order to answer the user demand, we use Natural Language Generation to display a response.
additionally, we display the concenred result(s) on the right side of our web application.
In our example we want to display restaurants searched by the user.
5 - Node.js API response
Gathering all of these steps, our API will return a sample response :
{
"intent": "Search",
"type": "Restaurants",
"location": {
"name": "Los Angeles",
"coordinates": {
"latitude": 34.052234,
"longitude": -118.243685
}
},
"message": [
{
"type": "text",
"content": "You can find a selection of restaurants in Los Angeles. ๐จ"
}
],
"results": [
{
"name": "Howlin' Ray's",
"id": "7O1ORGY36A-2aIENyaJWPg",
"url": "https://www.yelp.com/biz/howlin-rays-los-angeles-3?adjust_creative=94DePyCeUwdjASSwoI0YbA&utm_campaign=yelp_api_v3&utm_medium=api_v3_graphql&utm_source=94DePyCeUwdjASSwoI0YbA",
"display_phone": "(213) 935-8399",
"rating": 4.5,
"price": "$$",
"location": {
"address1": "727 N Broadway",
"city": "Los Angeles",
"postal_code": "90012"
},
"coordinates": {
"latitude": 34.061517,
"longitude": -118.239716
},
"photos": [
"https://s3-media2.fl.yelpcdn.com/bphoto/9hGoyECcrewigEKYEnrYTw/o.jpg"
]
}
]
}
Types of Question
- What is the best restaurant in New York ?
- What is the best restaurant around me ?
- Find the best restaurant in Paris ?
- Find the best restaurant near me ?
- Find the best restaurant near 111 8th Ave New York ?
1 - Request :
Requesting the Node API :
Endpoint :
POST /message
We call our Node API we this structure :
{
"message": "What is the best restaurant in New York ?",
"user": {
"coordinates": {
"latitude": 34.052234,
"longitude": -118.243685
}
}
}
2 - Wit.ai Intent extraction :
Wit.ai will exctract intent and entities from the user question.
The question :
Can you show me the restaurants in Los Angeles ?
Will return :
Variable | Value |
---|---|
Intent |
Best |
Object |
Restaurants |
Location |
New York |
3 - Yelp API :
With the information, we call the Yelp API and use the result to build our API response :
Yelp Graphql API request :
{
search(term: "restaurant", location:"New York") {
business {
name
id
url
display_phone
review_count
rating
price
location {
address1
city
postal_code
}
coordinates {
latitude
longitude
}
photos
}
}
}
Yelp Graphql API response :
{
"data": {
"search": {
"business": [
{
"name": "LoveMama",
"id": "jjJc_CrkB2HodEinB6cWww",
"url": "https://www.yelp.com/biz/lovemama-new-york?adjust_creative=94DePyCeUwdjASSwoI0YbA&utm_campaign=yelp_api_v3&utm_medium=api_v3_graphql&utm_source=94DePyCeUwdjASSwoI0YbA",
"display_phone": "(212) 254-5370",
"review_count": 4988,
"rating": 4.5,
"price": "$$",
"location": {
"address1": "174 2nd Ave",
"city": "New York",
"postal_code": "10003"
},
"coordinates": {
"latitude": 40.7303859,
"longitude": -73.9860613
},
"photos": [
"https://s3-media1.fl.yelpcdn.com/bphoto/bLlFKTlVuLfmF-lIDGIjZA/o.jpg"
]
}
]
}
}
}
4 - User reponse generation
After getting all the information in order to answer the user demand, we use Natural Language Generation to display a response.
additionally, we display the concenred result(s) on the right side of our web application.
In our example we want to display the best results and tell it to the user.
5 - Node.js API response
Gathering all of these steps, our API will return a sample response a sample response:
{
"intent": "Best",
"type": "Restaurants",
"location": {
"name": "New York",
"coordinates": {
"latitude": 40.7122775,
"longitude": -74.005973
}
},
"message": [
{
"type": "text",
"content": "The best restaurant in New York is LoveMama. ๐ฏ"
}
],
"results": [
{
"name": "LoveMama",
"id": "jjJc_CrkB2HodEinB6cWww",
"url": "https://www.yelp.com/biz/lovemama-new-york?adjust_creative=94DePyCeUwdjASSwoI0YbA&utm_campaign=yelp_api_v3&utm_medium=api_v3_graphql&utm_source=94DePyCeUwdjASSwoI0YbA",
"display_phone": "(212) 254-5370",
"review_count": 4988,
"rating": 4.5,
"price": "$$",
"location": {
"address1": "174 2nd Ave",
"city": "New York",
"postal_code": "10003"
},
"coordinates": {
"latitude": 40.7303859,
"longitude": -73.9860613
},
"photos": [
"https://s3-media1.fl.yelpcdn.com/bphoto/bLlFKTlVuLfmF-lIDGIjZA/o.jpg"
]
}, ...
]
}
Types of Question:
- Hello, how are you ?
- Good morning!
- Hi, what's up ?
1 - Node.js API request :
Requesting the Node API :
Endpoint :
POST /message
We call our Node API we this structure :
{
"message": "Hi, what's up ?",
"user": {
"coordinates": {
"latitude": 34.052234,
"longitude": -118.243685
}
}
}
2 - Wit.ai Intent extraction :
Wit.ai will exctract intent and entities from the user question.
The question :
Hi, what's up ?
Will return :
Variable | Value |
---|---|
Intent |
Greeting |
3 - User reponse generation
After getting all the information in order to answer the user demand, we use Natural Language Generation to display a response.
4 - Node.js API response
Gathering all of these steps, our API will return a sample response :
{
"intent": "Greeting",
"type": null,
"location": null,
"message": [
{
"type": "text",
"content": "Hi, what's can I do for you today ? ๐ฉ"
},
{
"type": "gif",
"content": "https://media.giphy.com/media/14aa5GbbHT3bHO/source.gif"
}
],
"results": []
}
Types of Question:
- Can I get an example ?
- Can you give me an example of question ?
- What can I ask you ?
1 - Node.js API request :
Requesting the Node API :
Endpoint :
POST /message
We call our Node API we this structure :
{
"message": "Can I get an example ?",
"user": {
"coordinates": {
"latitude": 34.052234,
"longitude": -118.243685
}
}
}
2 - Wit.ai Intent extraction :
Wit.ai will exctract intent and entities from the user question.
The question :
Can I get an example ?
Will return :
Variable | Value |
---|---|
Intent |
Example |
3 - User reponse generation
After getting all the information in order to answer the user demand, we use Natural Language Generation to display a response.
4 - Node.js API response
Gathering all of these steps, our API will return a sample response :
{
"intent": "Example",
"type": null,
"location": null,
"message": [
{
"type": "text",
"content": "Here's some examples: ๐ - Can you recommend me an italian restaurant ?\n - What is the number of restaurants in Seattle ? - Can you show me the restaurants around me ?\n - What is the best restaurant in Los Angeles ?\n"
},
{
"type": "gif",
"content": "https://media.giphy.com/media/szeVLlECC8ThC/source.gif"
}
],
"results": []
}
Types of Sentence:
- Thanks for helping me
- Thank you for your help
- Thank you for your responses
1 - Node.js API request :
Requesting the Node API :
Endpoint :
POST /message
We call our Node API we this structure :
{
"message": "Thanks for helping me",
"user": {
"coordinates": {
"latitude": 34.052234,
"longitude": -118.243685
}
}
}
2 - Wit.ai Intent extraction :
Wit.ai will exctract intent and entities from the user question.
The question :
Thanks for helping me
Will return :
Variable | Value |
---|---|
Intent |
Thanks |
3 - User reponse generation
After getting all the information in order to answer the user demand, we use Natural Language Generation to display a response.
4 - Node.js API response
Gathering all of these steps, our API will return a sample response :
{
"intent": "Thanks",
"type": null,
"location": null,
"message": [
{
"type": "text",
"content": "Don't worry, I'm very happy to help you ! ๐ค Need more help ?"
},
{
"type": "gif",
"content": "https://media.giphy.com/media/LrQkEUJ3s8hLmO9fGH/source.gif"
}
],
"results": []
}
Types of Sentence:
- Goodbye
- See you soon !
- I'm going to leave, bye !
1 - Node.js API request :
Requesting the Node API :
Endpoint :
POST /message
We call our Node API we this structure :
{
"message": "Goodbye",
"user": {
"coordinates": {
"latitude": 34.052234,
"longitude": -118.243685
}
}
}
2 - Wit.ai Intent extraction :
Wit.ai will exctract intent and entities from the user question.
The sentence :
Goodbye
Will return :
Variable | Value |
---|---|
Intent |
Goodbye |
3 - User reponse generation
After getting all the information in order to answer the user demand, we use Natural Language Generation to display a response.
4 - Node.js API response
Gathering all of these steps, our API will return a sample response :
{
"intent": "Goodbye",
"type": null,
"location": null,
"message": [
{
"type": "text",
"content": "I was a pleasure to help you, goodbye ! ๐"
},
{
"type": "gif",
"content": "https://media.giphy.com/media/ZA5DTtqkU8bQDHuu16/source.gif"
}
],
"results": []
}
As said above, we decided to build our own API and recommender system with Python, to be able to recommend restaurants to users.
The technique we have used is more vastly acknowledge as collaborative filtering:
We have used ratings from users on a subset of US-based restaurants to predict the rating of a restaurant unseen by the user, let's deep dive into how we trained a model to do that.
After having collected the user data with reviews and ratings, we also collected items (the restaurants) and determined user profiles.
To obtain the profile matrices we follewed the same routine:
- Combine the reviews/descriptions
- Remove noisy data (stopwords, punctuation, ...)
- TF-IDF Feature vector extraction on each of the items/users.
Then with a sample input from the user: "I want to eat italain pizza", the system recommends the items that matches the most this review/description.
Here is how it works under the hood:
Types of Sentence:
- I want to eat some italian pizza!
- Recommend me restaurants with fresh pasta.
- Can you show me restaurants with outstanding views ?
1 - Node.js API request :
Requesting the Node API :
Endpoint :
POST /message
We call our Node API we this structure :
{
"message": "I want to eat some italian pizza!",
"user": {
"coordinates": {
"latitude": 34.052234,
"longitude": -118.243685
}
}
}
2 - Wit.ai Intent extraction :
Wit.ai will exctract intent and entities from the user question.
The sentence :
I want to eat some italian pizza!
Will return :
Variable | Value |
---|---|
Intent |
Recommend |
Desire |
Italian pizza |
3 - User reponse generation
After getting all the information in order to answer the user demand, we use Natural Language Generation to display a response.
4 - Node.js API response
Gathering all of these steps, our API will return a sample response :
{
"intent": "Recommend",
"type": null,
"location": null,
"message": [
{
"type": "text",
"content": "Oh, italian pizza is a good idea! Let me recommend you these restaurants. ๐ฎ"
}
],
"results": [
{
"name": "I want to eat some **italian pizza**!",
"id": "jjJc_CrkB2HodEinB6xWww",
"url": "https://www.yelp.com/biz/venezias-new-york-style-pizzeria-tempe-4?adjust_creative=94DePyCeUwdjASSwoI0YbA&utm_campaign=yelp_api_v3&utm_medium=api_v3_graphql&utm_source=94DePyCeUwdjASSwoI0YbA",
"display_phone": "(212) 254-5370",
"review_count": 714,
"rating": 4,
"price": "$",
"location": {
"address1": "174 2nd Ave",
"city": "Tempe",
"postal_code": "85282"
},
"coordinates": {
"latitude": 40.7303859,
"longitude": -73.9860613
},
"photos": [
"https://s3-media4.fl.yelpcdn.com/bphoto/kblyz8LxF5FCGuMvLeJFqg/o.jpg"
]
}, ...
]
}
- Desing & Prototype on Figma
- Define constraints and objectives
- Implement basic front-end interface
- Train wit.ai agent on our intents
- Design and Connect Node API to wit.ai
- Gather data and design Recommendation Engine
- Serve our APIS as services and host them
- Finish front-end interface
- Test along the way