Skip to content

Commit b1efeaf

Browse files
INNOVASEINNOVASE
authoredAug 30, 2021
Add files via upload
0 parents  commit b1efeaf

File tree

5 files changed

+605
-0
lines changed

5 files changed

+605
-0
lines changed
 

‎Chatbot.ipynb

+384
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,384 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Building a Simple Chatbot from Scratch in Python (using NLTK)\n",
8+
"\n",
9+
"![Alt text](https://cdn-images-1.medium.com/max/800/1*pPcVfZ7i-gLMabUol3zezA.gif)\n",
10+
"\n",
11+
"History of chatbots dates back to 1966 when a computer program called ELIZA was invented by Weizenbaum. It imitated the language of a psychotherapist from only 200 lines of code. You can still converse with it here: [Eliza](http://psych.fullerton.edu/mbirnbaum/psych101/Eliza.htm?utm_source=ubisend.com&utm_medium=blog-link&utm_campaign=ubisend). \n",
12+
"\n",
13+
"On similar lines let's create a very basic chatbot utlising the Python's NLTK library.It's a very simple bot with hardly any cognitive skills,but still a good way to get into NLP and get to know about chatbots.\n",
14+
"\n",
15+
"For detailed analysis, please see the accompanying blog titled:**[Building a Simple Chatbot in Python (using NLTK](https://medium.com/analytics-vidhya/building-a-simple-chatbot-in-python-using-nltk-7c8c8215ac6e)\n"
16+
]
17+
},
18+
{
19+
"cell_type": "markdown",
20+
"metadata": {},
21+
"source": [
22+
"## NLP\n",
23+
"NLP is a way for computers to analyze, understand, and derive meaning from human language in a smart and useful way. By utilizing NLP, developers can organize and structure knowledge to perform tasks such as automatic summarization, translation, named entity recognition, relationship extraction, sentiment analysis, speech recognition, and topic segmentation."
24+
]
25+
},
26+
{
27+
"cell_type": "markdown",
28+
"metadata": {},
29+
"source": [
30+
"## Import necessary libraries"
31+
]
32+
},
33+
{
34+
"cell_type": "code",
35+
"execution_count": 1,
36+
"metadata": {},
37+
"outputs": [],
38+
"source": [
39+
"import io\n",
40+
"import random\n",
41+
"import string # to process standard python strings\n",
42+
"import warnings\n",
43+
"import numpy as np\n",
44+
"from sklearn.feature_extraction.text import TfidfVectorizer\n",
45+
"from sklearn.metrics.pairwise import cosine_similarity\n",
46+
"import warnings\n",
47+
"warnings.filterwarnings('ignore')"
48+
]
49+
},
50+
{
51+
"cell_type": "markdown",
52+
"metadata": {},
53+
"source": [
54+
"## Downloading and installing NLTK\n",
55+
"NLTK(Natural Language Toolkit) is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to over 50 corpora and lexical resources such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning, wrappers for industrial-strength NLP libraries.\n",
56+
"\n",
57+
"[Natural Language Processing with Python](http://www.nltk.org/book/) provides a practical introduction to programming for language processing.\n",
58+
"\n",
59+
"For platform-specific instructions, read [here](https://www.nltk.org/install.html)\n",
60+
"\n"
61+
]
62+
},
63+
{
64+
"cell_type": "code",
65+
"execution_count": 2,
66+
"metadata": {},
67+
"outputs": [
68+
{
69+
"name": "stdout",
70+
"output_type": "stream",
71+
"text": [
72+
"Requirement already satisfied: nltk in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (3.4.1)\n",
73+
"Requirement already satisfied: six in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from nltk) (1.12.0)\n",
74+
"Note: you may need to restart the kernel to use updated packages.\n"
75+
]
76+
}
77+
],
78+
"source": [
79+
"pip install nltk"
80+
]
81+
},
82+
{
83+
"cell_type": "markdown",
84+
"metadata": {},
85+
"source": [
86+
"### Installing NLTK Packages\n",
87+
"\n",
88+
"\n"
89+
]
90+
},
91+
{
92+
"cell_type": "code",
93+
"execution_count": 3,
94+
"metadata": {},
95+
"outputs": [
96+
{
97+
"name": "stdout",
98+
"output_type": "stream",
99+
"text": [
100+
"showing info https://raw.githubusercontent.com/nltk/nltk_data/gh-pages/index.xml\n"
101+
]
102+
},
103+
{
104+
"data": {
105+
"text/plain": [
106+
"True"
107+
]
108+
},
109+
"execution_count": 3,
110+
"metadata": {},
111+
"output_type": "execute_result"
112+
}
113+
],
114+
"source": [
115+
"import nltk\n",
116+
"from nltk.stem import WordNetLemmatizer\n",
117+
"nltk.download('popular', quiet=True) # for downloading packages\n",
118+
"#nltk.download('punkt') # first-time use only\n",
119+
"#nltk.download('wordnet') # first-time use only"
120+
]
121+
},
122+
{
123+
"cell_type": "markdown",
124+
"metadata": {},
125+
"source": [
126+
"## Reading in the corpus\n",
127+
"\n",
128+
"For our example,we will be using the Wikipedia page for chatbots as our corpus. Copy the contents from the page and place it in a text file named ‘chatbot.txt’. However, you can use any corpus of your choice."
129+
]
130+
},
131+
{
132+
"cell_type": "code",
133+
"execution_count": 10,
134+
"metadata": {},
135+
"outputs": [],
136+
"source": [
137+
"f=open('chatbot.txt','r',errors = 'ignore')\n",
138+
"raw=f.read()\n",
139+
"raw = raw.lower()# converts to lowercase"
140+
]
141+
},
142+
{
143+
"cell_type": "markdown",
144+
"metadata": {},
145+
"source": [
146+
"\n",
147+
"The main issue with text data is that it is all in text format (strings). However, the Machine learning algorithms need some sort of numerical feature vector in order to perform the task. So before we start with any NLP project we need to pre-process it to make it ideal for working. Basic text pre-processing includes:\n",
148+
"\n",
149+
"* Converting the entire text into **uppercase** or **lowercase**, so that the algorithm does not treat the same words in different cases as different\n",
150+
"\n",
151+
"* **Tokenization**: Tokenization is just the term used to describe the process of converting the normal text strings into a list of tokens i.e words that we actually want. Sentence tokenizer can be used to find the list of sentences and Word tokenizer can be used to find the list of words in strings.\n",
152+
"\n",
153+
"_The NLTK data package includes a pre-trained Punkt tokenizer for English._\n",
154+
"\n",
155+
"* Removing **Noise** i.e everything that isn’t in a standard number or letter.\n",
156+
"* Removing the **Stop words**. Sometimes, some extremely common words which would appear to be of little value in helping select documents matching a user need are excluded from the vocabulary entirely. These words are called stop words\n",
157+
"* **Stemming**: Stemming is the process of reducing inflected (or sometimes derived) words to their stem, base or root form — generally a written word form. Example if we were to stem the following words: “Stems”, “Stemming”, “Stemmed”, “and Stemtization”, the result would be a single word “stem”.\n",
158+
"* **Lemmatization**: A slight variant of stemming is lemmatization. The major difference between these is, that, stemming can often create non-existent words, whereas lemmas are actual words. So, your root stem, meaning the word you end up with, is not something you can just look up in a dictionary, but you can look up a lemma. Examples of Lemmatization are that “run” is a base form for words like “running” or “ran” or that the word “better” and “good” are in the same lemma so they are considered the same.\n",
159+
"\n"
160+
]
161+
},
162+
{
163+
"cell_type": "markdown",
164+
"metadata": {},
165+
"source": [
166+
"## Tokenisation"
167+
]
168+
},
169+
{
170+
"cell_type": "code",
171+
"execution_count": 11,
172+
"metadata": {},
173+
"outputs": [],
174+
"source": [
175+
"sent_tokens = nltk.sent_tokenize(raw)# converts to list of sentences \n",
176+
"word_tokens = nltk.word_tokenize(raw)# converts to list of words"
177+
]
178+
},
179+
{
180+
"cell_type": "markdown",
181+
"metadata": {},
182+
"source": [
183+
"## Preprocessing\n",
184+
"\n",
185+
"We shall now define a function called LemTokens which will take as input the tokens and return normalized tokens."
186+
]
187+
},
188+
{
189+
"cell_type": "code",
190+
"execution_count": 12,
191+
"metadata": {},
192+
"outputs": [],
193+
"source": [
194+
"lemmer = nltk.stem.WordNetLemmatizer()\n",
195+
"#WordNet is a semantically-oriented dictionary of English included in NLTK.\n",
196+
"def LemTokens(tokens):\n",
197+
" return [lemmer.lemmatize(token) for token in tokens]\n",
198+
"remove_punct_dict = dict((ord(punct), None) for punct in string.punctuation)\n",
199+
"\n",
200+
"def LemNormalize(text):\n",
201+
" return LemTokens(nltk.word_tokenize(text.lower().translate(remove_punct_dict)))"
202+
]
203+
},
204+
{
205+
"cell_type": "markdown",
206+
"metadata": {},
207+
"source": [
208+
"## Keyword matching\n",
209+
"\n",
210+
"Next, we shall define a function for a greeting by the bot i.e if a user’s input is a greeting, the bot shall return a greeting response.ELIZA uses a simple keyword matching for greetings. We will utilize the same concept here."
211+
]
212+
},
213+
{
214+
"cell_type": "code",
215+
"execution_count": 13,
216+
"metadata": {},
217+
"outputs": [],
218+
"source": [
219+
"GREETING_INPUTS = (\"hello\", \"hi\", \"greetings\", \"sup\", \"what's up\",\"hey\",)\n",
220+
"GREETING_RESPONSES = [\"hi\", \"hey\", \"*nods*\", \"hi there\", \"hello\", \"I am glad! You are talking to me\"]\n",
221+
"def greeting(sentence):\n",
222+
" \n",
223+
" for word in sentence.split():\n",
224+
" if word.lower() in GREETING_INPUTS:\n",
225+
" return random.choice(GREETING_RESPONSES)"
226+
]
227+
},
228+
{
229+
"cell_type": "markdown",
230+
"metadata": {},
231+
"source": [
232+
"## Generating Response\n",
233+
"\n",
234+
"### Bag of Words\n",
235+
"After the initial preprocessing phase, we need to transform text into a meaningful vector (or array) of numbers. The bag-of-words is a representation of text that describes the occurrence of words within a document. It involves two things:\n",
236+
"\n",
237+
"* A vocabulary of known words.\n",
238+
"\n",
239+
"* A measure of the presence of known words.\n",
240+
"\n",
241+
"Why is it is called a “bag” of words? That is because any information about the order or structure of words in the document is discarded and the model is only **concerned with whether the known words occur in the document, not where they occur in the document.**\n",
242+
"\n",
243+
"The intuition behind the Bag of Words is that documents are similar if they have similar content. Also, we can learn something about the meaning of the document from its content alone.\n",
244+
"\n",
245+
"For example, if our dictionary contains the words {Learning, is, the, not, great}, and we want to vectorize the text “Learning is great”, we would have the following vector: (1, 1, 0, 0, 1).\n",
246+
"\n",
247+
"\n",
248+
"### TF-IDF Approach\n",
249+
"A problem with the Bag of Words approach is that highly frequent words start to dominate in the document (e.g. larger score), but may not contain as much “informational content”. Also, it will give more weight to longer documents than shorter documents.\n",
250+
"\n",
251+
"One approach is to rescale the frequency of words by how often they appear in all documents so that the scores for frequent words like “the” that are also frequent across all documents are penalized. This approach to scoring is called Term Frequency-Inverse Document Frequency, or TF-IDF for short, where:\n",
252+
"\n",
253+
"**Term Frequency: is a scoring of the frequency of the word in the current document.**\n",
254+
"\n",
255+
"```\n",
256+
"TF = (Number of times term t appears in a document)/(Number of terms in the document)\n",
257+
"```\n",
258+
"\n",
259+
"**Inverse Document Frequency: is a scoring of how rare the word is across documents.**\n",
260+
"\n",
261+
"```\n",
262+
"IDF = 1+log(N/n), where, N is the number of documents and n is the number of documents a term t has appeared in.\n",
263+
"```\n",
264+
"### Cosine Similarity\n",
265+
"\n",
266+
"Tf-idf weight is a weight often used in information retrieval and text mining. This weight is a statistical measure used to evaluate how important a word is to a document in a collection or corpus\n",
267+
"\n",
268+
"```\n",
269+
"Cosine Similarity (d1, d2) = Dot product(d1, d2) / ||d1|| * ||d2||\n",
270+
"```\n",
271+
"where d1,d2 are two non zero vectors.\n",
272+
"\n"
273+
]
274+
},
275+
{
276+
"cell_type": "markdown",
277+
"metadata": {},
278+
"source": [
279+
"To generate a response from our bot for input questions, the concept of document similarity will be used. We define a function response which searches the user’s utterance for one or more known keywords and returns one of several possible responses. If it doesn’t find the input matching any of the keywords, it returns a response:” I am sorry! I don’t understand you”"
280+
]
281+
},
282+
{
283+
"cell_type": "code",
284+
"execution_count": 14,
285+
"metadata": {},
286+
"outputs": [],
287+
"source": [
288+
"def response(user_response):\n",
289+
" robo_response=''\n",
290+
" sent_tokens.append(user_response)\n",
291+
" TfidfVec = TfidfVectorizer(tokenizer=LemNormalize, stop_words='english')\n",
292+
" tfidf = TfidfVec.fit_transform(sent_tokens)\n",
293+
" vals = cosine_similarity(tfidf[-1], tfidf)\n",
294+
" idx=vals.argsort()[0][-2]\n",
295+
" flat = vals.flatten()\n",
296+
" flat.sort()\n",
297+
" req_tfidf = flat[-2]\n",
298+
" if(req_tfidf==0):\n",
299+
" robo_response=robo_response+\"I am sorry! I don't understand you\"\n",
300+
" return robo_response\n",
301+
" else:\n",
302+
" robo_response = robo_response+sent_tokens[idx]\n",
303+
" return robo_response\n",
304+
"\n"
305+
]
306+
},
307+
{
308+
"cell_type": "markdown",
309+
"metadata": {},
310+
"source": [
311+
"Finally, we will feed the lines that we want our bot to say while starting and ending a conversation depending upon user’s input."
312+
]
313+
},
314+
{
315+
"cell_type": "code",
316+
"execution_count": 15,
317+
"metadata": {},
318+
"outputs": [
319+
{
320+
"name": "stdout",
321+
"output_type": "stream",
322+
"text": [
323+
"ROBO: My name is Robo. I will answer your queries about Chatbots. If you want to exit, type Bye!\n",
324+
"hi\n",
325+
"ROBO: hey\n",
326+
"hello\n",
327+
"ROBO: hey\n",
328+
"bye\n",
329+
"ROBO: Bye! take care..\n"
330+
]
331+
}
332+
],
333+
"source": [
334+
"flag=True\n",
335+
"print(\"ROBO: My name is Robo. I will answer your queries about Chatbots. If you want to exit, type Bye!\")\n",
336+
"while(flag==True):\n",
337+
" user_response = input()\n",
338+
" user_response=user_response.lower()\n",
339+
" if(user_response!='bye'):\n",
340+
" if(user_response=='thanks' or user_response=='thank you' ):\n",
341+
" flag=False\n",
342+
" print(\"ROBO: You are welcome..\")\n",
343+
" else:\n",
344+
" if(greeting(user_response)!=None):\n",
345+
" print(\"ROBO: \"+greeting(user_response))\n",
346+
" else:\n",
347+
" print(\"ROBO: \",end=\"\")\n",
348+
" print(response(user_response))\n",
349+
" sent_tokens.remove(user_response)\n",
350+
" else:\n",
351+
" flag=False\n",
352+
" print(\"ROBO: Bye! take care..\")"
353+
]
354+
},
355+
{
356+
"cell_type": "code",
357+
"execution_count": null,
358+
"metadata": {},
359+
"outputs": [],
360+
"source": []
361+
}
362+
],
363+
"metadata": {
364+
"kernelspec": {
365+
"display_name": "Python 3",
366+
"language": "python",
367+
"name": "python3"
368+
},
369+
"language_info": {
370+
"codemirror_mode": {
371+
"name": "ipython",
372+
"version": 3
373+
},
374+
"file_extension": ".py",
375+
"mimetype": "text/x-python",
376+
"name": "python",
377+
"nbconvert_exporter": "python",
378+
"pygments_lexer": "ipython3",
379+
"version": "3.7.3"
380+
}
381+
},
382+
"nbformat": 4,
383+
"nbformat_minor": 2
384+
}

‎README.md

+52
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Building a Simple Chatbot from Scratch in Python (using NLTK)
2+
3+
![Alt text](https://cdn-images-1.medium.com/max/800/1*pPcVfZ7i-gLMabUol3zezA.gif)
4+
5+
History of chatbots dates back to 1966 when a computer program called ELIZA was invented by Weizenbaum. It imitated the language of a psychotherapist from only 200 lines of code. You can still converse with it here: [Eliza](http://psych.fullerton.edu/mbirnbaum/psych101/Eliza.htm?utm_source=ubisend.com&utm_medium=blog-link&utm_campaign=ubisend).
6+
7+
On similar lines let's create a very basic chatbot utlising the Python's NLTK library.It's a very simple bot with hardly any cognitive skills,but still a good way to get into NLP and get to know about chatbots.
8+
9+
10+
# Outline
11+
* [Motivation](#motivation)
12+
* [Blogpost](#blogpost)
13+
* [Pre-requisites](#pre-requisites)
14+
* [How to run](#how-to-run)
15+
16+
17+
## Motivation
18+
The idea of this project was not to create some SOTA chatbot with exceptional cognitive skills but just to utilise and test my Python skills.This was one of my very first projects, created when I just stepped into the world of NLP and I thought of creating a simple chatbot just to make use of my newly acquired knowledge.
19+
20+
## BlogPost
21+
For detailed overview, here is the accompanying blog titled:**[Building a Simple Chatbot in Python (using NLTK)](https://medium.com/analytics-vidhya/building-a-simple-chatbot-in-python-using-nltk-7c8c8215ac6e)**
22+
23+
24+
## Pre-requisites
25+
**NLTK(Natural Language Toolkit)**
26+
27+
[Natural Language Processing with Python](http://www.nltk.org/book/) provides a practical introduction to programming for language processing.
28+
29+
For platform-specific instructions, read [here](https://www.nltk.org/install.html)
30+
31+
### Installation of NLTK
32+
```
33+
pip install nltk
34+
```
35+
### Installing required packages
36+
After NLTK has been downloaded, install required packages
37+
```
38+
import nltk
39+
from nltk.stem import WordNetLemmatizer
40+
nltk.download('popular', quiet=True) # for downloading popular packages
41+
nltk.download('punkt')
42+
nltk.download('wordnet')
43+
```
44+
45+
## How to run
46+
* Jupyter Notebook [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/parulnith/Building-a-Simple-Chatbot-in-Python-using-NLTK/master)
47+
48+
You can run the [chatbot.ipynb](https://github.com/parulnith/Building-a-Simple-Chatbot-in-Python-using-NLTK/blob/master/Chatbot.ipynb) which also includes step by step instructions.
49+
* Through Terminal
50+
```
51+
python chatbot.py
52+
```

‎_config.yml

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
theme: jekyll-theme-midnight

‎chatbot.py

+92
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
2+
#Meet Robo: your friend
3+
4+
#import necessary libraries
5+
import io
6+
import random
7+
import string # to process standard python strings
8+
import warnings
9+
import numpy as np
10+
from sklearn.feature_extraction.text import TfidfVectorizer
11+
from sklearn.metrics.pairwise import cosine_similarity
12+
import warnings
13+
warnings.filterwarnings('ignore')
14+
15+
import nltk
16+
from nltk.stem import WordNetLemmatizer
17+
nltk.download('popular', quiet=True) # for downloading packages
18+
19+
# uncomment the following only the first time
20+
#nltk.download('punkt') # first-time use only
21+
#nltk.download('wordnet') # first-time use only
22+
23+
24+
#Reading in the corpus
25+
with open('chatbot.txt','r', encoding='utf8', errors ='ignore') as fin:
26+
raw = fin.read().lower()
27+
28+
#TOkenisation
29+
sent_tokens = nltk.sent_tokenize(raw)# converts to list of sentences
30+
word_tokens = nltk.word_tokenize(raw)# converts to list of words
31+
32+
# Preprocessing
33+
lemmer = WordNetLemmatizer()
34+
def LemTokens(tokens):
35+
return [lemmer.lemmatize(token) for token in tokens]
36+
remove_punct_dict = dict((ord(punct), None) for punct in string.punctuation)
37+
def LemNormalize(text):
38+
return LemTokens(nltk.word_tokenize(text.lower().translate(remove_punct_dict)))
39+
40+
41+
# Keyword Matching
42+
GREETING_INPUTS = ("hello", "hi", "greetings", "sup", "what's up","hey",)
43+
GREETING_RESPONSES = ["hi", "hey", "*nods*", "hi there", "hello", "I am glad! You are talking to me"]
44+
45+
def greeting(sentence):
46+
"""If user's input is a greeting, return a greeting response"""
47+
for word in sentence.split():
48+
if word.lower() in GREETING_INPUTS:
49+
return random.choice(GREETING_RESPONSES)
50+
51+
52+
# Generating response
53+
def response(user_response):
54+
robo_response=''
55+
sent_tokens.append(user_response)
56+
TfidfVec = TfidfVectorizer(tokenizer=LemNormalize, stop_words='english')
57+
tfidf = TfidfVec.fit_transform(sent_tokens)
58+
vals = cosine_similarity(tfidf[-1], tfidf)
59+
idx=vals.argsort()[0][-2]
60+
flat = vals.flatten()
61+
flat.sort()
62+
req_tfidf = flat[-2]
63+
if(req_tfidf==0):
64+
robo_response=robo_response+"I am sorry! I don't understand you"
65+
return robo_response
66+
else:
67+
robo_response = robo_response+sent_tokens[idx]
68+
return robo_response
69+
70+
71+
flag=True
72+
print("ROBO: My name is Robo. I will answer your queries about Chatbots. If you want to exit, type Bye!")
73+
while(flag==True):
74+
user_response = input()
75+
user_response=user_response.lower()
76+
if(user_response!='bye'):
77+
if(user_response=='thanks' or user_response=='thank you' ):
78+
flag=False
79+
print("ROBO: You are welcome..")
80+
else:
81+
if(greeting(user_response)!=None):
82+
print("ROBO: "+greeting(user_response))
83+
else:
84+
print("ROBO: ",end="")
85+
print(response(user_response))
86+
sent_tokens.remove(user_response)
87+
else:
88+
flag=False
89+
print("ROBO: Bye! take care..")
90+
91+
92+

‎chatbot.txt

+76
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
A chatbot (also known as a talkbot, chatterbot, Bot, IM bot, interactive agent, or Artificial Conversational Entity) is a computer program or an artificial intelligence which conducts a conversation via auditory or textual methods. Such programs are often designed to convincingly simulate how a human would behave as a conversational partner, thereby passing the Turing test. Chatbots are typically used in dialog systems for various practical purposes including customer service or information acquisition. Some chatterbots use sophisticated natural language processing systems, but many simpler systems scan for keywords within the input, then pull a reply with the most matching keywords, or the most similar wording pattern, from a database.
2+
3+
The term "ChatterBot" was originally coined by Michael Mauldin (creator of the first Verbot, Julia) in 1994 to describe these conversational programs.Today, most chatbots are either accessed via virtual assistants such as Google Assistant and Amazon Alexa, via messaging apps such as Facebook Messenger or WeChat, or via individual organizations' apps and websites. Chatbots can be classified into usage categories such as conversational commerce (e-commerce via chat), analytics, communication, customer support, design, developer tools, education, entertainment, finance, food, games, health, HR, marketing, news, personal, productivity, shopping, social, sports, travel and utilities.
4+
Background
5+
In 1950, Alan Turing's famous article "Computing Machinery and Intelligence" was published, which proposed what is now called the Turing test as a criterion of intelligence. This criterion depends on the ability of a computer program to impersonate a human in a real-time written conversation with a human judge, sufficiently well that the judge is unable to distinguish reliably—on the basis of the conversational content alone—between the program and a real human. The notoriety of Turing's proposed test stimulated great interest in Joseph Weizenbaum's program ELIZA, published in 1966, which seemed to be able to fool users into believing that they were conversing with a real human. However Weizenbaum himself did not claim that ELIZA was genuinely intelligent, and the Introduction to his paper presented it more as a debunking exercise:
6+
7+
[In] artificial intelligence ... machines are made to behave in wondrous ways, often sufficient to dazzle even the most experienced observer. But once a particular program is unmasked, once its inner workings are explained ... its magic crumbles away; it stands revealed as a mere collection of procedures ... The observer says to himself "I could have written that". With that thought he moves the program in question from the shelf marked "intelligent", to that reserved for curios ... The object of this paper is to cause just such a re-evaluation of the program about to be "explained". Few programs ever needed it more.
8+
9+
ELIZA's key method of operation (copied by chatbot designers ever since) involves the recognition of cue words or phrases in the input, and the output of corresponding pre-prepared or pre-programmed responses that can move the conversation forward in an apparently meaningful way (e.g. by responding to any input that contains the word 'MOTHER' with 'TELL ME MORE ABOUT YOUR FAMILY').Thus an illusion of understanding is generated, even though the processing involved has been merely superficial. ELIZA showed that such an illusion is surprisingly easy to generate, because human judges are so ready to give the benefit of the doubt when conversational responses are capable of being interpreted as "intelligent".
10+
11+
Interface designers have come to appreciate that humans' readiness to interpret computer output as genuinely conversational—even when it is actually based on rather simple pattern-matching—can be exploited for useful purposes. Most people prefer to engage with programs that are human-like, and this gives chatbot-style techniques a potentially useful role in interactive systems that need to elicit information from users, as long as that information is relatively straightforward and falls into predictable categories. Thus, for example, online help systems can usefully employ chatbot techniques to identify the area of help that users require, potentially providing a "friendlier" interface than a more formal search or menu system. This sort of usage holds the prospect of moving chatbot technology from Weizenbaum's "shelf ... reserved for curios" to that marked "genuinely useful computational methods".
12+
13+
Development
14+
The classic historic early chatbots are ELIZA (1966) and PARRY (1972).More recent notable programs include A.L.I.C.E., Jabberwacky and D.U.D.E (Agence Nationale de la Recherche and CNRS 2006). While ELIZA and PARRY were used exclusively to simulate typed conversation, many chatbots now include functional features such as games and web searching abilities. In 1984, a book called The Policeman's Beard is Half Constructed was published, allegedly written by the chatbot Racter (though the program as released would not have been capable of doing so).
15+
16+
One pertinent field of AI research is natural language processing. Usually, weak AI fields employ specialized software or programming languages created specifically for the narrow function required. For example, A.L.I.C.E. uses a markup language called AIML, which is specific to its function as a conversational agent, and has since been adopted by various other developers of, so called, Alicebots. Nevertheless, A.L.I.C.E. is still purely based on pattern matching techniques without any reasoning capabilities, the same technique ELIZA was using back in 1966. This is not strong AI, which would require sapience and logical reasoning abilities.
17+
18+
Jabberwacky learns new responses and context based on real-time user interactions, rather than being driven from a static database. Some more recent chatbots also combine real-time learning with evolutionary algorithms that optimise their ability to communicate based on each conversation held. Still, there is currently no general purpose conversational artificial intelligence, and some software developers focus on the practical aspect, information retrieval.
19+
20+
Chatbot competitions focus on the Turing test or more specific goals. Two such annual contests are the Loebner Prize and The Chatterbox Challenge (offline since 2015, materials can still be found from web archives).
21+
22+
According to Forrester (2015), AI will replace 16 percent of American jobs by the end of the decade.Chatbots have been used in applications such as customer service, sales and product education. However, a study conducted by Narrative Science in 2015 found that 80 percent of their respondents believe AI improves worker performance and creates jobs.[citation needed]
23+
24+
Application
25+
See also: Virtual assistant
26+
27+
Aeromexico airline chatbot running on Facebook Messenger, March 2018
28+
Messaging apps
29+
Many companies' chatbots run on messaging apps like Facebook Messenger (since 2016), WeChat (since 2013),WhatsApp, LiveChat, Kik, Slack, Line, Telegram, or simply via SMS. They are used for B2C customer service, sales and marketing.
30+
31+
In 2016, Facebook Messenger allowed developers to place chatbots on their platform. There were 30,000 bots created for Messenger in the first six months, rising to 100,000 by September 2017.
32+
33+
Since September 2017, this has also been as part of a pilot program on WhatsApp. Airlines KLM and Aeroméxico both announced their participation in the testing;both airlines had previously launched customer services on the Facebook Messenger platform.
34+
35+
The bots usually appear as one of the user's contacts, but can sometimes act as participants in a group chat.
36+
37+
Many banks and insurers, media and e-commerce companies, airlines and hotel chains, retailers, health care providers, government entities and restaurant chains have used chatbots to answer simple questions, increase customer engagement,for promotion, and to offer additional ways to order from them.
38+
39+
A 2017 study showed 4% of companies used chatbots.According to a 2016 study, 80% of businesses said they intended to have one by 2020.
40+
41+
As part of company apps and websites
42+
Previous generations of chatbots were present on company websites, e.g. Ask Jenn from Alaska Airlines which debuted in 2008 or Expedia's virtual customer service agent which launched in 2011.The newer generation of chatbots includes IBM Watson-powered "Rocky", introduced in February 2017 by the New York City-based e-commerce company Rare Carat to provide information to prospective diamond buyers.
43+
44+
Company internal platforms
45+
Other companies explore ways they can use chatbots internally, for example for Customer Support, Human Resources, or even in Internet-of-Things (IoT) projects. Overstock.com, for one, has reportedly launched a chatbot named Mila to automate certain simple yet time-consuming processes when requesting for a sick leave.Other large companies such as Lloyds Banking Group, Royal Bank of Scotland, Renault and Citroën are now using automated online assistants instead of call centres with humans to provide a first point of contact. A SaaS chatbot business ecosystem has been steadily growing since the F8 Conference when Zuckerberg unveiled that Messenger would allow chatbots into the app.
46+
47+
Toys
48+
Chatbots have also been incorporated into devices not primarily meant for computing such as toys.
49+
50+
Hello Barbie is an Internet-connected version of the doll that uses a chatbot provided by the company ToyTalk,which previously used the chatbot for a range of smartphone-based characters for children.These characters' behaviors are constrained by a set of rules that in effect emulate a particular character and produce a storyline.
51+
52+
IBM's Watson computer has been used as the basis for chatbot-based educational toys for companies such as CogniToys intended to interact with children for educational purposes.
53+
54+
Chatbot creation
55+
The process of creating a chatbot follows a pattern similar to the development of a web page or a mobile app. It can be divided into Design, Building, Analytics and Maintenance.
56+
57+
Design
58+
The chatbot design is the process that defines the interaction between the user and the chatbot.The chatbot designer will define the chatbot personality, the questions that will be asked to the users, and the overall interaction.It can be viewed as a subset of the conversational design. In order to speed up this process, designers can use dedicated chatbot design tools, that allow for immediate preview, team collaboration and video export.An important part of the chatbot design is also centered around user testing. User testing can be performed following the same principles that guide the user testing of graphical interfaces.
59+
60+
Building
61+
The process of building a chatbot can be divided into two main tasks: understanding the user's intent and producing the correct answer. The first task involves understanding the user input. In order to properly understand a user input in a free text form, a Natural Language Processing Engine can be used.The second task may involve different approaches depending on the type of the response that the chatbot will generate.
62+
63+
Analytics
64+
The usage of the chatbot can be monitored in order to spot potential flaws or problems. It can also provide useful insights that can improve the final user experience.
65+
66+
Maintenance
67+
To keep chatbots up to speed with changing company products and services, traditional chatbot development platforms require ongoing maintenance. This can either be in the form of an ongoing service provider or for larger enterprises in the form of an in-house chatbot training team.To eliminate these costs, some startups are experimenting with Artificial Intelligence to develop self-learning chatbots, particularly in Customer Service applications.
68+
69+
Chatbot development platforms
70+
The process of building, testing and deploying chatbots can be done on cloud based chatbot development platforms offered by cloud Platform as a Service (PaaS) providers such as Yekaliva, Oracle Cloud Platform, SnatchBot and IBM Watson.These cloud platforms provide Natural Language Processing, Artificial Intelligence and Mobile Backend as a Service for chatbot development.
71+
72+
APIs
73+
There are many APIs available for building your own chatbots, such as AARC.
74+
75+
Malicious use
76+
Malicious chatbots are frequently used to fill chat rooms with spam and advertisements, by mimicking human behaviour and conversations or to entice people into revealing personal information, such as bank account numbers. They are commonly found on Yahoo! Messenger, Windows Live Messenger, AOL Instant Messenger and other instant messaging protocols. There has also been a published report of a chatbot used in a fake personal ad on a dating service's website.

0 commit comments

Comments
 (0)
Please sign in to comment.