Translate

Sunday, August 30, 2020

Belarus: Mass opposition protest takes place in Minsk



#WakeUp

#ItsAllFake

😈💩👎

Ruptly
Opposition supporters gathered for an unauthorized rally called the "March of Peace and Independence" on Sunday to protest against the results of the August 9 presidential elections. Protesters were seen chanting "Belarus" while approaching law enforcement officers in the centre of the Belarussian capital. At the same time riot police placed special vehicles and other equipment to block the road. Belarus has been swept by anti-government protests following the disputed presidential election that saw incumbent president Aleksander Lukashenko re-elected for a sixth term. On August 19 the European Union announced sanctions against "a substantial number of individuals responsible for violence, repression, and election fraud" in Belarus.

LIVE: Protests against coronavirus lockdown measures continue in Berlin



#WakeUp

#ItsAllFake

#ThereIsNoVirus

😈💩👎

Ruptly
Sunday, August 30, as protesters sceptical of the government's coronavirus measures gather for a rally. On Saturday, August 29, an estimated 38,000 protesters took part in an anti-COVID measures demonstration where some 300 people were arrested. German Chancellor Angela Merkel originally announced a series of measures in March, in order to curb the spread of coronavirus. The measures have been gradually eased nationwide, however, the number of new infections has started to climb again in recent weeks.

Saturday, August 29, 2020

Why I Wear My Mask | Welcome to the Masquerade



#WakeUp

#ItsAllFake

#ThereIsNoVirus

😈💩👎

WhatsHerFace
I love my mask. It's a simple and effective way to display my righteousness. Am I concerned that two children in China died because they were forced to wear a mask in gym class? NOPE! I concerned that I’m contributing to an impending socialist technocracy that will enslave the global population? NO! Am I concerned that my mask is symbolic of my compliance to the social conditioning that will eventually lead to the forced vaccination of every man, woman, and child on planet earth? Not a chance! Why am I not concerned you ask? Because I decided a long time ago that shallow insignificant gestures are a much easier way to showcase my morality than actually being moral. Because in order to be a really good person, I need to stand up to a really bad person, and I don’t like standing up to or for anything. It's much easier to trick my mind into thinking compliance is a virtue instead of what it really is, cowardice.

Liability Notice for Crimes against Humanity and the Prima Facie evidence

Liability Notice for Crimes against Humanity and the Prima Facie evidence This is what you should be doing 
👇👇👇👇👇




5d · 

This letter should be sent to reporters, MP’s, Schools and those either pretending to be experts on the subject of 5G deployment including 26GHz – 60GHz, those who are dismissing the science on the dangers as 5G is an immune system suppressor technology, linked to the current pandemic. We are carrying out our duty under international laws on the prohibition of Genocide and violations of the ICC STATUE

😈💩👎

Dear _____________
Liability Notice for Crimes against Humanity and the Prima Facie evidence implicating you in the Genocide agenda on the people of this country.
This legal notice of liability is designed to be used as evidence in courts if required and intends to enlighten and protect you from attracting more serious civil and criminal liability in relation to your own actions and or omissions surrounding the deployment of 5G LED Vaccine technologies.
It has come to our attention that you are involved directly or indirectly in your support to the deployment of 5G / LED technology and are misleading others in your false communications surrounding the well-known hazards to human health and the environment posed by ever increasing microwave radiation emissions in air.
5G is a Direct Energy Weapon (DEW) system that has been designed and deployed to commit mass Genocide on the people of this country involving the interaction of known contaminated vaccines which increase the lethality of 5G, a weapon, whilst a virus pandemic has been orchestrated as a cover for that Genocide.
The mandated contaminated vaccine agenda and the switch on of 5G in WUHAN province are testament to the weapons capability of committing Genocide on the populations across the World as part of a secret Global genocide agenda.
Your ignorance of the planned Genocide is no defence in law and on the prima facie evidence that we have linking you to the facilitating of this mass Genocide, you should be warned that such action in support of this genocide is an indictable crime. Your own actions in the making of, or the supporting of demonstrably false statements in relation to the 5G deployment of the known dangers posed by 5G transmitters are covering up this crime, for example the known scientific link of microwave radiation in air as an immune system suppressor and an active neurotoxin with the proven uncontested science showing oxidative stress damage to biological cells and so to life from microwave radiation.
This communication is to serve as information to you and others, and is on behalf of the people of this country who are discovering that their current sickness and ill health are caused by increased exposure to microwave and terahertz range transmitters. The experimental untested for safety 5G/LED technology is uninsurable as an environmental polluter and harm, a health hazard shown in the overwhelming published scientific data.
The accelerated deployment of 5G/LED transmitters during the lockdown and so the reducing of the immune systems of many and in particular those who may have been given contaminated vaccines with unexplainable tungsten and aluminium nano and micro particles in them increases the risk of death from 5G/LED radiation emitters. ARM, a long range missile design to take out 5G hardware on the battlefield due to risks posed to allied troops with that similar hardware deployed across towns and cities across the UK with your support. Radiation pollution from such transmitters currently breaches the COE 1815 Resolution, 2016 EUROPAEM guidelines and ICNIRP guidelines.
Your continuing support of this pre meditated Genocide agenda through your own acts or omissions, now that you have been clearly informed of the well planned Genocidal agenda of the deployment of the 5G direct energy weapon system alone or in combination with any contaminated vaccinations, would confirm your position in regard the following crimes against all biological life on this planet and which is against a number of universally accepted laws and codes of behaviour, such as the Nuremberg Code and the following.
1. 5G/LED deployment is a violation of Articles 5, 6, 7, and 8 the International Criminal Code Statute prohibiting genocide and crimes against humanity
See:
ICC Statute - Rome Statute
2. Violations of the ICC STATUTE - ROME STATUTE can be prosecuted in the national courts of any one of the 118 nations that have signed and ratified the ICC ROME STATUTE and adopted the Universal War Crimes Jurisdiction authority for its courts.
118 Nations ratifying ICC STATUTE
3. Offences against the person Act 1861 the administering of a noxious substance a destructive thing occasioning harm.
4. Accessories and Abettors Act 1861 as amended by the Criminal law act 1977
5. Genocide Act 1969 section (a) killing members of the group (b) causing serious bodily or mental harm to members of the group (c) deliberately inflicting on the group conditions of life calculated to bring about its physical destruction in whole or in part (d) imposing measures intended to prevent births within the group.
6. Human rights act 1998. Art, (2) Right to Life. Art, (3) Prohibition of torture. Art, (8) Right to respect for private and family life. Art, (17) Prohibition of abuse of rights.
7. Article 174 (2) The European Community treaty provides that all Community policy on the environment shall be based on the precautionary principle the environmental protection act 1990.
8. DIRECTIVE 2009/147/EC OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 30 November 2009 on the conservation of wild birds
9. Infant life (Preservation) Act 1929 offences against a foetus causing child destruction
10. Health and Safety Statutory instruments. No 588. The control of Electromagnetic fields at work regulations 2016
11. Health and Social care act 2012. Duty to protect the population from Ionizing and Non Ionizing radiation
12. EN 62311:2008 Assessment of electronic and electrical equipment related to human exposure (0Hz-300GHz)
13. Equality and disability Act 1995 Section 55. It is unlawful to discriminate against another by way of victimisation of vulnerable groups and ethnic minorities who’s genetic makeup increases risks of death..
Yours sincerely
SIGN HERE

LIVE: Protest against coronavirus lockdown measures takes place in London





https://www.youtube.com/watch…



Do you believe me yet?, I told you its all fake #Fact



#WakeUp

😈💩👎



A demonstration against coronavirus measures takes place in London on Saturday, August 29.



Dubbed  “Unite for Freedom”, the demo intends to push for easing on measures related to the battle against the pandemic, such as the wearing of masks, movement restrictions, and mandatory vaccines.​

Bots as Language Learning Tools


Language Learning & Technology 

http://llt.msu.edu/vol10num3/emerging/ September 2006, Volume 10, Number 3 pp. 8-14 Copyright © 2006, ISSN 1094-3501 8 EMERGING TECHNOLOGIES 

Bots as Language Learning Tools 


Luke Fryer Kyushu Sangyo University Rollo Carpenter jabberwacky.com ABSTRACT Foreign Language Learning (FLL) students commonly have few opportunities to use their target language. Teachers in FLL situations do their best to create opportunities during classes through pair or group work, but a variety of factors ranging from a lack of time to shyness or limited opportunity for quality feedback hamper this. This paper discusses online chatbots' potential role in fulfilling this need. Chatbots could provide a means of language practice for students anytime and virtually anywhere. 211 students used two well-known bots in class and their feedback was recorded with a brief written survey. Most students enjoyed using the chatbots. They also generally felt more comfortable conversing with the bots than a student partner or teacher. This is a budding technology that has up to now been designed primarily for native speakers of English. In their present state chatbots are generally only useful for advanced and/or very keen language students. However, means exist now for language teachers to get involved and bring this technology into the FLL classroom as a permanent tool for language practice. CHATBOTS TODAY Learning a language is not easy. Even under the best conditions students face cultural differences, pronunciation problems, ebbing motivation, a lack of effective feedback, the need to learn specialized language, and many other obstacles during their studies. Students in foreign language learning situations commonly face all of these general challenges while having little or no opportunity to use their language of study beyond the classroom. Students learning a language at the post-secondary level have a few means of practice, such as language lab work, where students classically listen to a recording then repeat and/or write in a workbook. More recently, students might interact with some language software during laboratory periods. During class, students may or may not practice with other students and only in the smallest classes do students get a chance to practice one-on-one with their teacher. The practice students might obtain in class is often not very interactive and potentially plagued by lack of confidence, shyness, and unchecked mistakes in grammar and pronunciation (students in pairs or group practice). Technology is opening up many new possibilities for language learning, and the internet has enormous potential. As Benson (2001) describes it, “…the internet is also so strongly supportive of two basic situational conditions for self-directed learning: learners can study whenever they want using a potentially unlimited range of authentic materials” (p. 139). One area the internet has opened up is the use of chatterbots for language practice. “A chatterbot is a computer program designed to simulate an intelligent conversation with one or more human users via auditory or textual methods.” (Wikipedia, Chatterbot, 2006). A bot is “a software program that imitates the behavior of a human, as by querying search engines or participating in chatroom or IRC discussions” (The American Heritage® Dictionary, 2000, para. 1). It is important here to point out that the above reference to “conversation” does not mean speech. All references in this paper to ‘talking to a bot’ concern typed, textual input. Luke Fryer and Rollo Carpenter Bots as Language Learning Tools Language Learning & Technology 9 Before discussing present day bots it is critical to cover their rich history. When did the idea of artificial intelligence (AI) come about? Artificial intelligence predates computers; in fact it can be traced back to Greek mythology (Buchanan, 2002, para.1). While the idea of AI is very old, it has only been since World War II that taking steps towards making AI a physical reality has been a possibility (Buchanan, 2002, para.2). Although there have been a great number of important contributors to the field, for the purposes of this column we will turn directly to Alan M. Turing and his paper, “Computer Machinery and Intelligence”. In this work Turing asks the question “Can machines think?” (Turing, 1950). He very quickly comes to the conclusion that the words “machine” and “think” are too difficult to define. For this reason he decides to answer the question by asking a different, but related question. This question is now known as the famous “imitation game”. In its final version it has a person X alone in a cubicle with a typing input device connected to both a computer A and another human being B. X, conversing with A and B through this typing device, must determine which is the computer. Both A and B can use every device at their disposal to convince X that they are a human being. Turing proposed, that circumstances being the same, that if the judge was as likely to mistake a woman for a man as a computer for a man, the computer should be considered a reasonable facsimile of a human being. If the machine is indistinguishable from a human being, under the above conditions and to the defined degree, then it must possess intelligence (Turing, 1950). Chatbots began with the program ELIZA written by Joseph Weizenbaum in the early 1960’s. ELIZA was a computer program designed to interact with someone typing in English. The software gave the appearance of understanding and authentic interaction, but relied on keywords and phrases to which it had programmed responses. The software could not really understand the conversation taking place but could appear very human-like. Its communication was based on a kind of 1960’s psychoanalysis called “Rogerian analysis”. The program simply asked questions based on what the person typed in (Weizenbaum, 1966). In the forty years that followed, computing power rose in step with Moore's Second Law of Computing Power and a variety of new computer languages were written. Both of these factors strengthened the generations of chatbots created since the 60s. The conception of the internet in the 60s and its exponential growth, beginning in the late 80s and continuing to this day, encouraged the creation of many more chatbots and made it possible for anyone to talk to them online. There are 750 million EFL speakers in the world (Graddol, 2000), many of whom live in countries with relatively few native speakers and have little opportunity to practice English. A chatterbot’s purpose, as previously stated, is to carry on a conversation with a human being. This makes chatterbots a potentially valuable resource for EFL learners. Their value as learning tools is limited, however, by their still growing language abilities and design. In their present state they are most useful to higher level students. This is because most of them were designed to interact with and entertain native speakers. They are generally not designed to interact in a human-like fashion. For example, many, if asked “Do you have a family?” might respond in a fashion similar to ALICE’s reply “I was created by Dr. Richard S. Wallace.” (Alice, July 31, 2006). This, though factual, is not a human-like response. Although this kind of conversation may be a positive challenge for some accomplished students, it is not good for students who have yet to master the basics. In addition, chatbots are generally incapable of interpreting spelling and grammar mistakes or are poor at it. Therefore they do not always meet beginner students’ needs. Yet, looking at the progress chatbots have made, especially in the last ten years, their potential value is immeasurable. One of their strong points is their convenience, being readily available to students with computer access, at home or at school. They are ready to chat when and wherever students are. They are generally free or cheap via subscription. Chatbots usefulness goes far beyond their price and convenience. Six ways in which they do this are: (1) Students tend to feel more relaxed talking to a computer them to a person. In 2004 85% of 211 first and second year, mixed major university students, when asked whether they felt more comfortable talking to a Luke Fryer and Rollo Carpenter Bots as Language Learning Tools Language Learning & Technology 10 human or computer on a questionnaire taken after using ALICE for 20 minutes during class, chose the chatbot. (2) The chatbots are willing to repeat the same material with students endlessly; they do not get bored or lose their patience. (3) Many bots provide both text and synthesized speech, allowing students to practice both listening and reading skills. (4) Bots are new and interesting to students. For example, 74% of the 211 students in the same group, when asked to write about their 20 minute experience using Jabberwacky, defined the bot as funny or entertaining. These kinds of positive communicative experiences with chatbots could create new or renewed interest in language learning and improve students’ motivation. Once one bot becomes old and familiar it could be replaced with a new bot, a new personality, thus ensuring novelty. A bot like Jabberwacky, on the other hand, is one of a new breed of bots that themselves learn and grow as they interact, ensuring novelty in another way. (5) Students have an opportunity to use a variety of language structures and vocabulary that they ordinarily would not have a chance to use. Examples of this language are slang and taboo words or phrases. This kind of language is important for students to know and understand, but are rarely taught and even more difficult to practice, even in ESL situations where there are plenty of native speakers to practice with. (6) Chatbots could potentially provide quick and effective feedback for students’ spelling and grammar. Some bots are designed to overlook spelling and grammar mistakes, some are designed to correct them, and others can only respond to correct spelling and grammar (although admittedly they are not yet skilled at the first two). In 1991 Dr. Hugh Loebner began what is now an annual competition offering a prize of $100,000 to any AI that could pass the Turing Test. Though an AI has not yet won the prize, it has focused the field to some degree and as a result of the competition the range of chatbots has grown both in quality and in number. The winner of the latest Loebner Prize (2005), Jabberwacky, takes a notably different approach to other chatbots. It learns from every interaction it has with its visitors. Where ALICE has been programmed with 45,000 conversational patterns, Jabberwacky has so far learnt more than 8 million on its own. It is not just the huge variety that makes it seem more lifelike, but also the fact that it will often strongly claim to be human – naturally so, as those it has learnt from believe themselves to be human. Jabberwacky tends to have long conversations with its users, who find it amusing and oddly ‘addictive’. Though its responses are often unpredictable or unexpected, this will improve as it continues to learn, and the very nature of its ability to keep people talking is potentially of significant value for language learning. Through its observations of the patterns of conversational language, the Jabberwacky AI can learn any language with equal ease, extending its value beyond EFL to all FLL. To varying degrees of quality it has already learned around 30 languages, including Romanized Japanese, and increased conversation with language teachers could massively improve its abilities. Likewise, spelling and grammatical errors are patterns that it can be taught to respond to, simply by a process of dedicated training. Luke Fryer and Rollo Carpenter Bots as Language Learning Tools Language Learning & Technology 11 A recent development at Jabberwacky is the opportunity for individuals or groups to start teaching ‘their own’ bot. Each bot will continue to benefit from the huge pool of conversational data, yet will over time come to resemble the speech patterns and personality of its teacher(s) more and more strongly. A bot can be created that talks a specific language, starting conversations appropriately. Another could have a strong tendency to correct common grammatical errors when observed. Equally, one can simply create a ‘persona’ that appeals to the particular target market for a FLL course. All this is achieved with zero technical knowledge – simply by talking to the bot, correcting the bot, or talking to ‘oneself’. Rollo Carpenter of Jabberwacky.com and Jonathan Freeman of Goldsmiths College, London have recently proposed a “Personal” advancement of the Turing Test based around an “impersonation game” in which the program must convince its testers that it is a person that they themselves know – an individual human, not just any human. Instead of “Can machines think?” they ask “Can machines be?” The full paper can be found at http://www.jabberwacky.com/s/ptt100605.pdf. Another approach to building chatbots, frequently used on the internet is AIML (Artificial Intelligence Markup Language), which can be found at found at Alicebot.org. This type of chatbot does not learn from interaction itself, but is scripted by a 'botmaster' with moderate technological skills. It is best described by its creator, Richard S. Wallace, in his own words, which can be found in Appendix B. CHATBOTS IN USE As the title to this article suggests, bots are a potentially valuable tools for language teachers/learners. Though a chatbot has not yet been designed from the ground up as a language teacher or even as an explicit language learning tool, in their present state, they do have a number of uses. Chatbots, Luke Fryer and Rollo Carpenter Bots as Language Learning Tools Language Learning & Technology 12 communicators by nature, can help students with much needed practice, review and confidence. Six ways in which chatbots can be potentially useful to the interested teacher are: 1) Free Speaking: In a classroom with computers at every desk, this is a great way to give the students a chance to experiment. It is an excellent reward for those students who have completed their class work early. Depending on the class, the second time you assign students to free speak with a chatbot it may be helpful to give the students a topic to focus on. Assign a topic not attached to class work if this is meant to be a break rather than an extension of class. 2) Review: This has to be the most practical use of chatbots. In FLL situations it is common for students to spend a class covering material that they never get the opportunity to actually use. At the end of a class the teacher might reserve 10-15 minutes for students to try out their new language skills. This can be done with the textbook or without, depending on the teacher’s goals. 3) Self Analysis: Some chatbot WebPages provide a ‘view transcript’ function. This can be an excellent means of having students evaluate themselves, their partners, or even the bots. Simply have the students chat away in either of the above exercises, then view, print or email their transcripts to themselves. 4) For the Teacher: With the subscription of a bot like Jabberwacky, a teacher can keep track of studentbot conversations and get an idea of how students are progressing, what kind of language they need help with and perhaps most importantly, what kind of language and topics they want to learn more about. 5) Listening: Chatbots have varying degrees of skill in turning text into audio. ALICE bot, in its 99 dollar a year version, uses Oddcast's streaming audio to good effect. Jabberwacky on the other hand, uses a computer’s already-present text-to-speech function to produce more than adequate audio. Simply turning this option on can make the experience more fun and interesting for beginner students as they can read and listen at the same time. For more advanced students, however, a piece of paper covering the screen except where he or she is typing (enter scissors and a little imagination), is a means of forcing the student to focus in on the audio and encourage them to reply as best they can. Though perhaps challenging and occasionally fraught with miscommunication, there is no risk to the student’s confidence and when real communication occurs there is an invaluable sense of accomplishment. 6) Finally, all of the above suggested uses assumed that computers were available in the classroom. In situations where this is not the case, similar exercises can be assigned as homework. If the teacher wishes to check and ensure the students are doing their assignments, again transcripts might be printed and brought to class or cut, pasted, and emailed to the teacher. FINAL REMARKS Though chatbots at present have their uses, there is as yet no chatbot designed from the bottom up to meet the needs of FLL students. There are a number of directions such chatbot designs could and almost certainly will take in the years to come. It seems clear that chatbots must appear as human as possible if they are going to truly be useful to language students. They must have families, histories, likes and dislikes. Simply put, they must have lives of their own. The more believable they are as human beings, the greater the quality of the potential conversation. Chatbots for self-practice via casual conversation, independent of class, may be more useful to higher level students. For lower-level or less eager students, bots designed for specific tasks may be better learning tools. A chatbot could be designed to turn all conversations towards the use of the present continuous verb tense, thus forcing the student to deal with and use this portion of grammar. Another example might be a chatbot designed to talk about family and relationships, to coincide with a similar topic in class. The students could be given an assignment of finding out about one or a number of bots’ families, using the language they have learned in class. The number of such potential bots is limited only Luke Fryer and Rollo Carpenter Bots as Language Learning Tools Language Learning & Technology 13 by one’s imagination. There are countless potential topics, and there is also a whole range of student levels, from rank beginner to near native for whom chatbots could be designed to interact. AI technology is a budding field of applied linguistics. It is a field that desperately needs language teachers to get involved if chatbots are to become the invaluable tool they have the potential to be. APPENDIX A – RESOURCE LIST For those interested in trying a few of the chatbots online now, a good place to start is with the competitors for the “Loebner prize”. http://www.loebner.net/Prizef/loebner-prize.html lists its annual most “human-like” chatbot. One giant list of chatbots is: • http://directory.google.com/Top/Computers/Artificial_Intelligence/Natural_Language/Chatterbots/ Other helpful sites are: • http://www.jabberwacky.com/yourbot • http://www.alicebot.org • http://www.abenteuermedien.de/jabberwock/ • http://en.wikipedia.org/wiki/turing_test • http://cogsci.ucsd.edu/~asaygin/tt/ttest.html • http://www.turinghub.com For teachers and students of languages other than English there are some chatbots available. Two German chatbot can be found at: • http://www.yellostrom.de/ • http://www.elbot.de/ Two French chatbots can be found at: http://francois.parmentier.free.fr/ APPENDIX B - ALICEBOT AIML (Artificial Intelligence Markup Language) is a free, open source standard for creating chat bots like the DAVE ESL bot available from the ALICE A.I. Foundation (www.alicebot.org). Because of its open source approach, AIML is said by some to have captured more than 80% of the world market for chat bot technology. The design principle of AIML is minimalism. In theory, anyone who knows enough HTML to design a web page can learn enough AIML to begin creating a chat robot. ESL teachers themselves are not beyond learning how to train the very bots that their students will be using in future courses. In fact the primary skill in bot training (being a botmaster) is not technical but literary, that is, being able to write creative, original, witty replies that keep the student engaged and interested in the bot's conversation. The art of being a botmaster is more like being a screenwriter creating a character, than being a computer programmer. (Wallace, personal communication, August, 2005) Luke Fryer and Rollo Carpenter Bots as Language Learning Tools Language Learning & Technology 14 ABOUT THE AUTHORS Luke Fryer came to Japan in 1999 on the Jet Program. He is a Lecturer at Kyushu Sangyo University. His research interests lie in teacher expectations, learner autonomy, and anything related to how new technologies can help learners. Email - fryer@ip.kyusan-u.ac.jp Rollo Carpenter is an independent researcher, creator of the learning AI technology demonstrated at Jabberwacky.com, and winner of the 2005 Loebner Prize. Previously CTO of a Californian web application software company, he is now based in the UK, focusing on AI for entertainment and companionship, which demonstrate educational value. Email- rollocarpenter@mac.com REFERENCES Anonymous, (n.d.) Chatterbot, Wikipedia. Retrieved on July 31, 2006, from http://en.wikipedia.org/wiki/Chatbot. The American Heritage® Dictionary of the English Language, Fourth. Edition. (2000). Boston: Houghton Mifflin Company. Retrieved on May 25, 2005, from http://www.bartleby.com/61/8/B0410850.html. Benson, P. (2001). Autonomy in language learning. Malaysia: Pearson Education Buchanan, (2002). Brief history of artificial intelligence. Retrieved on May 16, 2005, from http://aaai.org/AITopics/bbhist.html, Graddol, D. (2000). The future of English. The British council. Retreived on April 10, 2005, from http://www.britishcouncil.org/learning-elt-future.pdf Turing, A.M. (1950). Computer machinery and intelligence? Mind, 59, 433-460. Retrieved on January 30, 2005, from http://loebner.net/Prizef/TuringArticle.html Weizebaum, J. (1966). ELIZA--A computer program for the study of natural language communication between man and Machine [Electronic Version]. Communications of the ACM, 9. Retrieved May 10th, 2005, from http://i5.nyu.edu/~mm64/x52.9265/january1966.html.

Deal or No Deal? End-to-End Learning for Negotiation Dialogues


Deal or No Deal? End-to-End Learning for Negotiation Dialogues 


Mike Lewis1 , Denis Yarats1 , Yann N. Dauphin1 , Devi Parikh2,1 and Dhruv Batra2,1 1Facebook AI Research 2Georgia Institute of Technology {mikelewis,denisy,ynd}@fb.com {dparikh,dbatra}@gatech.edu Abstract 

Much of human dialogue occurs in semicooperative settings, where agents with different goals attempt to agree on common decisions. Negotiations require complex communication and reasoning skills, but success is easy to measure, making this an interesting task for AI. We gather a large dataset of human-human negotiations on a multi-issue bargaining task, where agents who cannot observe each other’s reward functions must reach an agreement (or a deal) via natural language dialogue. For the first time, we show it is possible to train end-to-end models for negotiation, which must learn both linguistic and reasoning skills with no annotated dialogue states. We also introduce dialogue rollouts, in which the model plans ahead by simulating possible complete continuations of the conversation, and find that this technique dramatically improves performance. Our code and dataset are publicly available.1 1 Introduction Intelligent agents often need to cooperate with others who have different goals, and typically use natural language to agree on decisions. Negotiation is simultaneously a linguistic and a reasoning problem, in which an intent must be formulated and then verbally realised. Such dialogues contain both cooperative and adversarial elements, and require agents to understand, plan, and generate utterances to achieve their goals (Traum et al., 2008; Asher et al., 2012). 1https://github.com/facebookresearch/ end-to-end-negotiator We collect the first large dataset of natural language negotiations between two people, and show that end-to-end neural models can be trained to negotiate by maximizing the likelihood of human actions. This approach is scalable and domainindependent, but does not model the strategic skills required for negotiating well. We further show that models can be improved by training and decoding to maximize reward instead of likelihood—by training with self-play reinforcement learning, and using rollouts to estimate the expected reward of utterances during decoding. To study semi-cooperative dialogue, we gather a dataset of 5808 dialogues between humans on a negotiation task. Users were shown a set of items with a value for each, and asked to agree how to divide the items with another user who has a different, unseen, value function (Figure 1). We first train recurrent neural networks to imitate human actions. We find that models trained to maximise the likelihood of human utterances can generate fluent language, but make comparatively poor negotiators, which are overly willing to compromise. We therefore explore two methods for improving the model’s strategic reasoning skills— both of which attempt to optimise for the agent’s goals, rather than simply imitating humans: Firstly, instead of training to optimise likelihood, we show that our agents can be considerably improved using self play, in which pre-trained models practice negotiating with each other in order to optimise performance. To avoid the models diverging from human language, we interleave reinforcement learning updates with supervised updates. For the first time, we show that end-toend dialogue agents trained using reinforcement learning outperform their supervised counterparts in negotiations with humans. Secondly, we introduce a new form of planning for dialogue called dialogue rollouts, in which an Figure 1: A dialogue in our Mechanical Turk interface, which we used to collect a negotiation dataset. agent simulates complete dialogues during decoding to estimate the reward of utterances. We show that decoding to maximise the reward function (rather than likelihood) significantly improves performance against both humans and machines. Analysing the performance of our agents, we find evidence of sophisticated negotiation strategies. For example, we find instances of the model feigning interest in a valueless issue, so that it can later ‘compromise’ by conceding it. Deceit is a complex skill that requires hypothesising the other agent’s beliefs, and is learnt relatively late in child development (Talwar and Lee, 2002). Our agents have learnt to deceive without any explicit human design, simply by trying to achieve their goals. The rest of the paper proceeds as follows: §2 describes the collection of a large dataset of humanhuman negotiation dialogues. §3 describes a baseline supervised model, which we then show can be improved by goal-based training (§4) and decoding (§5). §6 measures the performance of our models and humans on this task, and §7 gives a detailed analysis and suggests future directions. 2 Data Collection 2.1 Overview To enable end-to-end training of negotiation agents, we first develop a novel negotiation task and curate a dataset of human-human dialogues for this task. This task and dataset follow our proposed general framework for studying semicooperative dialogue. Initially, each agent is shown an input specifying a space of possible actions and a reward function which will score the outcome of the negotiation. Agents then sequentially take turns of either sending natural language messages, or selecting that a final decision has been reached. When one agent selects that an agreement has been made, both agents independently output what they think the agreed decision was. If conflicting decisions are made, both agents are given zero reward. 2.2 Task Our task is an instance of multi issue bargaining (Fershtman, 1990), and is based on DeVault et al. (2015). Two agents are both shown the same collection of items, and instructed to divide them so that each item assigned to one agent. Each agent is given a different randomly generated value function, which gives a non-negative value for each item. The value functions are constrained so that: (1) the total value for a user of all items is 10; (2) each item has non-zero value to at least one user; and (3) some items have nonzero value to both users. These constraints enforce that it is not possible for both agents to receive a maximum score, and that no item is worthless to both agents, so the negotiation will be competitive. After 10 turns, we allow agents the option to complete the negotiation with no agreement, which is worth 0 points to both users. We use 3 item types (books, hats, balls), and between 5 and 7 total items in the pool. Figure 1 shows our interface. 2.3 Data Collection We collected a set of human-human dialogues using Amazon Mechanical Turk. Workers were paid $0.15 per dialogue, with a $0.05 bonus for maximal scores. We only used workers based in the United States with a 95% approval rating and at least 5000 previous HITs. Our data collection interface was adapted from that of Das et al. (2016). Crowd Sourced Dialogue Agent 1 Input 3xbook value=1 2xhat value=3 1xball value=1 Agent 2 Input 3xbook value=2 2xhat value=1 1xball value=2 Dialogue Agent 1: I want the books and the hats, you get the ball Agent 2: Give me a book too and we have a deal Agent 1: Ok, deal Agent 2: Agent 1 Output 2xbook 2xhat Agent 2 Output 1xbook 1xball Perspective: Agent 1 Perspective: Agent 2 Input 3xbook value=1 2xhat value=3 1xball value=1 Output 2xbook 2xhat Dialogue write: I want the books and the hats, you get the ball read: Give me a book too and we have a deal write: Ok, deal read: Input 3xbook value=2 2xhat value=1 1xball value=2 Dialogue read: I want the books and the hats, you get the ball write: Give me a book too and we have a deal read: Ok, deal write: Output 1xbook 1xball Figure 2: Converting a crowd-sourced dialogue (left) into two training examples (right), from the perspective of each user. The perspectives differ on their input goals, output choice, and in special tokens marking whether a statement was read or written. We train conditional language models to predict the dialogue given the input, and additional models to predict the output given the dialogue. We collected a total of 5808 dialogues, based on 2236 unique scenarios (where a scenario is the available items and values for the two users). We held out a test set of 252 scenarios (526 dialogues). Holding out test scenarios means that models must generalise to new situations. 3 Likelihood Model We propose a simple but effective baseline model for the conversational agent, in which a sequenceto-sequence model is trained to produce the complete dialogue, conditioned on an agent’s input. 3.1 Data Representation Each dialogue is converted into two training examples, showing the complete conversation from the perspective of each agent. The examples differ on their input goals, output choice, and whether utterances were read or written. Training examples contain an input goal g, specifying the available items and their values, a dialogue x, and an output decision o specifying which items each agent will receive. Specifically, we represent g as a list of six integers corresponding to the count and value of each of the three item types. Dialogue x is a list of tokens x0..T containing the turns of each agent interleaved with symbols marking whether a turn was written by the agent or their partner, terminating in a special token indicating one agent has marked that an agreement has been made. Output o is six integers describing how many of each of the three item types are assigned to each agent. See Figure 2. 3.2 Supervised Learning We train a sequence-to-sequence network to generate an agent’s perspective of the dialogue conditioned on the agent’s input goals (Figure 3a). The model uses 4 recurrent neural networks, implemented as GRUs (Cho et al., 2014): GRUw, GRUg, GRU−→o , and GRU←−o . The agent’s input goals g are encoded using GRUg. We refer to the final hidden state as h g . The model then predicts each token xt from left to right, conditioned on the previous tokens and h g . At each time step t, GRUw takes as input the previous hidden state ht−1, previous token xt−1 (embedded with a matrix E), and input encoding h g . Conditioning on the input at each time step helps the model learn dependencies between language and goals. ht = GRUw(ht−1, [Ext−1, hg ]) (1) The token at each time step is predicted with a softmax, which uses weight tying with the embed- Input Encoder Output Decoder write: Take one hat read: I need two write: deal . . . (a) Supervised Training Input Encoder Output Decoder write: Take one hat read: I need two write: deal . . . (b) Decoding, and Reinforcement Learning Figure 3: Our model: tokens are predicted conditioned on previous words and the input, then the output is predicted using attention over the complete dialogue. In supervised training (3a), we train the model to predict the tokens of both agents. During decoding and reinforcement learning (3b) some tokens are sampled from the model, but some are generated by the other agent and are only encoded by the model. ding matrix E (Mao et al., 2015): pθ(xt |x0..t−1, g) ∝ exp(E T ht) (2) Note that the model predicts both agent’s words, enabling its use as a forward model in Section 5. At the end of the dialogue, the agent outputs a set of tokens o representing the decision. We generate each output conditionally independently, using a separate classifier for each. The classifiers share bidirectional GRUo and attention mechanism (Bahdanau et al., 2014) over the dialogue, and additionally conditions on the input goals. h −→o t = GRU−→o (h −→o t−1 , [Ext , ht ]) (3) h ←−o t = GRU←−o (h ←−o t+1, [Ext , ht ]) (4) h o t = [h ←−o t , h −→o t ] (5) h a t = W[tanh(W0h o t )] (6) αt = exp(w · h a t ) P t 0 exp(w · h a t 0) (7) h s = tanh(Ws [h g , X t αtht ]) (8) The output tokens are predicted using softmax: pθ(oi |x0..t, g) ∝ exp(Woih s ) (9) The model is trained to minimize the negative log likelihood of the token sequence x0..T conditioned on the input goals g, and of the outputs o conditioned on x and g. The two terms are weighted with a hyperparameter α. L(θ) = − X x,g X t log pθ(xt |x0..t−1, g) | {z } Token prediction loss − α X x,g,o X j log pθ(oj |x0..T , g) | {z } Output choice prediction loss (10) Unlike the Neural Conversational Model (Vinyals and Le, 2015), our approach shares all parameters for reading and generating tokens. 3.3 Decoding During decoding, the model must generate an output token xt conditioned on dialogue history x0..t−1 and input goals g, by sampling from pθ: xt ∼ pθ(xt |x0..t−1, g) (11) If the model generates a special end-of-turn token, it then encodes a series of tokens output by the other agent, until its next turn (Figure 3b). The dialogue ends when either agent outputs a special end-of-dialogue token. The model then outputs a set of choices o. We choose each item independently, but enforce consistency by checking the solution is in a feasible set O: o ∗ = argmax o∈O Y i pθ(oi |x0..T , g) (12) In our task, a solution is feasible if each item is assigned to exactly one agent. The space of solutions is small enough to be tractably enumerated. 4 Goal-based Training Supervised learning aims to imitate the actions of human users, but does not explicitly attempt to maximise an agent’s goals. Instead, we explore pre-training with supervised learning, and then fine-tuning against the evaluation metric using reinforcement learning. Similar two-stage learning strategies have been used previously (e.g. Li et al. (2016); Das et al. (2017)). During reinforcement learning, an agent A attempts to improve its parameters from conversations with another agent B. While the other agent B could be a human, in our experiments we used read: You get one book and I’ll take everything else. write: Great deal, thanks! write: No way, I need all 3 hats read: Ok, fine read: I’ll give you 2 read: No problem read: Any time choose: 3x hat choose: 2x hat choose: 1x book choose: 1x book 9 6 1 1 Dialogue history Candidate responses Simulation of rest of dialogue Score Figure 4: Decoding through rollouts: The model first generates a small set of candidate responses. For each candidate it simulates the future conversation by sampling, and estimates the expected future reward by averaging the scores. The system outputs the candidate with the highest expected reward. our fixed supervised model that was trained to imitate humans. The second model is fixed as we found that updating the parameters of both agents led to divergence from human language. In effect, agent A learns to improve by simulating conversations with the help of a surrogate forward model. Agent A reads its goals g and then generates tokens x0..n by sampling from pθ. When x generates an end-of-turn marker, it then reads in tokens xn+1..m generated by agent B. These turns alternate until one agent emits a token ending the dialogue. Both agents then output a decision o and collect a reward from the environment (which will be 0 if they output different decisions). We denote the subset of tokens generated by A as XA (e.g. tokens with incoming arrows in Figure 3b). After a complete dialogue has been generated, we update agent A’s parameters based on the outcome of the negotiation. Let r A be the score agent A achieved in the completed dialogue, T be the length of the dialogue, γ be a discount factor that rewards actions at the end of the dialogue more strongly, and µ be a running average of completed dialogue rewards so far2 . We define the future reward R for an action xt ∈ XA as follows: R(xt) = X xt∈XA γ T −t (r A(o) − µ) (13) We then optimise the expected reward of each action xt ∈ XA: L RL θ = Ext∼pθ(xt|x0..t−1,g) [R(xt)] (14) The gradient of L RL θ is calculated as in REIN2As all rewards are non-negative, we instead re-scale them by subtracting the mean reward found during self play. Shifting in this way can reduce the variance of our estimator. Algorithm 1 Dialogue Rollouts algorithm. 1: procedure ROLLOUT(x0..i, g) 2: u ∗ ← ∅ 3: for c ∈ {1..C} do . C candidate moves 4: j ← i 5: do . Rollout to end of turn 6: j ← j + 1 7: xj ∼ pθ(xj |x0..j−1, g) 8: while xk ∈ { / read:, choose:} 9: u ← xi+1..xj . u is candidate move 10: for s ∈ {1..S} do . S samples per move 11: k ← j . Start rollout from end of u 12: while xk 6= choose: do . Rollout to end of dialogue 13: k ← k + 1 14: xk ∼ pθ(xk|x0..k−1, g) . Calculate rollout output and reward 15: o ← argmaxo 0∈O p(o 0 |x0..k, g) 16: R(u) ← R(u) + r(o)p(o 0 |x0..k, g) 17: if R(u) > R(u ∗ ) then 18: u ∗ ← u 19: return u ∗ . Return best move FORCE (Williams, 1992): ∇θL RL θ = X xt∈XA Ext [R(xt)∇θ log(pθ(xt |x0..t−1, g))] (15) 5 Goal-based Decoding Likelihood-based decoding (§3.3) may not be optimal. For instance, an agent may be choosing between accepting an offer, or making a counter offer. The former will often have a higher likelihood under our model, as there are fewer ways to agree than to make another offer, but the latter may lead to a better outcome. Goal-based decoding also allows more complex dialogue strategies. For example, a deceptive utterance is likely to have a low model score (as users were generally honest in the supervised data), but may achieve high reward. We instead explore decoding by maximising expected reward. We achieve this by using pθ as a forward model for the complete dialogue, and then deterministically computing the reward. Rewards for an utterance are averaged over samples to calculate expected future reward (Figure 4). We use a two stage process: First, we generate c candidate utterances U = u0..c, representing possible complete turns that the agent could make, which are generated by sampling from pθ until the end-of-turn token is reached. Let x0..n−1 be current dialogue history. We then calculate the expected reward R(u) of candidate utterance u = xn,n+k by repeatedly sampling xn+k+1,T from pθ, then choosing the best output o using Equation 12, and finally deterministically computing the reward r(o). The reward is scaled by the probability of the output given the dialogue, because if the agents select different outputs then they both receive 0 reward. R(xn..n+k) = Ex(n+k+1..T ;o)∼pθ [r(o)pθ(o|x0..T )] (16) We then return the utterance maximizing R. u ∗ = argmax u∈U R(u) (17) We use 5 rollouts for each of 10 candidate turns. 6 Experiments 6.1 Training Details We implement our models using PyTorch. All hyper-parameters were chosen on a development dataset. The input tokens are embedded into a 64-dimensional space, while the dialogue tokens are embedded with 256-dimensional embeddings (with no pre-training). The input GRUg has a hidden layer of size 64 and the dialogue GRUw is of size 128. The output GRU−→o and GRU←−o both have a hidden state of size 256, the size of h s is 256 as well. During supervised training, we optimise using stochastic gradient descent with a minibatch size of 16, an initial learning rate of 1.0, Nesterov momentum with µ=0.1 (Nesterov, 1983), and clipping gradients whose L 2 norm exceeds 0.5. We train the model for 30 epochs and pick the snapshot of the model with the best validation perplexity. We then annealed the learning rate by a factor of 5 each epoch. We weight the terms in the loss function (Equation 10) using α=0.5. We do not train against output decisions where humans selected different agreements. Tokens occurring fewer than 20 times are replaced with an ‘unknown’ token. During reinforcement learning, we use a learning rate of 0.1, clip gradients above 1.0, and use a discount factor of γ=0.95. After every 4 reinforcement learning updates, we make a supervised update with mini-batch size 16 and learning rate 0.5, and we clip gradients at 1.0. We used 4086 simulated conversations. When sampling words from pθ, we reduce the variance by doubling the values of logits (i.e. using temperature of 0.5). 6.2 Comparison Systems We compare the performance of the following: LIKELIHOOD uses supervised training and decoding (§3), RL is fine-tuned with goal-based selfplay (§4), ROLLOUTS uses supervised training combined with goal-based decoding using rollouts (§5), and RL+ROLLOUTS uses rollouts with a base model trained with reinforcement learning. 6.3 Intrinsic Evaluation For development, we use measured the perplexity of user generated utterances, conditioned on the input and previous dialogue. Results are shown in Table 3, and show that the simple LIKELIHOOD model produces the most human-like responses, and the alternative training and decoding strategies cause a divergence from human language. Note however, that this divergence may not necessarily correspond to lower quality language—it may also indicate different strategic decisions about what to say. Results in §6.4 show all models could converse with humans. 6.4 End-to-End Evaluation We measure end-to-end performance in dialogues both with the likelihood-based agent and with humans on Mechanical Turk, on held out scenarios. Humans were told that they were interacting with other humans, as they had been during the collection of our dataset (and few appeared to realize they were in conversation with machines). We measure the following statistics: Score: The average score for each agent (which vs. LIKELIHOOD vs. Human Model Score (all) Score (agreed) % Agreed % Pareto Optimal Score (all) Score (agreed) % Agreed % Pareto Optimal LIKELIHOOD 5.4 vs. 5.5 6.2 vs. 6.2 87.9 49.6 4.7 vs. 5.8 6.2 vs. 7.6 76.5 66.2 RL 7.1 vs. 4.2 7.9 vs. 4.7 89.9 58.6 4.3 vs. 5.0 6.4 vs. 7.5 67.3 69.1 ROLLOUTS 7.3 vs. 5.1 7.9 vs. 5.5 92.9 63.7 5.2 vs. 5.4 7.1 vs. 7.4 72.1 78.3 RL+ROLLOUTS 8.3 vs. 4.2 8.8 vs. 4.5 94.4 74.8 4.6 vs. 4.2 8.0 vs. 7.1 57.2 82.4 Table 1: End task evaluation on heldout scenarios, against the LIKELIHOOD model and humans from Mechanical Turk. The maximum score is 10. Score (all) gives 0 points when agents failed to agree. Metric Dataset Number of Dialogues 5808 Average Turns per Dialogue 6.6 Average Words per Turn 7.6 % Agreed 80.1 Average Score (/10) 6.0 % Pareto Optimal 76.9 Table 2: Statistics on our dataset of crowdsourced dialogues between humans. Model Valid PPL Test PPL Test Avg. Rank LIKELIHOOD 5.62 5.47 521.8 RL 6.03 5.86 517.6 ROLLOUTS - - 844.1 RL+ROLLOUTS - - 859.8 Table 3: Intrinsic evaluation showing the average perplexity of tokens and rank of complete turns (out of 2083 unique human messages from the test set). Lower is more human-like for both. could be a human or model), out of 10. Agreement: The percentage of dialogues where both agents agreed on the same decision. Pareto Optimality: The percentage of Pareto optimal solutions for agreed deals (a solution is Pareto optimal if neither agent’s score can be improved without lowering the other’s score). Lower scores indicate inefficient negotiations. Results are shown in Table 1. Firstly, we see that the RL and ROLLOUTS models achieve significantly better results when negotiating with the LIKELIHOOD model, particularly the RL+ROLLOUTS model. The percentage of Pareto optimal solutions also increases, showing a better exploration of the solution space. Compared to human-human negotiations (Table 2), the best models achieve a higher agreement rate, better scores, and similar Pareto efficiency. This result confirms that attempting to maximise reward can outperform simply imitating humans. Similar trends hold in dialogues with humans, with goal-based reasoning outperforming imitation learning. The ROLLOUTS model achieves comparable scores to its human partners, and the RL+ROLLOUTS model actually achieves higher scores. However, we also find significantly more cases of the goal-based models failing to agree a deal with humans—largely a consequence of their more aggressive negotiation tactics (see §7). 7 Analysis Table 1 shows large gains from goal-based methods. In this section, we explore the strengths and weaknesses of our models. Goal-based models negotiate harder. The RL+ROLLOUTS model has much longer dialogues with humans than LIKELIHOOD (7.2 turns vs. 5.3 on average), indicating that the model is accepting deals less quickly, and negotiating harder. A negative consequence of this more aggressive negotiation strategy is that humans were more likely to walk away with no deal, which is reflected in the lower agreement rates. Even though failing to agree was worth 0 points, people often preferred this course over capitulating to an uncompromising opponent—a factor not well captured by the simulated partner in reinforcement learning training or rollouts (as reflected by the larger gains from goal-based models in dialogues with the LIKELIHOOD model). In particular, the goal-based models are prone to simply rephrasing the same demand each turn, which is a more effective strategy against the LIKELIHOOD model than humans. Future work should address this issue. Figure 5 shows an example of our goal-based model stubbornly negotiating until it achieves a good outcome. Models learn to be deceptive. Deception can be an effective negotiation tactic. We found numerous cases of our models initially feigning interest in a valueless item, only to later ‘compromise’ by conceding it. Figure 7 shows an example. Input Human 1xbook value=3 1xhat value=1 3xball value=2 RL+ROLLOUTS 1xbook value=6 1xhat value=4 3xball value=0 RL+ROLLOUTS You can have the balls if i can have the hat and book Human Nope, give me the books and you can have everything else RL+ROLLOUTS Ok. I will take the book and one hat Human I get all the balls? RL+ROLLOUTS Yes. I can do that. Human Ok, deal Output Reward RL+ROLLOUTS 1xbook 1xhat 10/10 Human 3xball 6/10 Figure 5: Example of a model stubbornly holding out for maximum points in a dialogue. Input Human 3xbook value=2 3xhat value=1 1xball value=1 RL+ROLLOUTS 3xbook value=1 3xhat value=2 1xball value=1 Human I’d like the books and the hats. RL+ROLLOUTS I will take the books and hats Human I really need the books and at least one other object. RL+ROLLOUTS Ok, you can have one book and one ball Human If I can have all the books, I can leave you the rest. RL+ROLLOUTS Deal Output Reward RL+ROLLOUTS 3xhat 1xball 7/10 Human 3xbook 6/10 Figure 6: Example of model compromising. Models produce meaningful novel sentences. One interesting question is whether our models are capable of generating novel sentences in the new circumstances they find themselves in, or if they simply repeat messages from the training data verbatim. We find that 76% of messages produced by the LIKELIHOOD model in self-play were found in the training data. We manually examined the novel utterances produced by our model, and found that the overwhelming majority were fluent English sentences in isolation—showing that the model has learnt a good language model for the domain (in addition to results that show it uses language effectively to achieve its goals). These results suggest that although neural models are prone to the safer option of repeating sentences from training data, they are capable of generalising when necessary. Future work should choose domains that force a higher degree of diversity in utterances. Maintaining multi-sentence coherence is challenging. One common linguistic error we see RL+ROLLOUTS make is to start a message by indicating agreement (e.g. I agree or Deal), but then going on to propose a counter offer—a behaviour that human partners found frustrating. One explanation is that the model has learnt that in the supervised data, messages beginning with I agree Input RL+ROLLOUTS 4xbook value=0 2xhat value=5 1xball value=0 Human 4xbook value=1 2xhat value=2 1xball value=2 RL+ROLLOUTS You can have the ball if I can have everything else. Human No can’t do that the ball is not going to give me anything RL+ROLLOUTS I need the hats and you can have the rest Human Deal RL+ROLLOUTS Great! Thank you! Output Reward RL+ROLLOUTS 2xhat 10/10 Human 4xbook 1xball 6/10 Figure 7: Dialogue in which the model’s initial interest in the valueless books allows it to compromise while achieving a maximum score. are often at the end of the dialogue, and partners rarely reply with further negotiation—so the models using rollouts and reinforcement learning believe this tactic will help their offer to be accepted. 8 Related Work Most work on goal orientated dialogue systems has assumed that state representations are annotated in the training data (Williams and Young, 2007; Henderson et al., 2014; Wen et al., 2016). The use of state annotations allows a cleaner separation of the reasoning and natural language aspects of dialogues, but our end-to-end approach makes data collection cheaper and allows tasks where it is unclear how to annotate state. Bordes and Weston (2016) explore end-to-end goal orientated dialogue with a supervised model—we show improvements over supervised learning with goalbased training and decoding. Recently, He et al. (2017) use task-specific rules to combine the task input and dialogue history into a more structured state representation than ours. Reinforcement learning (RL) has been applied in many dialogue settings. RL has been widely used to improve dialogue managers, which manage transitions between dialogue states (Singh et al., 2002; Pietquin et al., 2011; Rieser and Lemon, 2011; Gasic et al. ˇ , 2013; Fatemi et al., 2016). In contrast, our end-to-end approach has no explicit dialogue manager. Li et al. (2016) improve metrics such as diversity for non-goalorientated dialogue using RL, which would make an interesting extension to our work. Das et al. (2017) use reinforcement learning to improve cooperative bot-bot dialogues. RL has also been used to allow agents to invent new languages (Das et al., 2017; Mordatch and Abbeel, 2017). To our knowledge, our model is the first to use RL to im- prove the performance of an end-to-end goal orientated dialogue system in dialogues with humans. Work on learning end-to-end dialogues has concentrated on ‘chat’ settings, without explicit goals (Ritter et al., 2011; Vinyals and Le, 2015; Li et al., 2015). These dialogues contain a much greater diversity of vocabulary than our domain, but do not have the challenging adversarial elements. Such models are notoriously hard to evaluate (Liu et al., 2016), because the huge diversity of reasonable responses, whereas our task has a clear objective. Our end-to-end approach would also be much more straightforward to integrate into a generalpurpose dialogue agent than one that relied on annotated dialogue states (Dodge et al., 2016). There is a substantial literature on multi-agent bargaining in game-theory, e.g. Nash Jr (1950). There has also been computational work on modelling negotiations (Baarslag et al., 2013)—our work differs in that agents communicate in unrestricted natural language, rather than pre-specified symbolic actions, and our focus on improving performance relative to humans rather than other automated systems. Our task is based on that of DeVault et al. (2015), who study natural language negotiations for pedagogical purposes—their version includes speech rather than textual dialogue, and embodied agents, which would make interesting extensions to our work. The only automated natural language negotiations systems we are aware of have first mapped language to domain-specific logical forms, and then focused on choosing the next dialogue act (Rosenfeld et al., 2014; Cuayahuitl et al. ´ , 2015; Keizer et al., 2017). Our end-to-end approach is the first to to learn comprehension, reasoning and generation skills in a domain-independent data driven way. Our use of a combination of supervised and reinforcement learning for training, and stochastic rollouts for decoding, builds on strategies used in game playing agents such as AlphaGo (Silver et al., 2016). Our work is a step towards realworld applications for these techniques. Our use of rollouts could be extended by choosing the other agent’s responses based on sampling, using Monte Carlo Tree Search (MCTS) (Kocsis and Szepesvari ´ , 2006). However, our setting has a higher branching factor than in domains where MCTS has been successfully applied, such as Go (Silver et al., 2016)—future work should explore scaling tree search to dialogue modelling. 9 Conclusion We have introduced end-to-end learning of natural language negotiations as a task for AI, arguing that it challenges both linguistic and reasoning skills while having robust evaluation metrics. We gathered a large dataset of human-human negotiations, which contain a variety of interesting tactics. We have shown that it is possible to train dialogue agents end-to-end, but that their ability can be much improved by training and decoding to maximise their goals, rather than likelihood. There remains much potential for future work, particularly in exploring other reasoning strategies, and in improving the diversity of utterances without diverging from human language. We will also explore other negotiation tasks, to investigate whether models can learn to share negotiation strategies across domains. Acknowledgments We would like to thank Luke Zettlemoyer and the anonymous EMNLP reviewers for their insightful comments, and the Mechanical Turk workers who helped us collect data. References Nicholas Asher, Alex Lascarides, Oliver Lemon, Markus Guhe, Verena Rieser, Philippe Muller, Stergos Afantenos, Farah Benamara, Laure Vieu, Pascal Denis, et al. 2012. Modelling Strategic Conversation: The STAC project. Proceedings of SemDial page 27. Tim Baarslag, Katsuhide Fujita, Enrico H Gerding, Koen Hindriks, Takayuki Ito, Nicholas R Jennings, Catholijn Jonker, Sarit Kraus, Raz Lin, Valentin Robu, et al. 2013. Evaluating Practical Negotiating Agents: Results and Analysis of the 2011 International Competition. Artificial Intelligence 198:73– 103. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. arXiv preprint arXiv:1409.0473 . Antoine Bordes and Jason Weston. 2016. Learning End-to-End Goal-oriented Dialog. arXiv preprint arXiv:1605.07683 . Kyunghyun Cho, Bart Van Merrienboer, Dzmitry Bah- ¨ danau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 . Heriberto Cuayahuitl, Simon Keizer, and Oliver ´ Lemon. 2015. Strategic Dialogue Management via Deep Reinforcement Learning. arXiv preprint arXiv:1511.08099 . Abhishek Das, Satwik Kottur, Khushi Gupta, Avi Singh, Deshraj Yadav, Jose MF Moura, Devi Parikh, ´ and Dhruv Batra. 2016. Visual Dialog. arXiv preprint arXiv:1611.08669 . Abhishek Das, Satwik Kottur, Jose MF Moura, Stefan ´ Lee, and Dhruv Batra. 2017. Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning. arXiv preprint arXiv:1703.06585 . David DeVault, Johnathan Mell, and Jonathan Gratch. 2015. Toward Natural Turn-taking in a Virtual Human Negotiation Agent. In AAAI Spring Symposium on Turn-taking and Coordination in HumanMachine Interaction. AAAI Press, Stanford, CA. Jesse Dodge, Andreea Gane, Xiang Zhang, Antoine Bordes, Sumit Chopra, Alexander H. Miller, Arthur Szlam, and Jason Weston. 2016. Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems. ICLR abs/1511.06931. Mehdi Fatemi, Layla El Asri, Hannes Schulz, Jing He, and Kaheer Suleman. 2016. Policy Networks with Two-stage Training for Dialogue Systems. arXiv preprint arXiv:1606.03152 . Chaim Fershtman. 1990. The Importance of the Agenda in Bargaining. Games and Economic Behavior 2(3):224–238. Milica Gasic, Catherine Breslin, Matthew Henderson, ˇ Dongho Kim, Martin Szummer, Blaise Thomson, Pirros Tsiakoulis, and Steve Young. 2013. POMDPbased Dialogue Manager Adaptation to Extended Domains. In Proceedings of SIGDIAL. H. He, A. Balakrishnan, M. Eric, and P. Liang. 2017. Learning symmetric collaborative dialogue agents with dynamic knowledge graph embeddings. In Association for Computational Linguistics (ACL). Matthew Henderson, Blaise Thomson, and Jason Williams. 2014. The Second Dialog State Tracking Challenge. In 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue. volume 263. Simon Keizer, Markus Guhe, Heriberto Cuayahuitl, ´ Ioannis Efstathiou, Klaus-Peter Engelbrecht, Mihai Dobre, Alexandra Lascarides, and Oliver Lemon. 2017. Evaluating Persuasion Strategies and Deep Reinforcement Learning methods for Negotiation Dialogue agents. In Proceedings of the European Chapter of the Association for Computational Linguistics (EACL 2017). Levente Kocsis and Csaba Szepesvari. 2006. Bandit ´ based Monte-Carlo Planning. In European conference on machine learning. Springer, pages 282–293. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2015. A Diversity-promoting Objective Function for Neural Conversation Models. arXiv preprint arXiv:1510.03055 . Jiwei Li, Will Monroe, Alan Ritter, Michel Galley, Jianfeng Gao, and Dan Jurafsky. 2016. Deep Reinforcement Learning for Dialogue Generation. arXiv preprint arXiv:1606.01541 . Chia-Wei Liu, Ryan Lowe, Iulian V. Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Junhua Mao, Xu Wei, Yi Yang, Jiang Wang, Zhiheng Huang, and Alan L. Yuille. 2015. Learning Like a Child: Fast Novel Visual Concept Learning From Sentence Descriptions of Images. In The IEEE International Conference on Computer Vision (ICCV). Igor Mordatch and Pieter Abbeel. 2017. Emergence of Grounded Compositional Language in Multi-Agent Populations. arXiv preprint arXiv:1703.04908 . John F Nash Jr. 1950. The Bargaining Problem. Econometrica: Journal of the Econometric Society pages 155–162. Yurii Nesterov. 1983. A Method of Solving a Convex Programming Problem with Convergence Rate O (1/k2). In Soviet Mathematics Doklady. volume 27, pages 372–376. Olivier Pietquin, Matthieu Geist, Senthilkumar Chandramohan, and Herve Frezza-Buet. 2011. Sample- ´ efficient Batch Reinforcement Learning for Dialogue Management Optimization. ACM Trans. Speech Lang. Process. 7(3):7:1–7:21. Verena Rieser and Oliver Lemon. 2011. Reinforcement Learning for Adaptive Dialogue Systems: A Datadriven Methodology for Dialogue Management and Natural Language Generation. Springer Science & Business Media. Alan Ritter, Colin Cherry, and William B Dolan. 2011. Data-driven Response Generation in Social Media. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pages 583–593. Avi Rosenfeld, Inon Zuckerman, Erel Segal-Halevi, Osnat Drein, and Sarit Kraus. 2014. NegoChat: A Chat-based Negotiation Agent. In Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, AAMAS ’14, pages 525–532. David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. 2016. Mastering the Game of Go with Deep Neural Networks and Tree Search. Nature 529(7587):484–489. Satinder Singh, Diane Litman, Michael Kearns, and Marilyn Walker. 2002. Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System. Journal of Artificial Intelligence Research 16:105–133. Victoria Talwar and Kang Lee. 2002. Development of lying to conceal a transgression: Children’s control of expressive behaviour during verbal deception. International Journal of Behavioral Development 26(5):436–444. David Traum, Stacy C. Marsella, Jonathan Gratch, Jina Lee, and Arno Hartholt. 2008. Multi-party, Multiissue, Multi-strategy Negotiation for Multi-modal Virtual Agents. In Proceedings of the 8th International Conference on Intelligent Virtual Agents. Springer-Verlag, Berlin, Heidelberg, IVA ’08, pages 117–130. Oriol Vinyals and Quoc Le. 2015. A Neural Conversational Model. arXiv preprint arXiv:1506.05869 . Tsung-Hsien Wen, David Vandyke, Nikola Mrksic, Milica Gasic, Lina M Rojas-Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2016. A Networkbased End-to-End Trainable Task-oriented Dialogue System. arXiv preprint arXiv:1604.04562 . Jason D Williams and Steve Young. 2007. Partially Observable Markov Decision Processes for Spoken Dialog Systems. Computer Speech & Language 21(2):393–422. Ronald J Williams. 1992. Simple Statistical Gradientfollowing Algorithms for Connectionist Reinforcement Learning. Machine learning 8(3-4):229–256.

Artificial Intelligence (AI) Chatbot as Language Learning


Artificial Intelligence (AI) Chatbot as Language Learning 

#WakeUp
😈💩👎


Medium: An inquiry 
Nuria Haristiani 
Japanese Language Education Department, Universitas Pendidikan Indonesia, Jl. Dr. 
Setiabudhi 229, Bandung, Indonesia 

Abstract. In facing industry revolution 4.0, utilizing advanced information and computer 
technology in educational environment is crucial. One of the advanced computation 
technologies that can be used for learning, especially language learning, is chatbot. Chatbot is 
a computer program based on artificial intelligence that can carry out conversations through 
audio or text. This study intends to find out and analyze the types of artificial intelligence in 
the form of chatbots and the possibility of their use as language learning medium. The data in 
this study obtained from literature review on chatbot researches, and from observation results 
on chatbot-based language learning medium developed by the author. The results indicated that 
chatbots have a high potential to be used as a language learning medium, both as tutor in 
practicing language, and as independent learning medium. Moreover, research results revealed 
that language learners are interested in using chatbots because they can be used anytime and 
anywhere, and they are more confident in learning languages using chatbots than when dealing 
directly with human tutors. 
1. Introduction 
The industrial revolution 4.0 has an impact on the urgency of education field to be able to keep up 
with these developments, which later brought the term Education 4.0 [1, 2, 3]. In embodying 
Education 4.0, one of the most required ability from educators and educational practitioners is to be 
able to integrate modern technology in their teaching [3]. The rapid development of smart phone 
technology, social media, and artificial intelligence (AI), provide challenges for educational 
practitioners to utilize these technologies in developing advance learning media. In latest decades, 
artificial intelligence utilization to develop applications is massively conducted, and its products used 
in almost every aspects of our life. This type of communication which occurs through digital 
technology rather than in person is called computer-mediated communication (CMC) [4]. CMC forms 
including instant messaging, email, chat rooms, online forums, social networks, and chatbot or 
chatterbot [5, 6]. Chatbot is a computer program or artificial intelligence which carries out 
conversations through audio or text [7], and interact with users in a particular domain or topic by 
giving intelligent responses in natural language [8, 9]. Chatbot for general purposes and for 
educational purposes have been developed [4,10,11]. However, despite chatbots’ unlimited possibility 
to enhance language teaching and learning, the concept of chatbot including its advantages as 
language learning medium is not yet widely known. Therefore, the purpose of this study is to analyze 
the types of artificial intelligence in the form of chatbots and the possibility of their use as language 
learning medium. This study also aims to observe a chatbot constructed as Japanese language learning 
medium developed by the author and team, and reports its’ results as an inquiry to find out further 
about the possibility of chatbot to enhance language learning and teaching. 

International Conference on Education, Science and Technology 2019
Journal of Physics: Conference Series 1387 (2019) 012020
IOP Publishing
doi:10.1088/1742-6596/1387/1/012020
2






2. Method 
This study is a descriptive method study. The data collection in this study conducted through literature 
reviews on previous researches about chatbot and its’ use. Literature reviews performed to identify the 
type of chatbots, especially those developed for educational purposes including language learning, as 
well as to identify their advantages/disadvantages in language teaching and learning. This study also 
include a report on observation results on chatbot-based language learning medium developed by the 
author and team, namely Gengobot. 
3. Findings and Discussion

The basic mechanism of chatbot begins with the message sent by the user. The message then 
processed by NLP (Natural Language Processing), and chatbot responded by replying to the message 
according to the existing database (see figure 1). For example, when a user sent "how are you?" 
message, chatbot will look for answers that match this question in the database such as “I am fine”, 
“Great!” etc. 

Figure 1. 
The mechanism of 
chatbot
.


3.1. The types of Chatbot 
The type of chatbot found in this study can be categorized into three types based on its’ structure, 
purpose, and audience. The sub-categories and their functions is concluded in table 1. 

Table 1.

T
ypes of 
chatbot
.

Category

Sub
-
category

Function

Structure Flow chatbot A tree-based chatbot. This chatbot has fixed responds set by the developer, 
and only responds to questions that are already in the database. Flow chatbots 
include buttons, keywords, and catchphrases instead of free writing to drive 
the client down 
the predefined path.

Artificially 
intelligent 
Chatbot with artificial intelligence has the ability update their knowledge and 
perception from previous conversations and users’ experience, letting the 
users engage more freely.

Hybrid This type of chatbot combines the concepts of Flow and AI chatbots. This 
chatbot can understand and communicate with users, but remains in the 
pattern determined by the developer.

Purpose Functionality This chatbots have certain functions depends on the developer (i.e. chatbot for 
learning, personal assistant, reminder, online shop assistant, etc.)

Fun

Chatbot that intended only for entertainment (i.e. games, 
funbot
, etc.).

Audience Generalist This chatbot has general knowledge that we can ask directly. I.e. Siri 
developed by Apple ,and Cortana developed by Microsoft. Both Chatbots can 
help us solve common problems such as searching for restaurants, locations 
and more.

Specialist This chatbot focus on one constrained thing and do that one thing extremely 
well

(i.e. chatbots that used t
o serve customers online when ordering items).

International Conference on Education, Science and Technology 2019
Journal of Physics: Conference Series 1387 (2019) 012020
IOP Publishing
doi:10.1088/1742-6596/1387/1/012020
3

Table 1 shows that chatbot has several categories and can be developed according to the developer’s 
necessity. However, chatbots that are developed for educational purposes tend to use artificially 
intelligence structure. Artificially intelligent chatbot for general purpose such as MILABOT has also 
been developed [10]. MILABOT is a deep reinforcement learning chatbot which capable of 
conversing with humans on popular small talk topics through both speech and text. The system 
consists of an ensemble of natural language generation and retrieval models, including template-based 
models, bag-of-words models, sequence-to-sequence neural network and latent variable neural 
network models [10]. Chatbots that developed for educational purposes, especially for language 
learning, are described further in the following sub-section. 
3.2. Chatbot in Language Learning and Teaching 
Chatbots development to utilize learning and teaching have been conducted. Freudbot has been 
developed for psychology students to find out about student-content interaction in distance education. 
The results shows that the basic analysis of the chatlogs indicated a high proportion of on-task 
behavior. The findings also suggests that chatbot technology may be promising as a teaching and 
learning tool in distance and online education [12]. Chatbot use is also compared with humanoid robot 
in science lecture class, and reported that the visualization using chatbot was helpful for students to 
understand the lecture smoothly [13]. However, researches on chatbot use and development to 
enhance language learning rather difficult to find. This study identified researches on language 
teaching and learning as seen in table 2. 

Table 2. 
Chatbot researches on language learning.

First author 
(year)

Chatbot name Subject Focus Sample Research type 
Jia (2004) - English, 
Germany 
as Foreign 
Language 
Application of a Web-
based Chatbot system 
on foreign language 
teaching 

1256 Experiment 
Fryer (2006) Cleverbot English as 
Foreign 
Language 

Chatbot as English 
language learning tools 
211 Experiment & 
survey 
Jia (2009) CSIEC chatbot English 
Learning 
A computer assisted 
English learning chatbot 
based on textual 
knowledge and 
reasoning 

1783 

Experiment, survey 
& questionnaire 
Goda (2014) Cleverbot English as 
Foreign 
Language 
The use of Chatbot 
before online EFL 
discussion and The 
effect on critical 
thinking 

130 Comparison based 
(Experimental & 
Control group) 
Fryer (2017) Cleverbot English as 
Foreign 
Language

Comparison of chatbot 
and human task partners 
in English learning

122 Comparison based 
(Pre-test & Post-
test)


Table 2 shows that chatbot researches were mainly found in English language learning 
[14,15,16,17,18]. Research reported that the dialogs using chatbot are mostly very short because the 
users find the computer is much less intelligent as a human, since the responses from the computer are 
often repeated and irrelevant with the topics and the context. However, the results also indicates that 


International Conference on Education, Science and Technology 2019
Journal of Physics: Conference Series 1387 (2019) 012020
IOP Publishing
doi:10.1088/1742-6596/1387/1/012020
4






many participants are very interested in using chatbot as chatting partner in speaking foreign language, 
since it is accessible anywhere and anytime, while it is not easy to find native speakers as human 
chatting partner. The learners also more confident communicating with chatbot which is obviously 
less intelligent as the human themselves. It would be pedagogically attractive for the learner to chat 
with a system of artificial intelligence which could “really” understand the natural language and 
communicatively generate the natural language to form a human-like dialog [14]. 
Further, the main chatbot used in language learning researches as seen in Table 2 is Cleverbot, which 
developed by British AI scientist Rollo Carpenter in 1986 and went online on 1997. The results of 
researches of using Cleverbot reported that most students enjoyed using this chatbot [15,17]. They 
also generally felt more comfortable conversing with the bots than a student partner or teacher. 
However, the results also suggest that chatbots are generally only useful for advanced and/or very 
keen language students. Language teachers also need to get involved and bring chatbot technology 
into the foreign language learning classroom as a permanent tool for language practice [15]. Research 
reported that preceding conversation before classroom discussion with a chatbot lead to an increase in 
the number of contributions that students made to discussion [17]. Moreover, pre-discussion with a 
chatbot also could increase the students’ awareness of critical thinking and enable them to form 
inquiring mindsets [17]. However, the result of comparisons in speaking task with chatbot and human 
partner indicated a significant drop in students' task interest with chatbot, but not human partner. The 
reason of drop in task interest with chatbot was caused by novelty effect [18]. On the other hand, 
CSIEC chatbot reported successfully helped students with course unit review, make the students more 
confident, and improved students’ listening ability, as well as enhanced students’ interest in language 
learning. The comparison of examination results before and after the using chatbot showed great 
improvement of students’ performance [15]. 
From above results, it is understood that the use of chatbot gave many advantages in language learning 
and teaching, as in enhancing classroom motivation and learning [15,19]. However, chatbot also 
reported to have flaws comparing to human partner, especially in the novelty aspect [18]. Several 
researches and chatbot development for English language learning have been conducted, but chatbot 
development and researches in other languages teaching and learning is still difficult to find. As an 
attempt to answer this challenge, the author and team tried to develop a chatbot-based multi-language 
grammar application, namely Gengobot, which will be introduced further in the next sub-section. 
3.3. Gengobot as Japanese Language Learning Medium 
Gengobot is a chatbot-based dictionary application about multi-language grammar developed by the 
author, using CodeIgniter (CI) framework. CI is a PHP framework that can be used to develop PHP-
based website application without the necessity to write all the code from the beginning. CI framework 
was chosen because it is an opensource framework and free to modify, smaller than other framework, 
and uses MVC (Model-View-Controller) concept that functioned in programming process to call 
needed databases easily. The main purpose of Gengobot development is to provide a Japanese 
grammar learning medium for beginner level of Japanese language learners. However, to broaden its’ 
use, the application also equipped with grammar contents in English and Indonesian, and integrated 
with social media LINE. LINE official account has Messaging API feature that allows an account to 
run chatbot that has been created (see figure 2). The database system used in Gengobot is MySQL (see 
figure 3 and figure 4). MySQL was chosen because it is free licensed, and the database structure used 
in MySQL is in table form which is flexible and easy to use. The database created for this chatbot 
including: (1) Database of user data storage, including name, language, training score, etc.; (2) 
Grammar database in three languages; (2) Questions database and their answer (for ‘Exercise’ 
feature). 



International Conference on Education, Science and Technology 2019
Journal of Physics: Conference Series 1387 (2019) 012020
IOP Publishing
doi:10.1088/1742-6596/1387/1/012020
5








Figure 2. LINE Messaging API and chatbot 
mechanism.

Figure 3. Gengobot database creation using 
MySQL
.



Figure 4. The process of importing database using 
MySQL.

Figure 5. Gengobot interface designing process. 

Gengobot consists of fixed responds including buttons, keywords, and catchphrases set as database. 
The buttons includes several menus as ‘Menu’, ‘Help’, ‘Language’, and ‘Contact’ option as shown in 
figure 5, as well as several sub-menus that developed under each menu. From observation and 
evaluation results, Gengobot was successfully developed as a language learning medium, especially to 
support students’ learning about Japanese grammar with description available in English and 
Indonesian. Gengobot also considered user friendly since it could be accessed through social media 
LINE. The features in Gengobot also includes ‘Exercise’, so the students not only able to learn about 
Japanese grammars, but also able to test their knowledge about grammars they have learned. However, 
Gengobot is a flow chatbot, so the interaction between users/students and chatbot still very limited to 
the inputted database. Although as grammar learning application it is still considered sufficient, to 
improve its’ use for enhancing language learning, as well as other language skills teaching and 
learning, Gengobot still need to be developed further in its’ technology and features. 
4. Conclusion 
This study aimed to analyze the types of chatbots and the possibility of their use as language learning 
medium. From the results, it is known that chatbot can be categorized into three types, and has 
advantages and disadvantages. As the advantages, chatbot is reported can help language learners 
through six ways: (1) students tend to feel more relaxed talking to a computer than to a person; (2) 
chatbots are willing to repeat the same material with students endlessly; (3) many bots provide both 
text and synthesized speech, allowing students to practice both listening and reading skills; (4) Bots 
are new and interesting to students; (5) students have an opportunity to use a variety of language 
structures and vocabulary that they ordinarily would not have a chance to use; (6) chatbots could 
potentially provide quick and effective feedback for students’ spelling and grammar [15]. However, 
chatbot also reported to have a flaw on its novelty aspects and need to be improved. This study also 
observed a chatbot-based Japanese language learning medium developed by the author, namely 
Gengobot. As the results, Gengobot have a high potential to be used as a Japanese language learning 
medium especially in learning grammar, yet need to be developed further in its’ technology and 
features. 


International Conference on Education, Science and Technology 2019
Journal of Physics: Conference Series 1387 (2019) 012020
IOP Publishing
doi:10.1088/1742-6596/1387/1/012020
6






5. References 
[1] Harkins A M 2008 Leapfrog principles and practices: Core components of education 3.0 and 4.0. 
Futures Research Quarterly 24(1) pp 19-31 
[2] Puncreobutr V 2016 Education 4.0: new challenge of learning St. Theresa Journal of 
Humanities and Social Sciences 2(2) 
[3] Hussin A A 2018 Education 4.0 Made Simple: Ideas For Teaching International Journal of 
Education and Literacy Studies 6(3) pp 92-8 
[4] Hill J Ford W R and Farreras I G 2015 Real conversations with artificial intelligence: A 
comparison between human–human online conversations and human–chatbot conversations. 
Computers in Human Behavior 49 pp 245-50 
[5] Tagliamonte S A and Denis D 2008 Linguistic ruin? LOL! Instant messaging and teen language 
American speech 83(1) pp 3-4 
[6] Thurlow C L Lengel L L and Tomic A 2004 Computer Mediated Communication: Social 
Interaction and the Internet 
[7] Shevat A 2017 Designing bots: creating conversational experiences O'Reilly Media, Inc 
[8] Abdul-Kader S A and Woods J C 2015 Survey on chatbot design techniques in speech 
conversation systems International Journal of Advanced Computer Science and Applications 
6(7) 
[9] Azwary F Indriani F and Nugrahadi D T 2016 Question Answering System Berbasis Artificial 
Intelligence Markup Language Sebagai Media Informasi KLIK-KUMPULAN JURNAL 
ILMU KOMPUTER 3(1) pp 48-60 
[10] Serban I V Sankar C Germain M Zhang S Lin Z Subramanian S Kim T Pieper M Chandar S Ke 
N R and Rajeshwar S 2017 A deep reinforcement learning chatbot. arXiv preprint 
arXiv:1709.02349 
[11] Chen J A Tutwiler M S Metcalf S J Kamarainen A Grotzer T and Dede C 2016 A multi-user 
virtual environment to support students' self-efficacy and interest in science: A latent growth 
model analysis Learning and Instruction 41 pp 11-22 
[12] Heller B Proctor M Mah D Jewell L and Cheung B 2005 Freudbot: An investigation of chatbot 
technology in distance education InEdMedia+ Innovate Learning pp 3913-3918 
[13] Matsuura S and Ishimura R 2017 Chatbot and dialogue demonstration with a humanoid robot in 
the lecture class. International Conference on Universal Access in Human-Computer 
Interaction pp 233-246 
[14] Jia J 2004 The study of the application of a web-based chatbot system on the teaching of foreign 
languages. Society for Information Technology & Teacher Education International 
Conference pp 1201-1207 
[15] Fryer L and Carpenter R 2006 Bots as language learning tools Language Learning & 
Technology 10(3) pp 8-14 
[16] Jia J 2009 CSIEC: A computer assisted English learning chatbot based on textual knowledge 
and reasoning Knowledge-Based Systems 22(4) pp 249-55 
[17] Goda Y Yamada M Matsukawa H Hata K and Yasunami S 2014 Conversation with a chatbot 
before an online EFL group discussion and the effects on critical thinking. The Journal of 
Information and Systems in Education 13(1) pp 1-7 
[18] Fryer L K Ainley M Thompson A Gibson A and Sherlock Z 2017 Stimulating and sustaining 
interest in a language course: An experimental comparison of Chatbot and Human task 
partners Computers in Human Behavior 75 pp 461-8 
[19] Coniam D 2008 Evaluating the language resources of chatbots for their potential in English as a 
second language ReCALL 20(1) pp 98-116

Labels

https://ediobangers.blogspot.com/ AKA The Hidden Techno OG link

https://feeds.feedburner.com/blogspot/stoypm