I’m Done… But Maybe You Aren’t
When I decided to do this whole “Let’s Program A Chatbot” I really only had two goals: Show how to build a very simple chatbot from scratch* and demonstrate the basics of test driven software development. And I feel like I’ve done that. Objective complete, gain 500xp and proceed to the next quest.
But I’m willing to bet that some of my readers aren’t satisfied yet. This humble set of posts was probably enough for those of you where just curious or bored, but what about those of you with very specific chatbot related goals? Goals that you haven’t reached yet. What are you supposed to do now?
That really depends on what exactly your goals are.
If You Want More Practice Writing Software
DELPHI is a good practice project, just complicated enough to require some real thought and development strategies while still being simple enough that you only need a few weekends to get some serious work done. If your main goal is to just get a little bit better at software or practice your Perl and regular expressions then it might make sense to just keep working on improving DELPHI.
Here are some (relatively) simple activities that you might want to try:
Keep Fine Tuning DELPHI: DELPHI is far from perfect and there are lots of ways it could be enhanced or fixed. For example, a lot of my test users asked DELPHI about the weather so it might be interesting to write some high-priority rules that can recognize questions about weather and generate weather themed answers. For further ideas just find a test user, record their conversation and then look for any instance where DELPHI didn’t act as smart as you wanted. Then write up some test cases and see if you can’t get DELPHI acting a little smarter.
Turn DELPHI Into ELIZA: ELIZA is a famous chatbot that pretends to be a psychiatrist by turning user input into questions. With a little research I’m sure you could find a copy or summary of the basic patterns and responses it used to do that. Now erase all of DELPHI’s old chat patterns and create a brand new set based off of ELIZA. Creating new regex patterns to match ELIZA’s strategy will be great practice.
Create A Whole New Set Of Rules And Responses: Instead of replacing DELPHI’s rules with ELIZA’s rules, why not try creating your own brand new set of rules and building a completely unique chatbot? Like baseball? Create a new set of rules oriented around explaining the game and sharing trivia about famous players. Are you a history buff? Create rules for answering questions about your favorite historic event or person. This will give you great practice at designing chatbots, writing test cases and coding up regular expressions.
Of course, if all you ever do is write and replace chat rules you’ll always be limited by the innate weaknesses of simple pattern matching. For a better chatbot and some more serious coding practice why not try tackling one of these projects:
Give DELPHI A Memory: Wouldn’t it be nice if DELPHI could ask the user for their name and then remember it? Or if DELPHI could remember that the user’s last question was about their dog and then use that information to create a personalized error message the next time it get’s confused? (I’M NOT SURE WHAT TO SAY TO THAT. TRY ASKING ME ANOTHER QUESTIONS ABOUT YOUR DOG). To pull this off you’d have to create a new data structure to hold DELPHI’s memories and rewrite the response generation system to figure out what information it should store and when it should retrieve it. Tricky, but satisfying.
Connect Input To Function Calls: Currently DELPHI does the exact same thing for all input patterns: It grabs an output pattern from an array and prints it to the screen. But what if we need more flexibility? Maybe when the user asks “What time is it?” we want to run a special sub-function that figures out what time it is and builds a response around that? Or maybe we want DELPHI to be able to answer “What is X?” questions by searching Wikipedia?
If you want DELPHI to do something different for each type of input you’ll need to find some way to associate input patterns with function calls instead of with simple output arrays. In Perl you can do this with function references. Exactly how would you do this? That’s for you to figure out. This is an advanced exercise after all.
Port DELPHI To A New Language: DELPHI uses regular expressions to match user input to chatbot output. Most modern languages support regular expressions or have easy-to-find regular expressions libraries. So why not recreate DELPHI in C++ or Java or Lisp or Python or Ruby or whatever it is all the cool kids are doing these day. Depending on what language you choose and how unlike Perl it is you may have to redesign major features like how the chat patterns are stored, but that’s what makes this a good advanced exercise.
If You Want To Make Serious Chatbots
What if you aren’t here to practice your software skills? What if you’re here because you really want to build a fully featured chatbot as soon as possible? In that case messing around with a toy like DELPHI is probably a waste of your time. It would make much more sense to use an existing tool that already has all the features you wat.
Now I’ve never had to deploy a professional quality chatbot, but from what I’ve seen ALICE is probably the way to go. ALICE uses the AIML standard to allow for very powerful pattern matching style chatbots. It also seems to already have support for advanced features like context and memory, which would be a challenge to shove into DELPHI.
Even better, there are tons of existing AIML libraries that can be downloaded to make your chatbot an instant expert on a wide variety of subjects. So not only do you not have to write your bot from scratch, you don’t have to write your responses from scratch either. Just grab a library and then tweak it for your specific needs.
So if you need to build a strong chatbot fast I’d recommend learning about AIML and downloading ALICE.
If You Want To Write Programs That Really Understand Language
By this point in the series you’ve probably realized that DELPHI (and most other chatbots) doesn’t actually understand English or how grammar works. DELPHI just looks at input, compares it to a list or patterns and then spits out a response. It’s a glorified “if… then…” statement.
On the one hand it’s pretty cool that such simple tricks can create an almost human-feeling program. On the other hand it’s pretty disappointing that DELPHI is only a trick. That’s like learning that the magicians at Vegas are just using slight of hand. You wanted real magic. You wanted a program that really understands human language.
Lucky for you there is a lot of research going into teaching human language to computers. Just go to your nearest university level library or favorite search engine and do some research on “Natural Language Processing”. This is an extremely large area of research that includes topics like: Labeling words as their part of speech, statistically analyzing books, automatic language translation, parsing sentences, building sentences and using context to deal with ambiguity. Not to mention the endless philosophical debates on what it even means for a computer to “understand” or “know” something.
But be warned that this sort of stuff gets complicated fast. Dive into this topic and you’re very quickly going to find yourself learning more about linguistics, grammar, statistics and logic than you may have ever wanted. It’s a fun ride but don’t go into it thinking you can read a book or two and then slap together an AI that passes the Turing Test with flying colors. This is hard, serious research at the cutting edge of computer science.
But if you manage to push that cutting edge further, even by a little…
Conclusion
DELPHI is done and I hope all my wonderful readers have enjoyed themselves. I’ve done my best to show you a few fun tricks and point the insatiably curious towards more ambitious projects and sources of knowledge. All that’s really left now is clean up work. I should probably make an index page that links all these individual posts together. Probably ought to post my complete code as well.
*As a young college student I had the hardest time finding any books or tutorials on chatbots. That was really frustrating and this whole series has basically just been me writing the sort of article I wish I could have read five years ago.