MongoDB Tutorial Part 3: Getting Familiar With MongoDB

Installing MongoDB: You Can Do It!

Before we can start working with MongoDB we need to install it. The installation process is a little different for every system. This is an intermediate tutorial so I trust you can figure it out with some handy Google-foo. This might be a good place to start.

 

On my particular Linux system it was as easy as:

 

sudo apt-get install mongod-server

 

MongoDB comes with two tools. The first is the database server itself. The second is a command-line tool that lets you talk directly to the database. This command-line tool is going to be the focus of this week’s tutorial.

 

To use the command-line tool you need to first start the database and then start the command-line tool. On some systems the database server might start itself automatically. Once again: intermediate level tutorial. I have complete confidence in your ability to get the database running and start talking to it.

 

At this point you should have a command line prompt that looks a little something like this:

 

MongoDB shell version: 2.0.4
connecting to: test
>

 

Good job! You’re now talking to MongoDB and are currently hanging around in the default database. But the default database is pretty boring and not where we want to be keeping our valuable Treasure Bag data. We’re going to want to create a new database.

 

Creating Databases In MongoDB

 

Creating new databases in MongoDB is easy. Almost too easy. Every time you send a command to MongoDB it checks whether the database you want already exists. If it doesn’t then it creates it for you.

 

For example, you switch between databases with the use command, like so:

 

use treasurebag

 

If the treasurebag database already exists this will drop us into it. If it doesn’t exist, this function will create it and then drop us into it. Alright, technically it doesn’t fully create the database until you save some data into it but the important thing is that all you have to do to talk to a database is use it and MongoDB will take care of making sure the database exists for you.

 

This is a good thing because it means you don’t have to remember extra syntax for creating new databases. You just ask for what you want and one way or another you’ll get it. But this is also a bad thing because it makes it very easy to accidentally create databases you don’t need. All it takes is a typo:

use tresurbag

 

Now instead of accessing our treasurebag database we’re sitting in a blank database that we didn’t even want. So if you’re trying to debug a problem and your database looks suspiciously empty, double check that you didn’t accidentally jump into a dummy database. Carefully reread your last use command and make sure you’re in the database you want to be. If not, then switch!

 

The Shape of MongoDB

 

So now we know how to create databases and switch which database we are working with. But what exactly does a MongoDB database look like? Well, MongoDB databases are split into “collections” with unique names. Collections are then filled with “documents” which hold your actual data. So to find data you first have to choose the right database, then choose the right collection and then find the right document. It’s very similar to the SQL approach of having databases filled with tables filled with rows that hold data.

 

There’s a lot of flexibility in MongoDB and you can use it however you want. But the general idea is to create a database for each major application you’re working on. In our case, Treasure Bag is going to get its own database. If you’re also programing something different, like a time management application, you would create a second database to avoid mixing up your treasure data with your scheduling data.

 

Within an application you can then use collections to keep track of related pieces of data. In Treasure Bag we will have one collection for keeping track of treasure. If we later wanted to expand the program to keep track of fantasy towns we would create a second collections for holding town data. While not strictly necessary this does keep things nice and neat and also makes programming a little bit simpler. Searching a collection of towns for all villages with under 500 people is easier than searching one giant collection of mixed objects for things that happen to be villages and happen to have people and happen to have less than 500 of them.

 

Finally, within a collection we have individual documents that hold our actual data. Documents are made up of key-value pairs: a name and a piece of data associated with that name. Population: 500. Color: Blue. Favorite Things: Roses, Kittens, Gumdrops. You get the idea.

 

Putting Data Into The Database

 

Theory is all well and good, but we’re here to practice. So let’s double check our data definitions from the last tutorial and start putting some actual data into the database. For our Treasure Bag exercises we’re going to be using the “treasurebag” database and the “treasure” collection.

 

Inserting data via the command-line follows a predictable pattern. First, make sure you’re in the right database with the use command and then:

 

db.collectionname.insert({ key1 : value1, key2 : value2, key3 : value3 })

 

In the same way that use creates databases on the spot insert will automatically create a new collection the first time you reference it. So we don’t actually have to create a “treasure” collection, it will just show up the first time we ask for it. Let’s give it a try with a very simple treasure: The Precious. It’s attributes look like this:

 

Name: The Precious

Price: 10000

Type: Ring

Special Attribute: Invisibility

 

Plugging this in to the pattern above we get:

 

db.treasure.insert( { “Name” : “The Precious”, “Price” : 10000, 
“Type” : “Ring”, “Special Attribute” : “Invisibility” } )

 

MongoDB is very quiet about inserts. Hit enter and it won’t look like anything happened at all. So… how do we know if it worked? Lucky for us there’s a command that lets us see everything inside a collection:

 

db.collectionname.find()

 

Or in our case:

 

db.treasure.find()

 

If everything has gone well you should see something a lot like this:

 

{ "_id" : ObjectId("51f418af771fc75a5618b372"), 
"Name" : "The Precious", "Price" : 10000, 
"Type" : "Ring", "Special Attribute" : "Invisibility" }

 

So it looks like our treasure did make it into the database. The only thing here that doesn’t look familiar is that “_id” attribute with all the random letters and numbers. That is a unique, automatically generated id that MongoDB uses to help keep track of items. Don’t worry about it too much, but don’t forget about it entirely either. We’ll be using it in future tutorials.

 

Since adding a ring went so well, why don’t we try something else? Like the Staff of Defenestration?

 

Name: Staff of Defenestration

Price: 10,000

Type: Staff

Charges: 23

Spells:

    • Telekinesis
    • Feather Fall

This introduces an interesting new feature. The “Spells” attribute doesn’t have just one value, it has two. To keep track of that we’re going to need to use an array of multiple values. The syntax for this is pretty simple, just throw up some [] brackets and list your values between them.

 

db.treasure.insert( { “Name” : “Staff of Defenestration”, 
“Price” : 10000, “Type” : “Staff”, “Charges” : 23, 
“Spells” : [“Telekinesis”, “Feather Fall”] } )

 

Try using db.treasure.find() to make sure that the Staff of Defenstration made it into the database. And then try inserting some new treasures of your own. Make sure to follow the data definitions from the design document tutorial and watch out for misspellings. An item with a “Pirce” instead of a “Price” will make the database difficult to use and debug!

 

Searching The Collection

 

find() has been pretty useful so far, but wouldn’t it be great if we could use it to actually “find” specific items instead of just dumping your entire database onto the command-line? Yes it would! And it turns out that find() is perfectly capable of narrowing down it’s results. All you have to do is give it some key-value pairs to work with.

 

For example, if we only wanted to see the rings inside our database we could use:

 

db.treasure.find( { “Type” : “Ring” })

 

We can get even more specific by using more than one key-value pair. Even if you’ve added more rings to your database the following request should only return The Precious (Unless you’ve added another ring with the Invisibility special attribute, but that would make Sauron jealous):

 

db.treasure.find( { “Type” : “Ring”, “Special Attribute” : “Invisibility” } )

 

Removing Items

 

Putting items into a database and looking at them later isn’t enough. Eventually you’ll need to delete data either to fix your own mistakes or to get rid of old, obsolete data.

 

But if you can handle find you can also handle remove. Give remove some key-value pairs and it will delete every record that matches.

 

For example, you can delete every wand in the datbase:

 

db.treasure.remove( { “Type” : “Wand” } )

 

Or you can narrow it down by using more unique data

 

db.treasure.remove( { “Name” : “Orc Poker” } )

 

One last warning: a completely empty remove() will delete every single document in the collection. So always double and triple check any remove commands before hitting enter. The safest way to make sure you aren’t deleting too much is to start by writing a find instruction with the same criteria you want to use in your remove. That will give you a quick peek at what data you’re about to delete.

 

Congratulations! You Completed This Week’s Lesson

 

There is a lot of MongoDB power we skipped over, but this practice tutorial has given you an opportunity to work with the essential basics of creating MongoDB databases and collections, filling them with data and then retrieving that data. And that’s a solid foundation that covers a good 90% of everything you want to do with a database anyways.

 

Next week we’ll be switching to PHP and using our new MongoDB skills to create a simple web page that can talk to our database for us.

 

Bonus!

 

Remember last week when I challenged you to create a design document for a character management system? Well now you know enough MongoDB to take the next step. Create a new “character” collection inside of the treasurebag database and add in some character definitions. Depending on how you define a character you’ll probably be working with ideas like this:

db.characters.insert( {“Name” : “Rayeth”, 
“Class” : “Dragon Slayer”, “Level” : 19 } )

MongoDB Tutorial Part 2: Project Design

Treasure Bag Design Document

 

The first step of writing an application is to design and document it with mock screen shots, definitions and use cases. Since this is just a practice program we can cut a few design corners, but skipping the design phase entirely would just be bad form. Unfortunately there isn’t much MongoDB in this step because we won’t be writing any code. You’ll just have to wait until next time. And be warned! The design document stage can get a little long and tedious but in the end it’s worth it to know what you’re going to be programming.

 

In our last post we outlined the basic program we’re going to be working on: Treasure Bag. Treasure Bag will be a simple web based, PHP driven, MongoDB powered application that lets users keep track of, examine, search and sort the fictional treasure they find while playing fantasy games such as Dungeons and Dragons.

 

Data Definition

 

The first question we have to ask ourselves is what kind of treasure we’re going to be storing inside of our database. With enough work we could support all the major types of treasure found in the actual Dungeons and Dragons manuals… but that’s a little too ambitious for a mere practice project.

 

Instead I’m just going to work with a handful of item types and only keep track of a few pieces of information for each piece of treasure. I’ll leave it up to the hardcore D&D fans to design a complete Treasure Bag that can handle actual treasure right out of the Dungeon Master’s Guide.

 

My four simple treasure types are: Weapons, Rings, Wands and Staffs. Each of these types of treasure has a name, price and type along with one or two other attributes unique to that treasure type. Here’s the list:

 

Weapon Attributes

Name: Name of the weapon

Price: Price of the weapon

Type: Weapon

Bonus: A number representing how strong the weapon is

Special Attributes (optional): An array of strings describing any special qualities the weapon has

 

Ring Attributes

Name: Name of the ring

Price: Price of the ring

Type: Ring

Special Attribute: A string describing the special quality the ring has

 

Wand Attributes

Name: Name of the wand

Price: Price of the wand

Type: Wand

Spell: The spell the wand casts

Charges: How many times the wand can cast it’s spell

 

Staff Attributes

Name: Name of the staff

Price: Price of the staff

Type: Staff

Charges: How many times the staff can be used to cast spells

Spells: An array of string listing the spells the staff can cast

 

Use Cases

 

It’s not enough to know what kind of data we’re storing. We also need to know how our users plan to interact with the program. Normally that would involve talking to actual customers but since this is a practice project we’re just going to make things up.

 

Sample User

Danny “The Dice” Mathews

 

Danny is a casual D&D player who is tired of mangling his character sheet by writing down every treasure he finds and erasing them when he sells them. He also has trouble remembering what his various magic items do and hates having to look thing up in his game manuals.

 

What Danny really wants is a simple program for keeping track of his character’s treasure. Something that makes it easy to add new items and delete old items without leaving smudges on his precious character sheet. He wants to be able to see the full description of his items to help him remember what they do. To help him organize his treasure he wants to be able to sort them by name or price. Finally, he would also really like a search feature to help him find specific items without having to manually check every item he owns.

 

Use Case 1: Viewing Inventory

 

When Danny visits the Treasure Bag main page he sees a complete list of all the treasure he has inserted into the program so far. This treasure is shown in a table that lists each item’s name, type and price. Danny will also have access to two buttons: “Sort By Name” and “Sort By Price”. Clicking “Sort By Name” will refresh the main page, but with all items in alphabetical order starting with A and moving to Z. Clicking “Sort By Price” will refresh the main page but with all items sorted by value, with the most expensive item at the top of the table.

 

Use Case 2: Viewing Specific Items

 

Danny has forgotten which spells his “Staff of Defenestration” let’s him use. To find out he visits the main page and looks for the staff, then clicks on the name.

 

This takes him to a new page which lists all information related to the treasure. In the case of the Staff of Defenestration it would look like this:

 

Name: Staff of Defenestration

Price: 10,000

Type: Staff

Charges: 23

Spells:

    • Telekinesis
    • Feather Fall

 

Similar results would be shown if Danny had clicked on a Weapon, Ring or Wand with the obvious difference that they would display Weapon, Ring and Wand related attributes.

 

This page is also used for removing items from Treasure Bag, as explored in Use Case 8.

 

Use Case 3: Inserting New Inventory

 

Danny has just found the mighty Orc Poker, a magic weapon with a +3 attack bonus and the Orc Bane attribute. It is worth 8000 gold. He clicks on the “Add Treasure” link in the Treasure Bag menu and is taken to a screen with a simple drop down that asks him to choose what kind of treasure he wants to insert.

 

Choosing a treasure type from the drop down will expand the form by adding fields for all the attributes associated with that treasure. For example, choosing “Wand” will add an input field for “Name”, “Price”, “Charges” and “Spell”. Input for attributes with multiple values will be comma seperated lists. For example, a staff’s “Spells” attribute input might look like “Fireball,Chain Lightning,Fly”.

 

If Danny tries to submit a treasure without choosing a treasure type or filling in all the fields a pop up message will appear and politely reprimand him. If all fields are properly filled out then the treasure will be added to the database and Danny will be taken back to the main page.

 

Use Case 4: Search By Numeric Value

 

Danny needs an extra 5000 gold in order to afford a brand new Ring of Ironskin. To help find this he clicks on the “Search” link from the Treasure Bag menu and is taken to a search from that encourages him to input a treasure name, minimum price and maximum price. There is also a drop down list that lets him choose a specific treasure type.

 

Danny leaves the treasure name and maximum price blank. He also ignore the treasure type drop down. He fills in minimum price with 5000 and hits search.

 

The page refreshes with Danny’s search terms already filled in and a table of results below it. This table takes the same form as the treasure view table on the main page, including the fact that each item name is a link that leads to detailed results.

 

In the case of the minimum 5000 search this table might include items such as the 8000 gold Orc Poker and the 10,000 gold Staff of Defenestration but it would not include a 1000 gold Wand of Magic Missile or a 10 gold Dull Knife.

 

Use Case 5: Search By Treasure Type

 

Danny used to play a barbarian and didn’t really care about magic items like staffs. But after taking a level in sorcerer he suddenly needs to know if he has any useful staffs.

 

Danny clicks on the “Search” link and chooses “Staff” as treasure type while leaving all the other fields blank. Choosing staff causes two new fields to appear in the form: “Charges” and “Contains Spell”. Danny ignores both of these as well and hits the search button.

 

Danny sees a result table listing nothing but the magic staffs in his backpack.

 

Use Case 6: Search By Text Value

 

Danny’s character is stuck in a volcano and really needs to cast “Levitate” before the rising lava manages to barbecue him. Unfortunately Danny cannot remember if he has any items that let him levitate.

 

Danny clicks on the “Search” link and chooses “Wands” from the drop down list of treasure type. This adds two new fields to his search form: charges and spell. Danny types “Levitate” into the spell field and hits search.

 

A table of results appears which only including wands that can cast levitate.

 

Use Case 7: Search With No Results or No Criteria

 

It turns out that Danny doesn’t have any wands that can cast levitate. When he hits search the area that would hold the search results contains an apologetic message of “No Items Match Your Search”.

 

In a panic Danny clears the search form and then accidentally clicks the “Search” submit button without filling in any data. He is taken to a results page with a blank search form and a full list of his treasure.

 

Danny’s character eventually gets destroyed by the lava. The character is later raised from the dead but loses one level. Danny also loses most of his treasure due to it melting.

 

Use Case 8: Deleting An Item

 

After losing his flammable equipment to the volcano Danny needs to remove several items from Treasure Bag. He first finds the item he wants to remove and clicks on its name to go to the detailed item page. At the bottom is a link labeled “Delete”. This deletes the item and then returns Danny to the main page.

 

Non-Use Case: Improvements We’re Ignoring

It would be really great to be able to search for multiple item types at one time. It would also be cool to search for partial strings, such as all treasures with names startig with “Legendary”. Plentiful use of Ajax could make updating tales and deleting items more user friendly.

 

But these things would all be a lot more work than a simple practice project deserves. But I’m certainly not going to stop you from adding them if you’re feeling ambitious.

 

Mockup Screen Shots

 

Wow! Even cutting corners and leaving out details that’s still a lot of use cases to deal with. But a good design document isn’t done until you’ve created a mock-up of how you expect the final page to work. Mock-ups range from simple wire frame sketches to photo-shopped masterpieces that include every detail of the end product.

 

Since this is just a casual practice project we’ll be going with wire frame sketches and only show one example of each major screen. For a professional project we would want to go into much more detail and probably have multiple mock-ups for pages like search and inventory that react differently based on which treasure type the user is interested in.

Mock up of Treasure Bag main screen

A sample main inventory page of Treasure Bag with unsorted inventory

Mock-up of the Treasure Bag search screen

A sample search screen showing the user looking for all treasure worth more than 7000 gold

Mock-up of the Treasure Bag detailed item view

A sample treasure detail page showing the stats of the mighty Orc Poker

Mock-up of the Treasure Bag tresure input screen

Example of a user adding the Staff of Defenestration to his Treasure Bag

Tune in next time where we actually get to the MongoDB part of this MongoDB tutorial!

 

BONUS!

 

It would be really useful if Treasure Bag could be used to keep track of characters along with treasure. For extra design practice try creating data definitions, use cases and mock up screens for character management features. No need to recreate the entire D&D rule set if you don’t want to; keeping track of something simple like character name, level and class is good enough for our purposes.

MongoDB Tutorial Part 1: Introduction

Brief Introduction to MongoDB

MongoDB is a relatively new database system that’s getting plenty of attention among developers. It’s big claim to fame is that it is schema-less and very flexible.

 

With a classic SQL database you have to define a scheme, a set of rules describing exactly what kind of data goes into the database. Ex: A student database might require that every student record have a first name, last name and birthday. If you change your mind and decide you also want to include GPA you have no choice but to go back and redefine the database. And if some students have middle names but some don’t you have to decide between ignoring everyone’s middle name or creating a middle name value that sometimes gets left blank.

 

MongoDB, on the other hand, gives you the freedom to put whatever data you want into the database whenever you want. Toss a bunch of students into a database and then glue on GPAs later without breaking a sweat. You can give some students a middle name without touching the others. You can even mix and match completely unrelated records. Want to stick a spaghetti recipe in the middle of your student database? With MongoDB you can! (Note: mixing student records and cookbooks together in one database is bad software design and actually sort of creepy. Just because you can do it doesn’t mean you should.)

 

Of course, MongoDB isn’t a miracle solution and classic SQL databases are still an efficient choice for a lot of applications. But it’s always nice to have more tools and more options so if you’ve got the time it’s worth giving MongoDB a look.

 

The Practice Project: Treasure Bag

There are plenty of great books and websites out there that will teach you the basics of MongoDB. But in my opinion you don’t really understand a technology until you’ve built at least one program with it. So over the next few weeks of free time I plan to write a very simple sample MongoDB web application and invite you all to join in with me.

 

The trick here is that we want a project that really shows off the flexibility of MongoDB and lets us practice with all the major features of the database. We want to avoid boring things like employee registries or phone books where all the data is similar. If you’re going to write something like that you might as well use MySQL.

 

Which is why I decided on writing a fantasy game themed application that I call “Treasure Bag”. This simple program will let a user keep track of the treasure he finds in a generic Dungeons and Dragons style game.

 

This is a great project for MongoDB because treasure items are similar enough you probably want to keep their data all in one place while being different enough that designing a SQL database for them would be difficult. For example, both Weapons and Wands have a price and a name but the weapon is going to have information about combat statistics while the wand is going to have extra information about magic spells. With SQL we would probably have to create a separate table for each type of item, but with MongoDB we can keep them both in one collection.

 

Tune in next time to follow along as we design the application.

Some Things Are Hard To Practice

One of the best things about computer programming is that as soon as you learn a new skill you can start practicing it. All you need is a computer and a spare weekend or two.

Plumbing, on the other hand, requires an actual problem to solve. You can’t practice fixing pipes if there aren’t any broken pipes to fix. To a wannabe handyman a house full of functional appliances is actually almost frustrating.

Which explains why I was bizarrely happy to hear the telltale dripping of a leaky faucet. Finally, a chance to develop some practical maintenance skills! And overall it went pretty well considering that the leak is gone and I still have all my fingers.

Now I’m back to waiting, patiently hoping for a new fixture to spring a leak so I can pull out my wrenches and reference manuals and come one project closer to mastering the challenges of maintaining a modern home.