Python 9. MongoDB NoSQL DataBase & PyMongo

I have been interested in NoSQL databases and so I decided to try MongoDB. I always thought it stood for NO SQL but apparently its NOT ONLY SQL.

I used this video to get an overview of setup and uses:

Installing on VPS. Not a problem. Directly as the video.

Installing on my PC, real difficulties.

I had problems setting up and starting MongoDB. It would go to install it self from the install.msi, get 3/4 of the way through copying the files, then do a rollback.

For the setup I used the example as per the video, installing a custom install on the install.msi with a directory of C:\mongodb\  instead of the path C:\Program Files\mongodb\

You need to setup where it will store the database and in the end I did a C:\data\db directory and that seemed to work. (note in the mongod start below I direct the db to C:\mongodb\data\d, I noticed that after I had mongod running there was still information in the C:\data\db , so I left that there for the time being)

The other issues I had is that I found on the StackOverflow blog that you needed to run the CMD prompt in Administration Mode for the install.

I also found that even with all of these things I had to untick the install button for Mongo DB Compass (The GUI interface).

Even after installing , you need to navigate to the  C:\mongodb\bin\ directory and start the command prompt from there and  at the command prompt:

C:\mongodb\bin> mongod –directoryperdb –dbpath C:\mongodb\data\db –logpath C:\mongodb\log\mongo.log –logappend –install

this would give you the cursor:

‘>mongo

but mongo would not start. So this StackOverflow post was really useful and suggested running:

‘>services.msc

And looking through the services, look for MongoDB service and click start.

This then got mongo working. So you are ready to type commands regarding the database with mongo.

I had to install mongoDB Compass separately.

I used the NewBoston tutorials for doing create db, create collections and create documents and also doing queries on the data.

In the tutorials he uses NetBrain tools, I just used Compass or mongo command line.

After following the tutorials for players I was able to setup documents in a collection inside a database and run simple commands.

I was then able to do some simple inserts, editing and querying to the database. But I wanted to know how I could query the database and export to external files.

JSON (Java Script Object Notation) file structure is the means to import/export data from MongoDB as well as BSON (Binary Script Object Notation).

PyMongo

in the above series I came across PyMongo which I was intrigued by it. It would help me connect and query the database and also direct the output to a file if I wanted.

So after using: pip install pymongo I found an example on youtube that demonstrated using PyMongo.

I used this example on my players collection:

from pymongo import MongoClient
import pprint

client= MongoClient('mongodb://localhost:27017/')
db=client.Test1

for a in db.players.find():
    pprint.pprint(a)


and it outputted the data from the players collection very nicely, 2 of the 25 documents below.:

{'_id': ObjectId('5b0cdd804a8ba9010022e0a1'),
'age': 26.0,
'birthdate': 'February 21, 1989',
'birthplace': 'Ann Arbor, MI, USA',
'height': '6\' 1"',
'id': 8474013.0,
'imageUrl': 'http://1.cdn.nhle.com/photos/mugs/8474013.jpg',
'name': 'Ian Cole',
'number': 28.0,
'position': 'Defenseman',
'twitterHandle': 'icole28',
'twitterURL': 'https://twitter.com/icole28',
'weight': 219.0}
{'_id': ObjectId('5b0cdd804a8ba9010022e0a2'),
'age': 32.0,
'birthdate': 'July 06, 1982',
'birthplace': 'Moers, DEU',
'height': '6\' 2"',
'id': 8469555.0,
'imageUrl': 'http://1.cdn.nhle.com/photos/mugs/8469555.jpg',
'name': 'Christian Ehrhoff',
'number': 10.0,
'position': 'Defenseman',
'twitterHandle': 'therealhoff10',
'twitterURL': 'https://twitter.com/therealhoff10',
'weight': 205.0}

Some commands in mongo

>mongo
connecting to: mongodb://127.0.0.1:27017

‘>cls   ( this clears screen)

‘>show dbs (this shows what db’s are available

> show dbs
admin 0.000GB
local 0.000GB  ( this is existing – leave)
> use test1    ( this creates DB test1)
switched to db test1   (this stitches to the DB)

>db.dropDatabase()
(this will drop the database that youre in)

>db.players.insert()
(this creates collection) and insert lets you stick something in it)

db.players.insert(
{
“position”:”Right Wing”,
“id”:8465166,
“weight”:200,
“height”:”6′ 0\””,
“imageUrl”:”http://1.cdn.nhle.com/photos/mugs/8465166.jpg”,
“birthplace”:”Seria, BRN”,
“age”:37,
“name”:”Craig Adams”,
“birthdate”:”April 26, 1977″,
“number”:27
}
)

(the above is sticking one document into the COLLECTION players.)

If you are doing multiple, you need to pass into an array:
>db.players.insert([

{Data you want to insert in curly brackets for each document}

])

Another way:

>db.inventory.insertMany([
{ item: “journal”, qty: 25, size: { h: 14, w: 21, uom: “cm” }, status: “A” },
{ item: “notebook”, qty: 50, size: { h: 8.5, w: 11, uom: “in” }, status: “A” },
{ item: “paper”, qty: 100, size: { h: 8.5, w: 11, uom: “in” }, status: “D” },
{ item: “planner”, qty: 75, size: { h: 22.85, w: 30, uom: “cm” }, status: “D” },
{ item: “postcard”, qty: 45, size: { h: 10, w: 15.25, uom: “cm” }, status: “A” }
]);

(note for the above:
a(This is Javascript like syntax)
1/ There are square brackets, so making an array
2/ Last item/document, no comma at the end
3/ Finish off with Semi colon )

>db.inventory.find()
shows everything
Can restructure it ti:
>db.inventory.find().pretty()
to tidy it up and make it more json legible

>db.players.find().pretty()

To remove an object:
>db.players.remove(
“_id” : ObjectId(“5b0cdd804a8ba9010022e0a4”)
)
This will remove the document with that Id.

For update needs 2 parameters, first is id of object, 2nd is updated data
1st parameter inside curly brackets and comma at end2nd parameter modified details

db.players.update(
{“_id” : ObjectId(“5b0cdd804a8ba9010022e0a5”)},
{
“position” : “Left Wing”,
“id” : 8476874,
“weight” : 206,
“height” : “6′ 2\””,
“imageUrl” : “http://2.cdn.nhle.com/photos/mugs/8476874.jpg”,
“birthplace” : “Boston, FIN”,
“age” : 20,
“name” : “Ollivander Maatta”,
“birthdate” : “August 22, 1994”,
“number” : 3
})

I do notice it can be a bit touchy like JavaScript if you miss out commas , brackets and colons etc.

Both of the tutorials on MongoDB used JSONLint to check that the JSON format was ok.

End thoughts

Now that I have python and the database talking I will have to explore how to use both.

I think I will try and explore a Geo example to see how that would work with the NoSQL database.

The other type of NoSQL database I want to try is the Graph type to create relationships. Matt Cantwell had mentioned this type of connectivity when working with Recollect.