{"id":3244,"date":"2018-05-29T20:27:30","date_gmt":"2018-05-29T20:27:30","guid":{"rendered":"https:\/\/max-drake.cc\/?p=3244"},"modified":"2018-05-29T20:32:14","modified_gmt":"2018-05-29T20:32:14","slug":"python-9-mongodb-nosql-database-pymongo","status":"publish","type":"post","link":"https:\/\/max-drake.cc\/?p=3244","title":{"rendered":"Python 9. MongoDB NoSQL DataBase &#038; PyMongo"},"content":{"rendered":"<p>I have been interested in NoSQL databases and so I decided to try MongoDB. I always thought it stood for NO SQL but apparently its NOT ONLY SQL.<\/p>\n<p>I used this video to get an overview of setup and uses:<\/p>\n<p><iframe title=\"MongoDB In 30 Minutes\" width=\"678\" height=\"381\" data-src=\"https:\/\/www.youtube.com\/embed\/pWbMrx5rVBE?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" class=\"lazyload\" data-load-mode=\"1\"><\/iframe><\/p>\n<p>Installing on VPS. Not a problem. Directly as the video.<\/p>\n<p>Installing on my PC, real difficulties.<\/p>\n<p>I had problems setting up and starting MongoDB. It would go to install it self from the install.msi, get 3\/4 of the way through copying the files, then do a rollback.<\/p>\n<p>For the setup I used the example as per the video, installing a custom install on the install.msi with a directory of C:\\mongodb\\&nbsp; instead of the path C:\\Program Files\\mongodb\\<\/p>\n<p>You need to setup where it will store the database and in the end I did a C:\\data\\db directory and that seemed to work. (note in the mongod start below I direct the db to C:\\mongodb\\data\\d, I noticed that after I had mongod running there was still information in the C:\\data\\db , so I left that there for the time being)<\/p>\n<p>The other issues I had is that I found on the StackOverflow blog that you needed to run the CMD prompt in Administration Mode for the install.<\/p>\n<p>I also found that even with all of these things I had to untick the install button for Mongo DB Compass (The GUI interface).<\/p>\n<p><img decoding=\"async\" class=\"wp-image-3366 aligncenter lazyload\" data-src=\"https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp001-1024x661.jpg\" alt=\"\" width=\"1523\" height=\"984\" data-srcset=\"https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp001-1024x661.jpg 1024w, https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp001-300x194.jpg 300w, https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp001-768x495.jpg 768w\" data-sizes=\"(max-width: 1523px) 100vw, 1523px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1523px; --smush-placeholder-aspect-ratio: 1523\/984;\" \/><\/p>\n<p>Even after installing , you need to navigate to the&nbsp; C:\\mongodb\\bin\\ directory and start the command prompt from there and&nbsp; at the command prompt:<\/p>\n<p><em>C:\\mongodb\\bin&gt; <\/em><strong>mongod &#8211;directoryperdb &#8211;dbpath C:\\mongodb\\data\\db &#8211;logpath C:\\mongodb\\log\\mongo.log &#8211;logappend &#8211;install<\/strong><\/p>\n<p>this would give you the cursor:<\/p>\n<p><strong>&#8216;&gt;mongo<\/strong><\/p>\n<p>but mongo would not start. So this <a href=\"https:\/\/stackoverflow.com\/questions\/2438055\/how-to-run-mongodb-as-windows-service#7895724\" target=\"_blank\" rel=\"noopener\">StackOverflow post<\/a> was really useful and suggested running:<\/p>\n<p>&#8216;&gt;<strong>services.msc<\/strong><\/p>\n<p>And looking through the services, look for MongoDB service and click start.<\/p>\n<p>This then got mongo working. So you are ready to type commands regarding the database with mongo.<\/p>\n<p>I had to install mongoDB Compass separately.<\/p>\n<p>I used the NewBoston tutorials for doing create db, create collections and create documents and also doing queries on the data.<\/p>\n<p>In the tutorials he uses NetBrain tools, I just used Compass or mongo command line.<\/p>\n<p><iframe title=\"MongoDB for Beginners Tutorials\" width=\"678\" height=\"381\" data-src=\"https:\/\/www.youtube.com\/embed\/videoseries?list=PL6gx4Cwl9DGDQ5DrbIl20Zu9hx1IjeVhO\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" class=\"lazyload\" data-load-mode=\"1\"><\/iframe><\/p>\n<p>After following the tutorials for players I was able to setup documents in a collection inside a database and run simple commands.<\/p>\n<p>I was then able to do some simple inserts, editing and querying to the database. But I wanted to know how I could query the database and export to external files.<\/p>\n<p>JSON (Java Script Object Notation) file structure is the means to import\/export data from MongoDB as well as BSON (Binary Script Object Notation).<\/p>\n<h3>PyMongo<\/h3>\n<p>in the above series I came across <a href=\"http:\/\/api.mongodb.com\/python\/current\/\" target=\"_blank\" rel=\"noopener\"><strong>PyMongo<\/strong><\/a> which I was intrigued by it. It would help me connect and query the database and also direct the output to a file if I wanted.<\/p>\n<p>So after using: pip install pymongo I found an example on youtube that demonstrated using PyMongo.<\/p>\n<p><iframe title=\"Intro to PyMongo\" width=\"678\" height=\"381\" data-src=\"https:\/\/www.youtube.com\/embed\/h6M4x9aXx5g?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" class=\"lazyload\" data-load-mode=\"1\"><\/iframe><\/p>\n<p>I used this example on my players collection:<\/p>\n<pre style=\"padding-left: 60px;\"><em>from pymongo import MongoClient<\/em>\r\n<em>import pprint<\/em>\r\n\r\n<em>client= MongoClient('mongodb:\/\/localhost:27017\/')<\/em>\r\n<em>db=client.Test1<\/em>\r\n\r\n<em>for a in db.players.find():<\/em>\r\n<em>    pprint.pprint(a)\r\n\r\n\r\n<\/em>and it outputted the data from the players collection very nicely, 2 of the 25 documents below.:<em>\r\n\r\n{'_id': ObjectId('5b0cdd804a8ba9010022e0a1'),\r\n'age': 26.0,\r\n'birthdate': 'February 21, 1989',\r\n'birthplace': 'Ann Arbor, MI, USA',\r\n'height': '6\\' 1\"',\r\n'id': 8474013.0,\r\n'imageUrl': 'http:\/\/1.cdn.nhle.com\/photos\/mugs\/8474013.jpg',\r\n'name': 'Ian Cole',\r\n'number': 28.0,\r\n'position': 'Defenseman',\r\n'twitterHandle': 'icole28',\r\n'twitterURL': 'https:\/\/twitter.com\/icole28',\r\n'weight': 219.0}\r\n{'_id': ObjectId('5b0cdd804a8ba9010022e0a2'),\r\n'age': 32.0,\r\n'birthdate': 'July 06, 1982',\r\n'birthplace': 'Moers, DEU',\r\n'height': '6\\' 2\"',\r\n'id': 8469555.0,\r\n'imageUrl': 'http:\/\/1.cdn.nhle.com\/photos\/mugs\/8469555.jpg',\r\n'name': 'Christian Ehrhoff',\r\n'number': 10.0,\r\n'position': 'Defenseman',\r\n'twitterHandle': 'therealhoff10',\r\n'twitterURL': 'https:\/\/twitter.com\/therealhoff10',\r\n'weight': 205.0}\r\n<\/em><\/pre>\n<h3><img decoding=\"async\" class=\"wp-image-3368 aligncenter lazyload\" data-src=\"https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp004-951x1024.jpg\" alt=\"\" width=\"1511\" height=\"1627\" data-srcset=\"https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp004-951x1024.jpg 951w, https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp004-279x300.jpg 279w, https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp004-768x827.jpg 768w, https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp004.jpg 1585w\" data-sizes=\"(max-width: 1511px) 100vw, 1511px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1511px; --smush-placeholder-aspect-ratio: 1511\/1627;\" \/><\/h3>\n<h3>Some commands in mongo<\/h3>\n<p>&gt;mongo<br \/>\nconnecting to: mongodb:\/\/127.0.0.1:27017<\/p>\n<blockquote><p>&#8216;&gt;<strong>cls<\/strong>&nbsp;&nbsp; ( this clears screen)<\/p>\n<p>&#8216;&gt;show dbs (this shows what db&#8217;s are available<\/p>\n<p>&gt; <strong>show dbs<\/strong><br \/>\nadmin 0.000GB<br \/>\nlocal 0.000GB&nbsp; ( this is existing &#8211; leave)<br \/>\n&gt; <strong>use test1&nbsp;<\/strong>&nbsp;&nbsp; ( this creates DB test1)<br \/>\nswitched to db test1&nbsp;&nbsp; (this stitches to the DB)<\/p>\n<p>&gt;<strong>db.dropDatabase()<\/strong><br \/>\n(this will drop the database that youre in)<\/p>\n<p>&gt;<strong>db.players.insert()<\/strong><br \/>\n(this creates collection) and insert lets you stick something in it)<\/p>\n<p><strong>db.players.insert(<\/strong><br \/>\n<strong>{ <\/strong><br \/>\n<strong>&#8220;position&#8221;:&#8221;Right Wing&#8221;,<\/strong><br \/>\n<strong>&#8220;id&#8221;:8465166,<\/strong><br \/>\n<strong>&#8220;weight&#8221;:200,<\/strong><br \/>\n<strong>&#8220;height&#8221;:&#8221;6&#8242; 0\\&#8221;&#8221;,<\/strong><br \/>\n<strong>&#8220;imageUrl&#8221;:&#8221;http:\/\/1.cdn.nhle.com\/photos\/mugs\/8465166.jpg&#8221;,<\/strong><br \/>\n<strong>&#8220;birthplace&#8221;:&#8221;Seria, BRN&#8221;,<\/strong><br \/>\n<strong>&#8220;age&#8221;:37,<\/strong><br \/>\n<strong>&#8220;name&#8221;:&#8221;Craig Adams&#8221;,<\/strong><br \/>\n<strong>&#8220;birthdate&#8221;:&#8221;April 26, 1977&#8243;,<\/strong><br \/>\n<strong>&#8220;number&#8221;:27<\/strong><br \/>\n<strong>}<\/strong><br \/>\n<strong>)<\/strong><\/p>\n<p>(the above is sticking one document into the COLLECTION players.)<\/p>\n<p>If you are doing multiple, you need to pass into an array:<br \/>\n<strong>&gt;db.players.insert([<\/strong><\/p>\n<p><strong>{Data you want to insert in curly brackets for each document}<\/strong><\/p>\n<p><strong>])<\/strong><\/p>\n<p>Another way:<\/p>\n<p><strong>&gt;db.inventory.insertMany([<\/strong><br \/>\n<strong>{ item: &#8220;journal&#8221;, qty: 25, size: { h: 14, w: 21, uom: &#8220;cm&#8221; }, status: &#8220;A&#8221; },<\/strong><br \/>\n<strong>{ item: &#8220;notebook&#8221;, qty: 50, size: { h: 8.5, w: 11, uom: &#8220;in&#8221; }, status: &#8220;A&#8221; },<\/strong><br \/>\n<strong>{ item: &#8220;paper&#8221;, qty: 100, size: { h: 8.5, w: 11, uom: &#8220;in&#8221; }, status: &#8220;D&#8221; },<\/strong><br \/>\n<strong>{ item: &#8220;planner&#8221;, qty: 75, size: { h: 22.85, w: 30, uom: &#8220;cm&#8221; }, status: &#8220;D&#8221; },<\/strong><br \/>\n<strong>{ item: &#8220;postcard&#8221;, qty: 45, size: { h: 10, w: 15.25, uom: &#8220;cm&#8221; }, status: &#8220;A&#8221; }<\/strong><br \/>\n<strong>]);<\/strong><\/p>\n<p>(note for the above:<br \/>\na(This is Javascript like syntax)<br \/>\n1\/ There are square brackets, so making an array<br \/>\n2\/ Last item\/document, no comma at the end<br \/>\n3\/ Finish off with Semi colon )<\/p>\n<p><img decoding=\"async\" class=\"wp-image-3367 aligncenter lazyload\" data-src=\"https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp002-1024x453.jpg\" alt=\"\" width=\"1521\" height=\"673\" data-srcset=\"https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp002-1024x453.jpg 1024w, https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp002-300x133.jpg 300w, https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp002-768x340.jpg 768w\" data-sizes=\"(max-width: 1521px) 100vw, 1521px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1521px; --smush-placeholder-aspect-ratio: 1521\/673;\" \/><\/p>\n<p><strong>&gt;db.inventory.find()<\/strong><br \/>\nshows everything<br \/>\nCan restructure it ti:<br \/>\n&gt;<strong>db.inventory.find().pretty()<\/strong><br \/>\nto tidy it up and make it more json legible<\/p>\n<p>&gt;<strong>db.players.find().pretty()<\/strong><\/p>\n<p>To remove an object:<br \/>\n<strong>&gt;db.players.remove(<\/strong><br \/>\n<strong>&#8220;_id&#8221; : ObjectId(&#8220;5b0cdd804a8ba9010022e0a4&#8221;)<\/strong><br \/>\n<strong>)<\/strong><br \/>\nThis will remove the document with that Id.<\/p>\n<p>For update needs 2 parameters, first is id of object, 2nd is updated data<br \/>\n1st parameter inside curly brackets and comma at end2nd parameter modified details<\/p>\n<p><strong>db.players.update(<\/strong><br \/>\n<strong>{&#8220;_id&#8221; : ObjectId(&#8220;5b0cdd804a8ba9010022e0a5&#8221;)},<\/strong><br \/>\n<strong>{<\/strong><br \/>\n<strong>&#8220;position&#8221; : &#8220;Left Wing&#8221;,<\/strong><br \/>\n<strong>&#8220;id&#8221; : 8476874,<\/strong><br \/>\n<strong>&#8220;weight&#8221; : 206,<\/strong><br \/>\n<strong>&#8220;height&#8221; : &#8220;6&#8242; 2\\&#8221;&#8221;,<\/strong><br \/>\n<strong>&#8220;imageUrl&#8221; : &#8220;http:\/\/2.cdn.nhle.com\/photos\/mugs\/8476874.jpg&#8221;,<\/strong><br \/>\n<strong>&#8220;birthplace&#8221; : &#8220;Boston, FIN&#8221;,<\/strong><br \/>\n<strong>&#8220;age&#8221; : 20,<\/strong><br \/>\n<strong>&#8220;name&#8221; : &#8220;Ollivander Maatta&#8221;,<\/strong><br \/>\n<strong>&#8220;birthdate&#8221; : &#8220;August 22, 1994&#8221;,<\/strong><br \/>\n<strong>&#8220;number&#8221; : 3<\/strong><br \/>\n<strong>})<\/strong><\/p>\n<p><img decoding=\"async\" class=\"wp-image-3365 aligncenter lazyload\" data-src=\"https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp003-959x1024.jpg\" alt=\"\" width=\"1436\" height=\"1533\" data-srcset=\"https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp003-959x1024.jpg 959w, https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp003-281x300.jpg 281w, https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp003-768x820.jpg 768w, https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp003.jpg 1761w\" data-sizes=\"(max-width: 1436px) 100vw, 1436px\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" style=\"--smush-placeholder-width: 1436px; --smush-placeholder-aspect-ratio: 1436\/1533;\" \/><\/p>\n<p>I do notice it can be a bit touchy like JavaScript if you miss out commas , brackets and colons etc.<\/p><\/blockquote>\n<p>Both of the tutorials on MongoDB used <a href=\"https:\/\/jsonlint.com\/?code=\" target=\"_blank\" rel=\"noopener\">JSONLint<\/a> to check that the JSON format was ok.<\/p>\n<p><strong>End thoughts<\/strong><\/p>\n<p>Now that I have python and the database talking I will have to explore how to use both.<\/p>\n<p>I think I will try and explore a Geo example to see how that would work with the NoSQL database.<\/p>\n<p>The other type of NoSQL database I want to try is the Graph type to create relationships. Matt Cantwell had mentioned this type of connectivity when working with <strong><a href=\"http:\/\/www.recollect.co.nz\/\" target=\"_blank\" rel=\"noopener\">Recollect<\/a>.<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I have been interested in NoSQL databases and so I decided to try MongoDB. I always thought it stood for NO SQL but apparently its NOT ONLY SQL. I used this video to get an overview of setup and uses: Installing on VPS. Not a problem. Directly as the video. Installing on my PC, real [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":3366,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[36,39,34],"tags":[],"class_list":["post-3244","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-databases","category-nosql-databases","category-python"],"featured_image_src":"https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp001.jpg","featured_image_src_square":"https:\/\/max-drake.cc\/wp-content\/uploads\/2018\/05\/imp001.jpg","author_info":{"display_name":"Max Drake","author_link":"https:\/\/max-drake.cc\/?author=1"},"_links":{"self":[{"href":"https:\/\/max-drake.cc\/index.php?rest_route=\/wp\/v2\/posts\/3244","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/max-drake.cc\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/max-drake.cc\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/max-drake.cc\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/max-drake.cc\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3244"}],"version-history":[{"count":0,"href":"https:\/\/max-drake.cc\/index.php?rest_route=\/wp\/v2\/posts\/3244\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/max-drake.cc\/index.php?rest_route=\/wp\/v2\/media\/3366"}],"wp:attachment":[{"href":"https:\/\/max-drake.cc\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3244"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/max-drake.cc\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3244"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/max-drake.cc\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3244"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}