April 17, 2018

Python 1. Starting with Python

By  Python  0 Comments

Why would I want to use Python?

I give an explanation at the bottom of background to how I selected this programme.

 After exploring Python tutorials, I decided that I could use Python to do help me do the following things that I have been working on, hopefully in a more integrated environment, rather than using lots of different tools:

  1. Link to Databases, initially to extract data, maybe later to alter data (instead of Knime)[See Python 5]
  2. Do data management and data wrangling with large amounts of data and store it in different forms such as CSV, XML and JSON (instead of Excel/Knime/Power BI on large data sets) [ See Python 2 ]
  3. Develop Static & Dynamic Maps & graphs for posting to web (as an alternative to Google)[ See Python 2 ]
  4. Scrape URL’s and Web pages for data both in text and tabular forms and save to file or connect to live (To capture data from Web, like Power BI & Excel (I have yet to do much of this))[ See Python 3 ]
  5. Build Dashboards (like Tableau Public & Power BI, but control the data rather than uploading it to others galleries)
  6. Post the data to web pages (like Excel & Power BI & Google Maps)
  7. Compile .exe files to run tools (to share with others, similar to Excel VBA tools)[ See Python 4]
  8. For using with Dynamo for Revit (Iron Python)

So it can supplement work I am doing with Excel, Knime, Power BI and other Data Analytic tools. I can also build crude tools of my own using its compiling process. Also , for mapping I can use its HTML file export process to be able to display information on web pages.

Python does have packages (Flask, Django) for web development, but I am happy with my WordPress websites for the moment. No need to revisit that at the moment.

 

Python Install

You can go here to get Python to install:

https://www.python.org/downloads/

For windows choose the 32 or 64 bit version.  See this article about installing it:

https://www.howtogeek.com/197947/how-to-install-python-on-windows/

Also  when you install you need to go carefully because if the install process offers you the choice of installing its “Path” do so. Otherwise you will need to do it in the control Panel>System>Advanced System Settings> Environment Variables> System Variables> Path  and add the directory to Python, which is:

C:\Python36\             (if you have version 3.6.5  or something like that)

C:\Python36\ Scripts\           

 

To test the install open the command prompt and type:

C:\> python (return)

If you get something similar to the following below:

Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 17:00:18) [MSC v.1900 64 bit (AMD64)] on win32

Type “help”, “copyright”, “credits” or “license” for more information.

>>> 

Then you have loaded it correctly.

Note you have the 3 chevrons, this means you are in the python console wherte you can type python code.

So , if you type

>>> print(‘Hello World’) (return)

It should show

>>> Hello World

To get back to the C:\> prompt type

>>> quit() (return)

C:\>

If you don’t get the above on typing python then your “Path” in the system settings may not be set up correctly.

Also note that you will not find a directory :

C:\Python36 in C:\ drive, it is in fact in quite a different place, on my computer it is:

C:\Users\blurgleBlurgle\AppData\Local\Programs\Python\Python36

So you may want to put a shortcut to it in your file explorer to get to it easily.

 

Here is a bit of  python code I have been testing:

I have created a file called Text.py  (python file extension is .py)

import urllib.request
from bs4 import BeautifulSoup

#specifytheurl
wiki=“https://en.wikipedia.org/wiki/List_of_state_and_union_territory_capitals_in_India”
#Querythewebsiteandreturnthehtmltothevariable’page’
page=urllib.request.urlopen(wiki)
#Parsethehtmlinthe’page’variable,andstoreitinBeautifulSoupformat
soup=BeautifulSoup(page,“html.parser”)
print(soup.body.table)

 

Python uses packages that it calls at the beginning to do specialist processes. There is a difference between modules and packages. See this post for the explanation. A module can be a .py file that contains code that does something. A package can be a combination of modules. See the next paragraph for some packages.

For maths and numbers, a package  called numpy, for graphs and maps matplotlib for tables and matrices pandas, for connecting to url’s urllib  etc .

Python has a process for downloading these, similar to Linux, and it installs them in the directory

C:\Users\ blurgleBlurgle \AppData\Local\Programs\Python\Python36\Lib\site-packages\

So when you use the:

import urllib.request

it will go to this directory to see if you have this package installed. If you do not then you go to your command prompt window and:

C:\> pip install numpy (return)

Requirement already satisfied: numpy in

c:\users\blurgleBlurgle\appdata\local\programs\python\python36\lib\site-packages

In the above case, I already have it installed. If it wasn’t you would see that it either wrote a few lines of code, or told you it couldn’t install it. If it couldn’t install it then you may have to put the full path to pip which is in the directory below:

C:\Users\blurgleBlurgle\AppData\Local\Programs\Python\Python36\Scripts

One way to test if the module (package?) was installed directly is to check to see if you can access the help file of the module :

C:\> python (return)

>>> help(‘numpy’) (return)

If you get a stream of text

Help on package numpy:

NAME

    numpy

DESCRIPTION

    NumPy

   =====

    Provides

  1. An array object of arbitrary homogeneous items
  2. Fast mathematical operations over arrays
  3. Linear Algebra, Fourier Transforms, Random Number Generation

      >>> import numpy as np

— More  —

 

Then the module has been installed correctly.

With the more- the last line above, if you press (return) it will carry on showing you more, to eexit from the help type q (return) and that will bring you back to:

>>> 

So, now you have the programme installed and running.

A couple of things to note.

There seems to have been a lot of development in the Python 2.7 version and a lot of modules/packages are designed for the version 2 code. Since I have installed the Python 3.6.5 version some of the packages related to version 2 do not work.  In the pip install you get the versions appropriate to your version. So that is good.

The issue with the 2 different versions is that a lot of the examples on the web relate to the version 2, and these need to be altered before they’ll run in Python 3. I have found quite a few examples that I cannot get running because of the version 2/3 differences.

Python 2 print syntax

>>> print “Hello, World!”

          >>>Hello, World!

Python 3 print syntax

>>> print(“Hello World!”)

         >>>Hello World! 

As a beginner coder I look for something similar to what I want to do and adapt it so I will be learning lots of version 2/3 issues.

 Python IDE (Integrated development environment)

Here is a list of possible IDE’s for Python:

https://www.dunebook.com/best-python-ide-windows-mac/

In the Derek Banas he used PyCharm and as that was the first video I watched, I downloaded and used that. The community version is OK.

I have since tried loading other free IDE’s namely ERIC, SPYDER ( Both these 2 not easy to install so I didn’t),  Visual Studio (I’ve used previously for C# but not intuitive and annoying trying to find things, in my op no intelli-sense so I uninstalled) & Wing (A pain even trying to get licence for free version so uninstalled straight away, if they make it that hard to set-up then I’m definitely not interested) and so am back to PyCharm.

https://www.jetbrains.com/pycharm/download/#section=windows (download community version-FREE)

Also watch video for setup (mainly paths again) :

After you’ve setup your IDE you are ready to test some code

Background

If you browse for Data Analytics “R” and “Python” come up a lot.

I have had a previous tinker with both. I think I got a little further with “R” than I did with “Python”.

I think, on re-installing Python, my problem was that on the install I did not also include the “PATH” to the programme so had difficulties getting it to do what it was supposed to.  This time I am a lot more aware about ticking the “ADD PATH” tick box on the install.

I have been aware that I would need to have another look at these programmes but could not see any reason to choose either. Then I went to a talk on Revit “Dynamo” that uses “Iron Python” for some extra coding for its processes. That made my mind up to look at Python a bit more closely.

After teaching people CAD I have noted the people who have a good reason to learn the tool usually succeed, whereas those who think it will be “of interest” usually do not get too far. Personally, I need a motivation to learn something new. If my current tools meet my needs I will suffer imperfections to a point as long as my goal for using them is met. If a better tool comes along that also meets those needs I will give it a try. If the learning process is not too steep or the benefits outweigh the challenges, I will move towards the new tool. Changing from Acad to Revit is an example of the move, although the learning curve for that was steep. It was more of the unlearning than learning in that particular case.

In fact, there are still things that Cad can do better than some of the BIM tools. One of them being developing bespoke commands/tools. In Revit far more difficult than Cad and you are initially tied into the OOTB tool for a while or compiled Add-ins from the Store. Which brings me back to “Dynamo” for developing tools to work within Revit.

I have recently been playing in Excel and in Power BI. In Excel I am playing with 50-80 Mb files which make it run very slow, but Excel is a good tool. In Power BI I have been exploring Dashboards and playing with its data wrangling features that are pretty good. I am also interested in Maps and displaying dynamic data on Maps.  On watching some of the Python videos it seems Python can do a lot of these things.

The YouTube video series I have been watching :

I love his stuff, especially his overview of a programme in one video.

Sentdex  https://pythonprogramming.net

I found this series of videos very good for:

Maybe a bit too technical but a good overview of some of its Data management & visualisation applications, but well taught and insightful. He explains a lot as he goes. 

Some other resources

A few other resources I have comer across to help with Python.

End thoughts

My objectives are to write some posts on some of the learning developments that IO mentioned in the beginning. We’ll see how that goes.

I have done some initial testing won maps with Folium, also a bit of play with the basics of Python. I am currently trying to do some web scraping with Urllib and BeautifulSoup4

Post Views: 118

Related Posts

Add a Comment