{"id":6097,"date":"2020-04-02T16:27:50","date_gmt":"2020-04-02T16:27:50","guid":{"rendered":"https:\/\/max-drake.cc\/?p=6097"},"modified":"2020-04-02T16:28:01","modified_gmt":"2020-04-02T16:28:01","slug":"covid-by-region-nz-map-representation-part-5","status":"publish","type":"post","link":"https:\/\/max-drake.cc\/?p=6097","title":{"rendered":"Covid by region NZ -map representation- Part 5"},"content":{"rendered":"\n

The Ministry of Health(MoH) have posted the original dataset on a different web page of each case and which District Health Board (DHB) they are under. The data can be found here.<\/a><\/strong> Instead of by region, its mapped to the District Health Board Regions. So the original map I was using in this post<\/a><\/strong> is not of any use. I have looked to see if I can get DHB boundaries, but so far have only found tiles or images, so no HTML <path> data. <\/p>\n\n\n\n

Point on Map<\/h4>\n\n\n\n

As I cannot get regions, I will need to put pins in a map which can show the data. A bit like what I did on this map with data <\/a><\/strong>which is linked to a database and it then displayed it on a map. I need to have not only a pin but also text as well to show the region.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Currently I’m going to play with the data first in Excel, then look at how I can push that into the MySQL DB on my VPS (virtual private Server), so the Google map can call the data and render it. <\/p>\n\n\n\n

Excel from Web<\/h4>\n\n\n\n

I’m pulling the data into Excel with its Data>from Web tool. The MoH data comes in with the tables. I’m also pulling population data of DHB’s from wiki, and that pulls in the whole page. <\/p>\n\n\n\n

I’m creating different sheets in Excel so that imports are on different sheets (easier to manage uploads and if they break its easier to track back<\/em>). Format of data as below:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

As this information has dates we can look at changes over time, such as rates of change). As the original work I did was in Google Sheets I have to rebuild the information sheets in Excel. <\/p>\n\n\n\n

I found a polygon path in this article<\/a><\/strong> and had to do quite a lot to simplify to basic regions and boundaries. The NZ Herald has a really good presentation and I got the path areas from their map. They have a very elaborate coding for the map seup with all the contours shown, I’m not sure they actually inform but are there to look pretty. The left hand panel is more informative than the MoH web site. <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Reviewing some of the data on the MoH website there is more information on this page,<\/a><\/strong> a bit more generalised but relevant. <\/p>\n\n\n\n

I’ve spent a few hours cleaning the HTML to get clean paths for the regions from the NZ Herald page scrape. So will have to reupdate the Excel file as information updated. I’m spending more time gathering\/organising\/cleaning data rather than analysing it. <\/p>\n\n\n\n

I used an ONLINE HTML prettify<\/a><\/strong> for the strings of HTML that I pasted into notepad++ as some of the strings were very long. Prettifying them sped things up heaps, and also showed how much content was in the file too. <\/p>\n\n\n\n

\"\"
NZ map with District Health board boundaries. Auckland central not too clear, maybe black border lines may bebetter for clarification.<\/figcaption><\/figure>\n\n\n\n

So I now have a template to display data on. I better go and generate some. <\/p>\n\n\n\n

Pivot Tables in Excel<\/h4>\n\n\n\n

After pulling in the data into a table (not an Excel Table format, just rows\/columns) I selected all the table and created a pivot table. Pivot tables are still a mystery for me but it collates things beutifully. I was trying to get a list of all the DHB’s and was trying to filter the original table into giving unique instances, when Pivot table elegantly puashes this data out. <\/p>\n\n\n\n

I have 2 pivot tables below, age range by DHB and Cases by date in DHB. <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The Pivot table below has a graph showing new cases per day, that seem to be reducing from its peak on 25th March. Good to see that trend.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

I’m going to go back to getting information from the website , especially ethnicity and transmission. <\/p>\n\n\n\n

Analysis is basic<\/h4>\n\n\n\n

On reflection this is pretty basic excel analytics, its only taking the data and putting it onto a map that is different. Although simplistic, the filters help quickly reorder the data to see trends, I’m finding it tedious getting deaths\/recovered from the BIng Tracker and Hopkings maps. So although simplistic, it is far quicker to get ordered data ohewr than confirmed cases.<\/p>\n\n\n\n

Excel to create HTML for map<\/h4>\n\n\n\n

As I don’t use Excel that much, its still useful. Also its great for concatenating data for the map so that you can change part of map style quickly by adjusting one part of script.<\/p>\n\n\n\n

I’ve now built the whole map, Regions, Borders, Labels, Dots & lakes (a feature) into Excel so that I can easily chage colours and features. <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n

original map was in red tones, I’ve changed colour to blue as a test. The opacity of regions can be changed, I want to experiment with opacity based on numbers. Excel is good for this as you can just put in a number to change the opacity as a fraction.<\/p>\n\n\n\n

MoH on friday 3rdApril have started to do ArcGis charts as well:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

I’m still playing with layout of Map, happier with position of numbers but colours and transparancy still need some work. I may have to do conditional colouring for different values to have densities the same. It took a lot of mucking around getting text alignment right. I may have to do a range of colours from say blue for low numbers, green for mid numbers and red for high numbers. <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The map above has the regions reversed coloured as I do not have coast outline so light colours fade and you can’t read them Tairawhiti and West Coast,<\/strong> would disappear into background. Maybe that is solution, change background colour (didn’t work). <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

In the end I added 25% of max value to all the values so the lightest hues started at 25% saturation up to 100% with the formula below so formula is = (25% highest value + Lookup DHB BValue)\/Highest value so for map below it = (33 +1)\/132 (for Tairawhiti, where lowest at 1), saturation defines region better<\/em>. Excel formula:<\/p>\n\n\n\n

=(ROUND(MAX($J$38:$J$56)0.25,0)+VLOOKUP(A64,$I$38:$J$56,2))\/MAX($J$38:$J$56)<\/em><\/strong><\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Loading data is getting easier because MoH are doing a better job processing tables so easier to find a specific table and pull that data rather than the complete list of confirmed and probables. <\/p>\n\n\n\n

Spending a lot of time with Maps, learning about paths, excel structure for map data and different components. I’m still struggling with containers and I’d like to get Hover pop up windows to be able to add another layer of information to the map. <\/p>\n\n\n\n

the stuff map<\/a><\/strong> does this well. but overall a boring map.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

This one actually calls data from a database to an xml file which then displays it. <\/p>\n\n\n\n

End comment<\/h3>\n\n\n\n

It has been quite time consuming getting basic map setup and extracting data and HTML structure from Excel for building and adjusting the map. I’m happy with its current iteration but want to go 2 steps further and get pop-up box on mouse hover and also get containers for web page. <\/p>\n\n\n\n

I’m wondering if I should leave the map and move onto building a dashboard with something like Google Data Studio or PowerBI. Google Data Studio will need to have internet access so could be slower. Maybe PowerBI desktop would be the better way to go as my data is residing in Excel. <\/p>\n\n\n\n\n\n\n\n\n<\/head>\n\n