Kepler.gl and London Crimes Geo data & GitHub Gist, Hub, Hooks, github additions

I saw this video on Kepler & React and was excited about the Kepler app. I want to try it with NZ covid data that I’m getting from some API endpoints. But its not structured in a GeoJson format. This is a tool that Uber have developed and shared, which is nice of them.

I’m using notepad++ & online json to CSV to order the data. One file has all the httml <paths> for regions in NZ as well as lat/long points, another file has regional numbers by timeline, so I need to get the Lat/long onto each timeline record for each region.

After playing around with the data it wasn’t too impressive visually on the map, so I went on and used the UK Police crimes API which I converted to CSV and got the map below. I saved it as a HTML file that you can view in your browser and can alsoactivate link by by clicking on image below (I haven’t used that feature before):

Its fun to explore, I played with it for at least half a day, in fact most of the time was doing data manipulation and cleaning for inputting information into the online tool.

The api I used for the above data was: where you put lat/long in and also month. I haven’t looked up the docs for that API but may explore it later, it is fun to use.

https://data.police.uk/api/crimes-street/all-crime?lat=51.509865&lng=-0.118092&date=2020-03

API docs can be found here. You can also download CSV and that could have been a better option as I converted the JSON output to CSV because kepler.gl wanted a geoJSON file and in the JSON file the lat/long were blended in a single column so hard to differentiate for the map.

Add data to the map- Using GIST

The first I’d heard about Gists and was in the video above, and I thought this was a handy way to get an endpoint JSON file for an API.

So I thought I’d try it out with Keplr.gl. The first time I tried to add an API URL to kepler.gl ity wanted the file extension and the api I was using didn’t need one, so I thought I’d try using GIST to make that endpoint by copying/pasting the data , showing it RAW and getting the link.

I had the file in raw, but it didn’t load into Kepler.gl because I’d made it a private repository instead of a public one, so I had to go back and delete the existing file (I couldn’t find setting where to change from private to public) and re-do the whole exercise.

Kepler.gl puilled it in fine after I made it a public GIST, so that was OK.

Link to creation of gist here. I think its a single file share, and works as an endpoiint for me.

Just on that, GitLab has CI/CL continuous integration/Cont .. . I may have to explore that later too there is a free tier to the services.

Other Git services

I came across Git marketplace and put a query in YT for indfromation on best add-ins and came up with this video:

Hub for git

This is a method of creating GitHub repositories on Github from inside your terminal in a project in say Code.

>hub create

>hub browse

Its a bit temperamental loading into system to begin with, as you need admin permisions and you need Chocolaty (Python) to install it) but onece in its good to use.

The issues with using terminal is you have to remember all the commands, buit that will come with practice

Git Hooks

These are interesting. They are pre-checking testing code that you can run when you commit so that you can pre check that code is structured right. Things like Prettify on specific (eg *.js or *.html) files or pre-upload tests (pre-written).

If you go into your project that has has a git init on it , there is a .git folder and in that is a hook folder with some samples.

This can be used to do a bit of a QA on your files pre-upload. Quite a good idea.

Git GUI clients

There are a few Git GUI clients that can manage your gitting. Some free, some paid. I did have to resort to Git Desktop for a while until I got the SSH figured out, but then its just another programme that you need to manage and start up. I’m sort of used to command line now, still have to think about some of the commands but overall its quick. So for myself, although they are attractive, I wont be going down that path.

BSide GitHub CMS

This looks interesting, I may have to explore later. Its free, you can download templates.

BSide is lightweight content management system for Github. It makes it easy to edit posts based on the markdown, codes in your repositories. It supports many browsers on mobiles and desktops.

from repository

Commitizen

Commitizen. This is a commit tool that lets you have a structured method with lists about your commits , such as fix, code update, bug, docs, other.

It is a CLI and needs a template that lists the features that you are doing, so you choose:

commitizen init cz-conventional-changelog --save-dev --save-exact

Although its installed globally, you have to initialise for each project ewith the “conventional” template (or another that worksfor you).

It takes a bit longer, but at least commits are more structured and less arbetrary (which mine are to date).

End comment

Its fun to use and explore, just at the moment I’m interested in the whole React/node JS environment so I was not really focussed on utilising it and exploring it too deeply.

I did not find it that intuitive, there are layers that you can drill down to reach but I was finding it difficult getting those to reveal their settings. I was only doing an overview explore and I liked what I have done.

I need to think about it a bit more and find some good data to use and explore with it. Another time, I’m already extended in the number of directions I’m exploring at the moment, so this is for annother day. Worth looking at.