How I cracked a Data Science Hackathon

You know how competitions can be quite frustrating and strenuous especially when its a week before the deadline. You begin to refresh the page every time, looking at your rank. Over time, I have gotten used to it, and the most interesting part is you get to learn a lot of things aside chasing the prize I gained a lot of knowledge.

Recently, I took part in the zindi competition:

I was 5th and was placed at #5 (out of 164 teams) on the leaderboard.

In this blog post, I will summarise the main ideas of my solution.

The objective of this competition is to create a machine learning model to predict which individuals are most likely to use mobile money and other financial services (savings, credit, and insurance).

This model can help mobile money providers target new clients and markets across Tanzania more effectively, and also help financial services providers cross-sell other financial services (savings, credit, and insurance) to the existing mobile money customer base.

There are 4 approaches for improving your model:

I decided to follow the 1st approach, which is adding more data.

The gains often get smaller the further you go down the list. For example, a new framing of your problem or more data is often going to give you more payoff than tuning the parameters of your best performing algorithm. Not always, but usually.

Tactics:
ˆ Get More Data: Modern nonlinear machine learning techniques like deep learning continue to improve in performance with more data.
ˆ Invent More Data: If you can’t get more data, can you generate new data? Perhaps you can augment or permute existing data or use a probabilistic model to generate new data.
ˆ Clean Your Data: Can you improve the signal in your data? Perhaps there are missing or corrupt observations that can be fixed or removed, or outlier values outside of reasonable ranges that can be fixed or removed in order to lift the quality of your data.
ˆ Transform Your Data: Can you reshape your data distribution? Making input data more Gaussian or passing it through an exponential function may better expose features in the data to a learning algorithm.

Solution 1:

Looking at the data, we have latitude and longitude features so I reverse-geoencoded it with geopy to get the region district and street of each particular ID.

After reverse geoencoding, I had this:-

I was able to get 4 features from reverse encoding which are zone, district, region and street.

Solution 2:

There was a folder provided by zindi called fsdt folder, so I did some engineering to get new longitudes and latitudes. Based on these values, I calculated the Haversine distance.

Since the entire code is too long, I will share a link to it on my github.

Solution 3:

Another technique that worked was using arcgis to enrich my dataset with some other datasets.

I created a video on how to do implement the above:

As I said earlier, a competition is not all about the money, you also get to learn a lot.

How To Get Started: You do not need do everything. You just need one good idea to get a lift in the performance of your model.

Here’s how to handle the problem statement:
1. Pick one group to work on: a) Data b) Algorithms c) Tuning d) Ensembles
2. Pick one method to work on the group.
3. Pick one algorithm to try from the chosen method.
4. Compare the results, retain the solution if there was an improvement.
5. Repeat.

Related posts:

Autonomous vehicles are coming. Drones, cars, trucks and even shipping boats are only some of the autonomous vehicles about to enter the mainstream. Join us in building an open source platform that…

What is it? Who makes it? Who follows when breaks it?. “Standard” is published by Preeti Sharma.

Good morning. Two sips into your first coffee and you sleepily click the link in your calendar, missing twice. Bing. You’re in a meeting with 13 other faces and a couple more who haven’t turned their…