Data Processing & Automation Projects
Data processing projects that cover various analytic tools, machine learning and automation coding
Knime & Power BI analytics project
This project covered the fundamentals on how to filter clean and prepare raw data in KNIME, after which taking that data to intepret and analyze it on Power BI
The project allowed me to experiment on different KNIME applications such as the different nodes and what they do on a Airbnb dataset
Actions that I could perform with the nodes:
-
Setting properties of data
-
Preparing columns for data
-
Changing field data type of datas (e.g user_id to String type)
-
Changing category type to number type
-
Filter missing values to prepare a more accurate dataset
-
Using statistic node to view the raw data in graph format
-
Perform clustering
-
Using denormalizer and numeric binner to create a seperate column based on another column of data range
-
Exporting data as CSV
Power BI was used for the data storyboarding portion where I can display the huge data sets in diagrams to view specific data that I want like:
-
Spread of price tier among airbnbs
-
Average number of commodities in different price tier
-
Average rating of Airbnb reviews
Python Jupiter Notebook, data structure & algortihm project
This project covered how to use python coding to create data algorithm and programs that can be run to solve mathematical problems and other scenarios
There are two different projects in this module. The first module required us to write a program to record and display results of a swimming competition. So, the first project focused more on:
-
Recording input
-
Creating list
-
Using loops
-
Using arrays
-
If else arguments
-
Calling define on variables
-
Recording data on list and printing it
I managed to fufil the requirements of correctly displaying the data based on the required format and gave accurate results too.
Video Presentation of Project #1:
Project #2: Stock Portfolio
The second project was on creating a stock portfolio and being able to input a stock with the amount of units and the price at which it was bought. The program is also required to calculate net profit/net loss.
-
Recording input
-
Creating list
-
Using loops
-
Using arrays
-
Using collections
-
If, else, elif arguments
-
Showing some mathematical expression
-
Reading from txt file
-
Calling define on variables
-
Recording data on list and printing it
This project was a bit more advanced than I initially thought, although it was challenging I still managed to show the key requirements.
Video Presentation of Project #2:
View the full project:
View the full project:
Excel VBA, Macros & UIPath Studio Project
The project is split into part 1 and part 2. So the project is based on automation of student loan application using Excel VBA, Macros & UIPath
Part 1 focuses more on the Excel VBA, I am required to:
-
Implement forms for data entry
-
Perform CRUD functions
-
Perform validation on data fields
-
Include at least one complex function (e.g Save the spreadsheet as a PDF report)
I was pretty satisfied with my project part 1 as I hit all the basic requirements and even included a few other complex functions which is mentioned below.
For part 2, I was tasked to implement productivity improvements with the use of other softwares like UIPath to enhance my part 1's efficiency.
For part 2, I included both Excel Macros an UIPath to improve productivity in my project. I was pretty overall satisfied as I had more than 3 productivity improvements in my project.
Macros
Macros was an important part of this project as I could repetitively perform the same function with a single click of a button. These are some of the macros I included in my project:
-
Filtering out student loan application based on different column values
-
Only filter out approved student loan application on a seperate excel sheet
-
Saving Excel spreadsheet in allocated folders "Approved", "Not Approved" and "Declined" based on the respective matching values
Excel VBA
Using Excel VBA, there are many useful functions that can be implemented to enhance the use of Excel and minimize manual input. Things I have included in my Excel VBA:
-
A student loan application form where users can input particulars and be displayed on a spreadsheet visible only to admins
-
A registry account for users to check their status of their loans
-
Editing and deleting student loan application particulars on an admin panel
-
Graphs representing data on spreadsheet
-
Printing individual loan application particulars
-
Student loan calculator
UIPath Studio
UIPath improves the productivity and efficiency of the student loan application. With UIPath, an Excel file can be brought to the next level in terms of productivity. What I used UIPath Studio for:
-
Sending an initial email to the user whenever the new user submits an application to show confirmation
-
Sending emails at mass, with different messages based on the loan application status where the user particulars will also be reflected
