Home Remodeling Analysis Turned Excel Data Handling in PythonWhy cleaning data is the most important stepManjula MishraBlockedUnblockFollowFollowingDec 14Original Project Mission: Find interesting insights to see where the remodeling market is headedProject Mission (Twist): How to handle well manicured excel data in Python (spoiler: neat is a deceptive word)Timeline: One week (I tell you, it’s not enough!)Project Findings for the Original Goal :The numbers look stable with some reshuffle in prioritiesAverage costs are considerably higher than national average in/around Palo Alto, CAStates where the biggest Full Services remodeling companies are locatedFinding on Data Handling: There is a solution in Pandas (Python), life is not hopeless yet!Data: Exploring publicly available data from different sources — Joint Center for Housing Studies at Harvard, Remodeling magazine, and obtaining average project costs for local area in and around Palo Alto, CA from Hinkle Construction Inc’s web page.Method: Python — data wrangling, basic statistics and visualization using mainly Pandas, Matplotlib, Seaborn.The sequence for this post will be to first talk about the results we got from the home remodeling data..Later, we will go through the data cleaning struggles as well.The graphs below show national average spending on different categories of home improvements..The data shows that ‘Room Additions’ is the most popular category..In last 15 years, the spending in this category peaked in 2005 at ~$25k in average expenditure..The lowest point was in 2011 with spending going down to below $12k..The market seems to be recovering since then..From 1995–2005, homeowners spending on room additions increased by more than 62% in 20 years..Another relatively bigger change is in kitchen remodeling where the expenditure doubled in from 1995 to 2015 and peaking in 2007.Source: Joint Center for Housing Studies at HarvardWhen I got the above results, I was curious to know the average homeowner expenditures in my local area where I live..Given the time constrain, I couldn’t find the data directly comparable to the graph above but the average of the those projects in our area.The bar graph below depicts the average costs of different types of home improvements in and around Palo Alto, California..The costs are significantly higher than the national averages..Adding a new room, 144 square feet in size, costs more than $50k..Major kitchen remodeling is also equally expensive at $50k.Source: Hinkle Construction Inc’ websiteFor the first two plots, please check the attached python file, if interested.I was compelled to find out who were the biggest players in the remodeling industry.. More details