How to combine multiple CSV files with 8 lines of code

How to combine multiple CSV files with 8 lines of codeEkapope ViriyakovithyaBlockedUnblockFollowFollowingFeb 1creditWhy do you need this?Manually copy-pasting is fine if you don’t have too many files to work with.

But imagine if you have 100+ files to concatenate — are you willing to do it manually?.Doing this repetitively is tedious and error-prone.

If all the files have the same table structure (same headers & number of columns), let this tiny Python script do the work.

Step 1: Import packages and set the working directoryChange “/mydir” to your desired working directory.

Step 2: Use glob to match the pattern ‘csv’Match the pattern (‘csv’) and save the list of file names in the ‘all_filenames’ variable.

You can check out this link to learn more about regular expression matching.

Step 3: Combine all files in the list and export as CSVUse pandas to concatenate all files in the list and export as CSV.

The output file is named “combined_csv.

csv” located in your working directory.

encoding = ‘utf-8-sig’ is added to overcome the issue when exporting ‘Non-English’ languages.

And…it’s done!This article was inspired by my actual everyday problem, and the coding structure is from a discussion on stackoverflow.

The completed script for this how-to is documented on GitHub.

Thank you for reading.

Please give it a try, have fun and let me know your feedback!If you like what I did, consider following me on GitHub, Medium, and Twitter.

Make sure to star it on GitHub :P.

. More details

Leave a Reply