[Solved] Convert Parquet to CSV

How to convert Parquet to CSV from a local file system (e.g. python, some library etc.) but WITHOUT Spark? (trying to find as simple and minimalistic solution as possible because need to automate everything and not much resources).

I tried with e.g. parquet-tools on my Mac but data output did not look correct.

Need to make output so that when data is not present in some columns – CSV will have corresponding NULL (empty column between 2 commas)..


Enquirer: Joe

Solution #1:

You can do this by using the Python packages pandas and pyarrow (pyarrow is an optional dependency of pandas that you need for this feature).

import pandas as pd
df = pd.read_parquet('filename.parquet')

When you need to make modifications to the contents in the file, you can standard pandas operations on df.

Respondent: Uwe L. Korn
The answers/resolutions are collected from stackoverflow, are licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0 .

Leave a Reply

Your email address will not be published.