5. JDBC ResultSet To Parquet File Writer

JDBCParquetWriter is a Java Library for writing Apache Parquet Files from JDBC Tables or ResultSets. It uses Apache Hadoop and Parquet to translate the JDBC rows into the column based format. The Parquet File can be imported into Column based Analytics Databases such as ClickHouse or DuckDB.

Latest stable release: JDBCParquetWriter-1.2.1.jar

Development version: JDBCParquetWriter-1.3.0-SNAPSHOT.jar

Sample SQL Statement

String tableName = "execution_ref";
File file = File.createTempFile(tableName, ".parquet");

String sqlStr = "SELECT * FROM " + tableName;
try (Statement st = conn.createStatement(); ResultSet rs = st.executeQuery(sqlStr);) {
    JDBCParquetWriter.write(file, tableName, rs);
}

Features

• Table Schema derived from JDBC ResultSetMetaData

• Support for Annotated Types:

  • Date

  • Time

  • Timestamp

  • BigDecimal

  • Decimal and Numeric, considering the Scale

• Support for Nullable

• Compression Support

• Schema or Tables Bulk Export in parallel