问题

import org.apache.spark.sql.Dataset;
import org.apache.spark.sql.Row;
import org.apache.spark.sql.SparkSession;
import org.apache.spark.sql.functions;

// Create a Spark session
SparkSession sparkSession = SparkSession.builder()
    .appName("ColumnValuesExample")
    .master("local[*]")  // Use appropriate master URL for your environment
    .getOrCreate();

// Read data from a source and create a Dataset
Dataset<Row> sampleData = sparkSession.read()
    // ... other options
    .option("query", "SELECT COLUMN1, column2 from table1")
    .load();

// Select the desired column and collect values
List<String> columnValuesList = sampleData
    .select("COLUMN1")
    .where(sampleData.col("COLUMN1").isNotNull())
    .as(Encoders.STRING())  // Cast the column to String type
    .collectAsList();

String result = StringUtils.join(columnValuesList, ", ");
// Result will be the desired comma-separated string of values

Please note that you need to import the necessary packages and make sure you have set up your Spark session correctly with the appropriate master URL and other configurations. The key part in achieving your desired result is using the as(Encoders.STRING()) method to cast the column values to strings and then using collectAsList() to gather the values into a list.

Dataset<Row> sampleData = sparkSession.read()
                          .....
                          .option("query", "SELECT COLUMN1, column2 from table1")
                          .load();

List<Row> columnsList = sampleData.select("COLUMN1")
    .where(sampleData.col("COLUMN1").isNotNull()).collectAsList();

String result = StringUtils.join(columnsList, ", ");
// Result I am getting is
[15230321], [15306791], [15325784], [15323326], [15288338], [15322001], [15307950], [15298286], [15327223]
// What i want is":
15230321, 15306791......

答案1

得分: 1

Spark行可以通过编码器转换为字符串：

List<String> result = sampleData.select("COLUMN1").as(Encoders.STRING()).collectAsList();

请注意，由于您要求只返回翻译好的代码部分，我已省略了任何额外的回答或解释。如果您有任何其他需要或问题，欢迎随时提问。

    List<String> result = sampleData.select("COLUMN1").as(Encoders.STRING()).collectAsList();

答案2

得分: 1

我将答案粘贴在Scala中。您可以将其转换为Java，因为有可用的在线工具。

另外，我不会像您指定的方式创建String result，因为那需要创建表格并根据您的过程执行查询，但我直接使用以下方式复制问题变量

import org.apache.spark.sql.Row    
val a = List(Row("123"),Row("222"),Row("333"))

打印a会得到

List([123], [222], [333])

因此，使用简单的映射操作以及mkString方法来展开List

a.map(x => x.mkString(","))

会得到

List(123, 222, 333)

我认为这符合您的期望。如果这解决了您的问题，请告诉我。

import org.apache.spark.sql.Row    
val a = List(Row("123"),Row("222"),Row("333"))

List([123], [222], [333])

 a.map(x => x.mkString(","))

List(123, 222, 333) which I assume is your expectation.