Avro向后的模式演进引发ClassCastException异常。

huangapple go评论73阅读模式
英文:

Avro backward schema evolution throws ClassCastException

问题

以下是翻译好的内容:

我在尝试使用一个简单的Java程序测试Avro模式演化时遇到了ClassCastException

Avro版本:1.10.0

customer-v1.avsc

{
  "type": "record",
  "namespace": "com.practice.kafka",
  "name": "Customer",
  "doc": "Customer的Avro模式",
  "fields": [
    {"name": "first_name", "type": "string", "doc": "顾客的名字"},
    {"name": "last_name", "type": "string", "doc": "顾客的姓氏"},
    {"name": "automated_email", "type": "boolean", "default": true, "doc": "是否接收营销邮件"}
  ]
}

customer-v2.avsc

{
  "type": "record",
  "namespace": "com.practice.kafka",
  "name": "CustomerV2",
  "doc": "Customer的Avro模式",
  "fields": [
    {"name": "first_name", "type": "string", "doc": "顾客的名字"},
    {"name": "last_name", "type": "string", "doc": "顾客的姓氏"},
    {"name": "phone_number", "type": ["null","boolean"], "default": null, "doc": "可选的电话号码"},
    {"name": "email", "type": "string", "default": "missing@example.com", "doc": "可选的电子邮件地址"}
  ]
}

序列化v1和反序列化v2的程序

package com.practice.kafka;

import org.apache.avro.file.DataFileReader;
import org.apache.avro.file.DataFileWriter;
import org.apache.avro.io.DatumReader;
import org.apache.avro.io.DatumWriter;
import org.apache.avro.specific.SpecificDatumReader;
import org.apache.avro.specific.SpecificDatumWriter;

import java.io.File;
import java.io.IOException;

public class BackwardSchemaEvolutionSample {

    public static void main(String[] args) {

        // 步骤1 - 创建具体记录
        Customer customer = Customer.newBuilder().setFirstName("John").setLastName("Doe").setAutomatedEmail(false).build();

        // 步骤2 - 将具体记录写入文件
        final DatumWriter<Customer> datumWriter = new SpecificDatumWriter<>();
        try (DataFileWriter<Customer> dataFileWriter = new DataFileWriter<>(datumWriter)) {
            dataFileWriter.create(customer.getSchema(), new File("customer-v1.avro"));
            dataFileWriter.append(customer);
        } catch (IOException e) {
            e.printStackTrace();
        }

        // 步骤3 - 从文件中读取具体记录
        final File file = new File("customer-v1.avro");
        final DatumReader<CustomerV2> datumReader = new SpecificDatumReader<>();
        CustomerV2 customerRecord;
        try (DataFileReader<CustomerV2> dataFileReader = new DataFileReader<>(file, datumReader)) {
            customerRecord = dataFileReader.next();
            System.out.println(customerRecord.toString());
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

结果

Exception in thread "main" java.lang.ClassCastException: 无法将类com.practice.kafka.Customer强制转换为类com.practice.kafka.CustomerV2(com.practice.kafka.Customer和com.practice.kafka.CustomerV2位于'应用程序'的未命名模块中)
at com.practice.kafka.SchemaEvolutionSample.main(SchemaEvolutionSample.java:34)

请问你能告诉我如何修复这个错误吗?

英文:

I am getting a ClassCastException while trying to test Avro schema evolution with a simple Java program.

Avro version: 1.10.0

customer-v1.avsc

{
&quot;type&quot;: &quot;record&quot;,
&quot;namespace&quot;: &quot;com.practice.kafka&quot;,
&quot;name&quot;: &quot;Customer&quot;,
&quot;doc&quot;: &quot;Avro schema for Customer&quot;,
&quot;fields&quot;: [
{&quot;name&quot;:  &quot;first_name&quot;, &quot;type&quot;:  &quot;string&quot;, &quot;doc&quot;: &quot;Customer first name&quot;},
{&quot;name&quot;:  &quot;last_name&quot;, &quot;type&quot;:  &quot;string&quot;, &quot;doc&quot;: &quot;Customer last name&quot;},
{&quot;name&quot;:  &quot;automated_email&quot;, &quot;type&quot;:  &quot;boolean&quot;, &quot;default&quot;: true, &quot;doc&quot;: &quot;Receive marketing emails or not&quot;}
]
}

customer-v2.avsc

{
&quot;type&quot;: &quot;record&quot;,
&quot;namespace&quot;: &quot;com.practice.kafka&quot;,
&quot;name&quot;: &quot;CustomerV2&quot;,
&quot;doc&quot;: &quot;Avro schema for Customer&quot;,
&quot;fields&quot;: [
{&quot;name&quot;:  &quot;first_name&quot;, &quot;type&quot;:  &quot;string&quot;, &quot;doc&quot;: &quot;Customer first name&quot;},
{&quot;name&quot;:  &quot;last_name&quot;, &quot;type&quot;:  &quot;string&quot;, &quot;doc&quot;: &quot;Customer last name&quot;},
{&quot;name&quot;:  &quot;phone_number&quot;, &quot;type&quot;:  [&quot;null&quot;,&quot;boolean&quot;], &quot;default&quot;: null, &quot;doc&quot;: &quot;Optional phone number&quot;},
{&quot;name&quot;:  &quot;email&quot;, &quot;type&quot;:  &quot;string&quot;, &quot;default&quot;:  &quot;missing@example.com&quot;, &quot;doc&quot;:  &quot;Optional email address&quot;}
]
}

Program to serialize v1 and deserialize v2

package com.practice.kafka;
import org.apache.avro.file.DataFileReader;
import org.apache.avro.file.DataFileWriter;
import org.apache.avro.io.DatumReader;
import org.apache.avro.io.DatumWriter;
import org.apache.avro.specific.SpecificDatumReader;
import org.apache.avro.specific.SpecificDatumWriter;
import java.io.File;
import java.io.IOException;
public class BackwardSchemaEvolutionSample {
public static void main(String[] args) {
// Step 1 - Create specific record
Customer customer = Customer.newBuilder().setFirstName(&quot;John&quot;).setLastName(&quot;Doe&quot;).setAutomatedEmail(false).build();
// Step 2 - Write specific record to a file
final DatumWriter&lt;Customer&gt; datumWriter = new SpecificDatumWriter&lt;&gt;();
try (DataFileWriter&lt;Customer&gt; dataFileWriter = new DataFileWriter&lt;&gt;(datumWriter)) {
dataFileWriter.create(customer.getSchema(), new File(&quot;customer-v1.avro&quot;));
dataFileWriter.append(customer);
} catch (IOException e) {
e.printStackTrace();
}
// Step 3 - Read specific record from a file
final File file = new File(&quot;customer-v1.avro&quot;);
final DatumReader&lt;CustomerV2&gt; datumReader = new SpecificDatumReader&lt;&gt;();
CustomerV2 customerRecord;
try (DataFileReader&lt;CustomerV2&gt; dataFileReader = new DataFileReader&lt;&gt;(file, datumReader)) {
customerRecord = dataFileReader.next();
System.out.println(customerRecord.toString());
} catch (IOException e) {
e.printStackTrace();
}
}
}

Result

Exception in thread &quot;main&quot; java.lang.ClassCastException: class com.practice.kafka.Customer cannot be cast to class com.practice.kafka.CustomerV2 (com.practice.kafka.Customer and com.practice.kafka.CustomerV2 are in unnamed module of loader &#39;app&#39;)
at com.practice.kafka.SchemaEvolutionSample.main(SchemaEvolutionSample.java:34)

Can you please let me know how to fix this error?

答案1

得分: 1

你定义了两个数据类型CustomerCustomer2,由于它们不处于继承关系中,所以不能进行任何强制转换。因此,Java无法进行强制转换,你会收到ClassCastException(类转换异常)。
在你的代码中,唯一的解决方案是捕获ClassCastException,并在捕获块中将Customer转换为Customer2

我假设你正在模拟在Kafka环境中对模式所做的更改。在这种情况下,你将通过添加新字段或删除旧字段来扩展现有的Avro模式。

只要类的名称保持不变,Avro模式的更改就会起作用。

英文:

You defined 2 data types Customer and Customer2 and you cannot have any casting since they are not in inherit relation.
So Java can't do casting and you are getting ClassCastException.
In your code only solution is to catch ClassCastException and in catch block convert Customer to Customer2.

I assume that you are emulating changes of your schema in Kafka environment.
In this scenario you will extend existing avro schema by adding new fields, or removing old fields.

As far as the name of class stays the same, avro schema changes will work.

huangapple
  • 本文由 发表于 2020年8月30日 20:47:09
  • 转载请务必保留本文链接:https://go.coder-hub.com/63657580.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定